Changeset 66518

Timestamp:

Oct 16, 2015, 12:14:06 PM (9 years ago)

Author:

wenzeslaus

Message:

get the page description for addons index also from meta comment and text

When the page does not have the standard format or content, try to
use meta in HTML comments and when this fails try to extract first
sentece from the text.

Also don't search the description line in the whole file when
Keywords section is missing.

Location:

grass-addons/tools/addons

Files:

• get_page_description.py (modified) (2 diffs)
• test/data/g.broken.example.html (modified) (1 diff)
• test/data/g.no.keywords.html (copied) (copied from grass-addons/tools/addons/test/data/g.broken.example.html ) (1 diff)
• test/data/wxGUI.example.html (modified) (1 diff)
• test/test_description_extraction.sh (modified) (1 diff)

grass-addons/tools/addons/get_page_description.py

@@ -24 +24 @@
 
 
+def get_desc_from_comment_meta_line(text):
+    """
+    >>> get_desc_from_comment_meta_line("<!-- meta page description: Abc abc-->")
+    'Abc abc'
+    """
+    text = text.split("<!-- meta page description:", 1)[1]
+    text = text.split("-->", 1)[0]
+    return text.strip()
+
+
+def get_desc_from_desc_text(text):
+    r"""Get description defined as first sentence in the given text.
+
+    Sentence is defined as text which ends with dot and space.
+    The string is expected to contain this. The other case not handled.
+
+    >>> get_desc_from_desc_text("Abc abc.abc abc.")
+    'Abc abc.abc abc.'
+    >>> get_desc_from_desc_text("Abc abc.abc abc. ")
+    'Abc abc.abc abc.'
+    >>> get_desc_from_desc_text("Abc abc.abc\n abc.\n")
+    'Abc abc.abc\n abc.'
+    """
+    # this matches the sentence but gives also whole string even if it
+    # is not the sentence
+    text = re.split(r"\.(\s|$)", text, 1)[0]
+    # strip spaces at the beginning and add the tripped dot back
+    return text.lstrip() + '.'
+
+
 def main(filename):
     with open(filename) as page_file:
         desc = None
         in_desc_block = False
+        in_desc_section = False
+        desc_section = ''
+        desc_section_num_lines = 0
         desc_block_start = re.compile(r'NAME')
-        desc_block_end = re.compile(r'KEYWORDS')
+        # the incomplete manual pages have NAME followed by DESCRIPTION
+        desc_block_end = re.compile(r'KEYWORDS|DESCRIPTION')
+        desc_section_start = re.compile(r'DESCRIPTION')
         desc_line = re.compile(r' - ')
+        comment_meta_desc_line = re.compile(r'<!-- meta page description:.*-->')
         for line in page_file:
             line = line.rstrip()  # remove '\n' at end of line
@@ -40 +76 @@
                 if desc_line.search(line):
                     desc = get_desc_from_manual_page_line(line)
+            # if there was nothing in the generated section of the page
+            # try find manually added meta comments which are placed
+            # at the beginning of the manually edited part of the page
+            if not desc and comment_meta_desc_line.search(line):
+                desc = get_desc_from_comment_meta_line(line)
+            # if there was nothing else, last thing to try is get the first
+            # sentence from the description section (which is also last
+            # item in the file from all things we are trying
+            if in_desc_section:
+                desc_section += line + "\n"
+                desc_section_num_lines += 1
+                if desc_section_num_lines > 4:
+                    in_desc_section = False
+            if not desc and desc_section_start.search(line):
+                in_desc_section = True
+        if not desc and desc_section:
+            desc = get_desc_from_desc_text(desc_section)
         if not desc:
             desc = "(incomplete manual page, please fix)"

grass-addons/tools/addons/test/data/g.broken.example.html

@@ -14 +14 @@
 <h2>DESCRIPTION</h2>
 
-This is a test page which should be emulate a broken manual page.
+This is a test page which should emulate a broken manual page.
 This can happen for example, when module cannot generate a proper
 description (broken imports, not using parser, etc.).
-
 
 <h2>SEE ALSO</h2>

grass-addons/tools/addons/test/data/g.no.keywords.html

@@ -10 +10 @@
 
 <h2>NAME</h2>
-<em><b>r.broken.example</b></em> <h2>KEYWORDS</h2>
+<em><b>r.broken.example</b></em>
 
 <h2>DESCRIPTION</h2>
 
-This is a test page which should be emulate a broken manual page.
+This is a test page which should emulate a broken manual page
+without keywords section.
 This can happen for example, when module cannot generate a proper
 description (broken imports, not using parser, etc.).
 
+ - This line is supposed to look like description line but it is in a wrong place.
 
 <h2>SEE ALSO</h2>

grass-addons/tools/addons/test/data/wxGUI.example.html

@@ -15 +15 @@
 <h2>DESCRIPTION</h2>
 
-This is a test page which should be similar to a wxGUI manual pages.
+This is a test page which should be similar to wxGUI manual pages.
 
 

grass-addons/tools/addons/test/test_description_extraction.sh

@@ -9 +9 @@
 ../get_page_description.py data/wxGUI.example.html
 ../get_page_description.py data/g.broken.example.html
+../get_page_description.py data/g.no.keywords.html