From b15e189f5e34e7c63bab60e5e29fdd3a2f6d43f2 Mon Sep 17 00:00:00 2001
From: gnosygnu <gnosygnu@gmail.com>
Date: Mon, 1 Feb 2016 00:13:42 -0500
Subject: [PATCH] v3.1.5.1

---
 blog.html     | 114 +++++++++++++++++++++++++++++++++++++++++++++++++-
 download.html |  91 ++++++++++++----------------------------
 2 files changed, 140 insertions(+), 65 deletions(-)

diff --git a/blog.html b/blog.html
index 682dabb46..668902c18 100644
--- a/blog.html
+++ b/blog.html
@@ -17,6 +17,118 @@
     <div id="mw-content-text" lang="en" dir="ltr" class="mw-content-ltr">
 
 <!-- page_bgn -->
+    <h2>
+      <span class="mw-headline" id="Release:_v3.1.5.1_.282016-01-31_22:00_Sun.29">Release: v3.1.5.1 (2016-01-31 22:00 Sun)</span>
+    </h2>
+    <p>
+      The desktop app is a major release. It resolves reference errors in 2016-01 English Wikipedia, fixes wide images on German Wikipedia, and adds a new PageBanner extension
+    </p>
+    <p>
+      The Android app is also a major release. It shows more tables, heiroglyphics, and images with a ? in title.
+    </p>
+    <h3>
+      <span class="mw-headline" id="Luaj_fixes_for_references_in_2016-01-13_English_Wikipedia">Luaj fixes for references in 2016-01-13 English Wikipedia</span>
+    </h3>
+    <p>
+      The 2016-01-13 dump had many erros in the reference section due to bugs in Luaj. One of the following would be displayed:
+    </p>
+    <ul>
+      <li>
+        Check |archiveurl= value (help)
+      </li>
+      <li>
+        Specials character in |{some word}
+      </li>
+    </ul>
+    <p>
+      These were bugs in Luaj with the frontier pattern and multi-byte chars. They were both resolved in this release.
+    </p>
+    <h3>
+      <span class="mw-headline" id="Fix_for_German_Wikipedia_and_extremely_wide_pie_charts">Fix for German Wikipedia and extremely wide pie charts</span>
+    </h3>
+    <p>
+      I'm extremely thankful for this report at <a href="https://github.com/gnosygnu/xowa/issues/38" rel="nofollow" class="external free">https://github.com/gnosygnu/xowa/issues/38</a> There were HTML issues in German Wikipedia because all the wikis rely implicitly on American formatting rules for decimals. For example:
+    </p>
+    <ul>
+      <li>
+        The German Wikipedia page for Hausmannstätten specifies a pie chart with a width (Breite) of 0.9 . See: <a href="https://de.wikipedia.org/wiki/Hausmannst%C3%A4tten?title=Hausmannst%C3%A4tten&amp;action=edit">https://de.wikipedia.org/w/index.php?title=Hausmannst%C3%A4tten&amp;action=edit</a>
+      </li>
+      <li>
+        This value gets multiplied by various numbers in the template. For example: <code>width:{{#expr:18*{{{Breite}}} }}em;</code>. See: <a href="https://de.wikipedia.org/wiki/Vorlage:Sitzverteilung/?title=Vorlage:Sitzverteilung/&amp;action=edit">https://de.wikipedia.org/w/index.php?title=Vorlage:Sitzverteilung/&amp;action=edit</a>
+      </li>
+    </ul>
+    <p>
+      XOWA was doing this multiplication using the user's locale.
+    </p>
+    <ul>
+      <li>
+        For US users (like me), <code>18*.9</code> was calculated correctly as <code>16.2</code>
+      </li>
+      <li>
+        For German users though, <code>18*.9</code> was calculated incorrectly as <code>162</code>. 
+        <ul>
+          <li>
+            This was because Java was interpreting the <code>.9</code> using German number-format rules. Since "." is the thousands separator, and "," is the decimal separator, <code>.9</code> became <code>9</code>. For US users, imagine <code>,9</code> becoming <code>9</code>.
+          </li>
+        </ul>
+      </li>
+    </ul>
+    <p>
+      v3.1.5 fixes this issue by asserting that all decimal-parsing is done in the US locale.
+    </p>
+    <h3>
+      <span class="mw-headline" id="New_PageBanner_extension_for_Wikivoyage">New PageBanner extension for Wikivoyage</span>
+    </h3>
+    <p>
+      This was an extension introduced by Wikimedia late last year to standardize the display of the image, icons and table of contents at the top of the page. This involved some considerable work on my side (including a Mustache parser), but XOWA now reproduces the same look.
+    </p>
+    <h3>
+      <span class="mw-headline" id=".28Android.29_Show_navboxes_and_other_tables">(Android) Show navboxes and other tables</span>
+    </h3>
+    <p>
+      The Wikipedia Mobile app deliberately hides navboxes as well as other tables. I think this is done for the following reasons:
+    </p>
+    <ul>
+      <li>
+        These tables were meant for desktop display, and often had wide widths
+      </li>
+      <li>
+        These tables clutter the page.
+      </li>
+    </ul>
+    <p>
+      In contrast, I really do like showing as much content as possible, so I decided to show these tables. I'll probably put an option to disable this later, but I'm going to wait first for some feedback from others
+    </p>
+    <h3>
+      <span class="mw-headline" id=".28Android.29_Always_center_table_content">(Android) Always center table content</span>
+    </h3>
+    <p>
+      This is another deviation from the Wikipedia Mobile app style. Basically most tables are left-aligned and would end before reaching the right-hand side of the page. I changed the CSS to have these tables fill up the entire page and center the content.
+    </p>
+    <p>
+      This has one unfortunate sideback in that it breaks the "Collapse" buttons. I'll try to fix this in a future release, but if I can't, I may remove them entirely.
+    </p>
+    <h3>
+      <span class="mw-headline" id=".28Android.29_Show_hieroglyphics">(Android) Show hieroglyphics</span>
+    </h3>
+    <p>
+      The early versions of the XOWA Android app did not show hieroglyphics. This is now supported in this build
+    </p>
+    <h3>
+      <span class="mw-headline" id=".28Android.29_Show_images_with_.3F_in_file_title">(Android) Show images with ? in file title</span>
+    </h3>
+    <p>
+      This was a bug wherein images that had a ? were not being displayed within the Android WebView. I fixed this by replacing ? with %3F.
+    </p>
+    <h3>
+      <span class="mw-headline" id="Next_release:_v3.2.1">Next release: v3.2.1</span>
+    </h3>
+    <p>
+      I ran into several issues with the 2016-01 English Wikipedia dump that set me back a few days. I'm planning to upload English Wikipedia images and HTML this week, and then continue on with the other English wikis
+    </p>
+    <p>
+      For code, I'm going to tackle performance on the Android search again as well as a few minor UI issues on Android (collapse-table, pinch-zoom)
+    </p>
     <h2>
       <span class="mw-headline" id="Release:_v3.1.4.2_.282016-01-28_11:00_Thu.29">Release: v3.1.4.2 (2016-01-28 11:00 Thu)</span>
     </h2>
@@ -143,7 +255,7 @@
       The Android app is a major release. It has a Random feature, shows more images, and adds CSS-tweaks.
     </p>
     <h3>
-      <span class="mw-headline" id="Documentation_for_html-dump_script._See_Help:Import.2FCommand-line.2FDumps">Documentation for html-dump script. See <a href="/wiki/Help:Import/Command-line/Dumps" id="xolnki_11" title="Help:Import/Command-line/Dumps">Help:Import/Command-line/Dumps</a></span>
+      <span class="mw-headline" id="Documentation_for_html-dump_script._See_Help:Import.2FCommand-line.2FDumps">Documentation for html-dump script. See [[Help:Import/Command-line/Dumps]]
     </h3>
     <p>
       This item is self-explanatory. The XOWA Android app is getting more stable, so I felt it would be time to document the generation of the HTML databases.
diff --git a/download.html b/download.html
index ec0cfb961..d54af9392 100644
--- a/download.html
+++ b/download.html
@@ -21,37 +21,37 @@
 
 <h2><span class='mw-headline' id='Windows'> Windows </span></h2>
 <ul>
-  <li> <a href="https://github.com/gnosygnu/xowa/releases/download/v3.1.4.2/xowa_app_windows_64_v3.1.4.2.zip" class="external text" rel="nofollow">Windows (64-bit)</a>
+  <li> <a href="https://github.com/gnosygnu/xowa/releases/download/v3.1.5.1/xowa_app_windows_64_v3.1.5.1.zip" class="external text" rel="nofollow">Windows (64-bit)</a>
   </li>
-  <li> <a href="https://github.com/gnosygnu/xowa/releases/download/v3.1.4.2/xowa_app_windows_v3.1.4.2.zip" class="external text" rel="nofollow">Windows (32-bit)</a>
+  <li> <a href="https://github.com/gnosygnu/xowa/releases/download/v3.1.5.1/xowa_app_windows_v3.1.5.1.zip" class="external text" rel="nofollow">Windows (32-bit)</a>
   </li>
 </ul>
 
 <h2><span class='mw-headline' id='Linux'> Linux </span></h2>
 <ul>
-  <li> <a href="https://github.com/gnosygnu/xowa/releases/download/v3.1.4.2/xowa_app_linux_64_v3.1.4.2.zip" class="external text" rel="nofollow">Linux (64-bit)</a>
+  <li> <a href="https://github.com/gnosygnu/xowa/releases/download/v3.1.5.1/xowa_app_linux_64_v3.1.5.1.zip" class="external text" rel="nofollow">Linux (64-bit)</a>
   </li>
-  <li> <a href="https://github.com/gnosygnu/xowa/releases/download/v3.1.4.2/xowa_app_linux_v3.1.4.2.zip" class="external text" rel="nofollow">Linux (32-bit)</a>
+  <li> <a href="https://github.com/gnosygnu/xowa/releases/download/v3.1.5.1/xowa_app_linux_v3.1.5.1.zip" class="external text" rel="nofollow">Linux (32-bit)</a>
   </li>
 </ul>
 
 <h2><span class='mw-headline' id='Mac_OS_X'> Mac OS X </span></h2>
 <ul>
-  <li> <a href="https://github.com/gnosygnu/xowa/releases/download/v3.1.4.2/xowa_app_macosx_64_v3.1.4.2.zip" class="external text" rel="nofollow">Mac OS X (64-bit)</a>
+  <li> <a href="https://github.com/gnosygnu/xowa/releases/download/v3.1.5.1/xowa_app_macosx_64_v3.1.5.1.zip" class="external text" rel="nofollow">Mac OS X (64-bit)</a>
   </li>
-  <li> <a href="https://github.com/gnosygnu/xowa/releases/download/v3.1.4.2/xowa_app_macosx_v3.1.4.2.zip" class="external text" rel="nofollow">Mac OS X (32-bit)</a>
+  <li> <a href="https://github.com/gnosygnu/xowa/releases/download/v3.1.5.1/xowa_app_macosx_v3.1.5.1.zip" class="external text" rel="nofollow">Mac OS X (32-bit)</a>
   </li>
 </ul>
 
 <h2><span class='mw-headline' id='Android'> Android </span></h2>
 <ul>
-  <li> <a href="https://github.com/gnosygnu/xowa/releases/download/v3.1.4.1/xowa_app_android_v3.1.4.1.apk" class="external text" rel="nofollow">Android</a>
+  <li> <a href="https://github.com/gnosygnu/xowa/releases/download/v3.1.5.1/xowa_app_android_v3.1.5.1.apk" class="external text" rel="nofollow">Android</a>
   </li>
 </ul>
 
 <h2><span class='mw-headline' id='Source'> Source code </span></h2>
 <ul>
-  <li> <a href="https://github.com/gnosygnu/xowa/archive/v3.1.4.2.zip" class="external text" rel="nofollow">Source code</a>
+  <li> <a href="https://github.com/gnosygnu/xowa/archive/v3.1.5.1.zip" class="external text" rel="nofollow">Source code</a>
   </li>
 </ul>
 
@@ -65,7 +65,7 @@
       <span class='mw-headline'>Change log</span>
     </h2>
     <h2>
-      <span class="mw-headline" id="v3.1.4.2_.282016-01-28.29">v3.1.4.2 (2016-01-28)</span>
+      <span class="mw-headline" id="v3.1.5.1_.282016-01-31.29">v3.1.5.1 (2016-01-31)</span>
     </h2>
     <h3>
       <span class="mw-headline" id="Windows_.2F_Linux_.2F_Mac_OS_X">Windows / Linux / Mac OS X</span>
@@ -75,42 +75,16 @@
     </p>
     <ul>
       <li>
-        Scribunto: Add built-in function: {{PROTECTIONEXPIRY}} (resolves script error on many en.w pages; Script error: @/xowa/bin/any/xowa/xtns/Scribunto/engines/Luaj/MWServer.lua:59 vm error: gplx.Err: callParserFunction: function was not found: function=PROTECTIONEXPIRY). See: <a href="/site/en.wikipedia.org/wiki/Alaska">https://en.wikipedia.org/wiki/Alaska</a> <a href="/site/en.wikipedia.org/wiki/Earth">https://en.wikipedia.org/wiki/Earth</a>
+        Luaj: Fix frontier pattern not working. Resolves error messages for every url reference on en.w pages. EX: <code>Check |archiveurl= value (help)</code>. See: <a href="https://en.wikipedia.org/wiki/A">https://en.wikipedia.org/wiki/A</a> <a href="https://en.wikipedia.org/wiki/Earth">https://en.wikipedia.org/wiki/Earth</a>
       </li>
       <li>
-        Xtn.FlaggedRevs: Add scribunto function: getStabilitySettings (resolves script error on many en.w pages; Script error: =Module:Effective protection level:16 attempt to index ? (a nil value)). See: <a href="/site/en.wikipedia.org/wiki/A">https://en.wikipedia.org/wiki/A</a> <a href="/site/en.wikipedia.org/wiki/Animal_Farm">https://en.wikipedia.org/wiki/Animal_Farm</a> <a href="/site/en.wikipedia.org/wiki/Agriculture">https://en.wikipedia.org/wiki/Agriculture</a>
-      </li>
-    </ul>
-    <p>
-      <b>trivial</b>
-    </p>
-    <ul>
-      <li>
-        Log: Do not log warnings when building category version1 (resolves numerous non-meaningful error messages in the log file).
-      </li>
-    </ul>
-
-    <h2>
-      <span class="mw-headline" id="v3.1.4.1_.282016-01-24.29">v3.1.4.1 (2016-01-24)</span>
-    </h2>
-    <h3>
-      <span class="mw-headline" id="PC">Windows / Linux / Mac OS X</span>
-    </h3>
-    <p>
-      <b>major</b>
-    </p>
-    <ul>
-      <li>
-        Luaj: Return position as int not string when using "()" regex (resolves script error on several dozen en.d / ru.d pages ("attempt to compare string with number expected, got string")). See: <a href="https://en.wiktionary.org/wiki/%D0%BA">https://en.wiktionary.org/wiki/к</a> <a href="https://en.wiktionary.org/wiki/%D0%BA%D0%B0%D0%B3%D0%B5%D0%B1%D0%B5%D1%88%D0%BD%D0%B8%D0%BA">https://en.wiktionary.org/wiki/кагебешник</a> <a href="https://ru.wiktionary.org/wiki/плавлавка" rel="nofollow" class="external free">https://ru.wiktionary.org/wiki/плавлавка</a>
+        Luaj: Handle backslash-escaped UTF8 sequences. Resolves error messages for many references on en.w pages. EX: <code>Specials character in |encyclopedia= at position 11 (help)</code>. See: <a href="https://en.wikipedia.org/wiki/A">https://en.wikipedia.org/wiki/A</a> <a href="https://en.wikipedia.org/wiki/Earth">https://en.wikipedia.org/wiki/Earth</a>
       </li>
       <li>
-        Scribunto: Handle nested gsub calls when using callback functions (resolves script error on a few dozen en.d pages ("Unrecognized 3ms suffix pattern.")). See: <a href="https://en.wiktionary.org/wiki/%D7%9B%D7%9C%D7%91">https://en.wiktionary.org/wiki/כלב</a>
+        Core: Parse decimals using US locale, not user locale. Resolves extremely wide pie charts on many de.w pages. {detected by anonymous} See: <a href="https://de.wikipedia.org/wiki/Hausmannst%C3%A4tten">https://de.wikipedia.org/wiki/Hausmannstätten</a>
       </li>
       <li>
-        Luaj: Handle multi-byte chars in lua variables (resolves script error on a few dozen en.d pages ("attempt to concatenate nil and string")). See: <a href="https://en.wiktionary.org/wiki/%E3%81%A9%E3%81%86">https://en.wiktionary.org/wiki/どう</a> <a href="https://en.wiktionary.org/wiki/%EB%A5%BC">https://en.wiktionary.org/wiki/를</a>
-      </li>
-      <li>
-        Hzip: Handle jtidy moving &lt;span class='headline'&gt; outside of &lt;h#&gt; tag in rare combination of &lt;center&gt;, &lt;span&gt;, &lt;div&gt; (resolves a few dozen pages in en.s not showing). See: <a href="https://en.wikisource.org/wiki/On_the_Vital_Principle/Whole_text" rel="nofollow" class="external free">https://en.wikisource.org/wiki/On_the_Vital_Principle/Whole_text</a> <a href="https://en.wikisource.org/wiki/Avenarius_and_the_Standpoint_of_Pure_Experience/1" rel="nofollow" class="external free">https://en.wikisource.org/wiki/Avenarius_and_the_Standpoint_of_Pure_Experience/1</a>
+        Xtn.Pgbnr: Add {{PAGEBANNER}} extension. Resolves pagebanner not showing up on top of wikivoyage pages. See: <a href="https://en.wikivoyage.org/wiki/Stonehenge" rel="nofollow" class="external free">https://en.wikivoyage.org/wiki/Stonehenge</a> <a href="https://en.wikivoyage.org/wiki/Paris" rel="nofollow" class="external free">https://en.wikivoyage.org/wiki/Paris</a> <a href="https://en.wikivoyage.org/wiki/Munich" rel="nofollow" class="external free">https://en.wikivoyage.org/wiki/Munich</a> <a href="https://en.wikivoyage.org/wiki/Cumbernauld" rel="nofollow" class="external free">https://en.wikivoyage.org/wiki/Cumbernauld</a> [[Help:Diagnostics/Xtns/Wikivoyage/All]]
       </li>
     </ul>
     <p>
@@ -118,19 +92,10 @@
     </p>
     <ul>
       <li>
-        Scribunto: Handle multi-byte characters for lua bracketed regular expressions (resolves nothing known). See: <a href="https://en.wiktionary.org/wiki/%E3%81%A9%E3%81%86">https://en.wiktionary.org/wiki/どう</a> <a href="https://en.wiktionary.org/wiki/%EB%A5%BC">https://en.wiktionary.org/wiki/를</a>
+        Css: Remove from core.css #content a:link {color: #0645ad;}. Resolves wikivoyage banner links showing in blue instead of white. See: <a href="https://en.wikipedia.org/wiki/Stonehenge">https://en.wikipedia.org/wiki/Stonehenge</a>
       </li>
       <li>
-        Luaj: Use HOUR_OF_DAY for os.time, not HOUR (resolves hour off by 12 for os.time calls). See: [[Help:Diagnostics/Scribunto/Luaj]]
-      </li>
-      <li>
-        Xtns.Dpl: Write href using full_db not page_txt; EX:Category:A B -&gt; Category:A_B x&gt; A B. See: <a href="https://en.wikiquote.org/wiki/Wikiquote:Speedy_deletions">https://en.wikiquote.org/wiki/Wikiquote:Speedy_deletions</a>
-      </li>
-      <li>
-        Xtns.Dpl: Encode quotes in href title (resolves broken links if page title has quotes). See: <a href="https://en.wikibooks.org/wiki/Wikibooks:Alphabetical_classification/All_Books" rel="nofollow" class="external free">https://en.wikibooks.org/wiki/Wikibooks:Alphabetical_classification/All_Books</a>
-      </li>
-      <li>
-        Dumps.HTML: Add support for non-english File namespace (resolves many log messages when generating html dumps for non-english wikis).
+        Html: Add 'class="visible"' to content-sub node and 'client-js' to html node. Resolves extra gap at bottom of content-sub on wikivoyage pages. See: <a href="https://en.wikivoyage.org/wiki/Stonehenge" rel="nofollow" class="external free">https://en.wikivoyage.org/wiki/Stonehenge</a>
       </li>
     </ul>
     <p>
@@ -138,18 +103,10 @@
     </p>
     <ul>
       <li>
-        Hzip: Write file-title with correct case-sensitivity for wikis with a case-sensitive [[File:]] namespace.
+        Xtn.Mapsources: Parse key-value pairs from nested template calls. Resolves potential incorrect values for {{#dd2dms}} .
       </li>
       <li>
-        Html: Never change title bytes when passed to Xof_file_wkr_.Ttl_standardize(title) (nothing known in pc; affects hzip check for image links). See: <a href="https://en.wiktionary.org/wiki/freedom_of_speech">https://en.wiktionary.org/wiki/freedom_of_speech</a>
-      </li>
-    </ul>
-    <p>
-      <b>doc</b>
-    </p>
-    <ul>
-      <li>
-        Home: Redefine categories for Change log.
+        Import.Dansguardian: Fix page-redirection-deletion for en.wikipedia.org.
       </li>
     </ul>
     <h3>
@@ -160,16 +117,22 @@
     </p>
     <ul>
       <li>
-        Search: Do additional searches to return some results sooner.
+        Css: Show navbox and other tables.
       </li>
       <li>
-        Image: Load images directly in html if page has already been viewed.
+        Css: Always center table.
       </li>
       <li>
-        Url: Convert plus to space (affects pages not loading if canonical_url has plus for space). See: <a href="https://en.wikipedia.org/wiki/List_of_Fire_Emblem:Shadow_Dragon_characters">https://en.wikipedia.org/wiki/List_of_Fire_Emblem:Shadow_Dragon_characters</a>
+        File: Replace img.src ? with %3F. Resolves images with question marks not showing. See: <a href="https://en.wikipedia.org/wiki/Cleopatra">https://en.wikipedia.org/wiki/Cleopatra</a> <a href="https://en.wikipedia.org/wiki/Cave_paintings">https://en.wikipedia.org/wiki/Cave_paintings</a>
       </li>
       <li>
-        App: Add "send email" to crash reporting.
+        Xtn.Hiero: Show hieroglyphics (available in 2016-01 English Wikipedia release). Resolves hieroglyphics not appearing. See: <a href="https://en.wikipedia.org/wiki/Cleopatra">https://en.wikipedia.org/wiki/Cleopatra</a>
+      </li>
+      <li>
+        Css: Force table content to be transparent color in night mode.
+      </li>
+      <li>
+        Log: Deactivate logging to external SD card (again). {detected by anonymous}
       </li>
     </ul>
     <p>