+
diff --git a/home/wiki/App/Bookmarks.html b/home/wiki/App/Bookmarks.html
index 238302038..521babfb8 100644
--- a/home/wiki/App/Bookmarks.html
+++ b/home/wiki/App/Bookmarks.html
@@ -158,13 +158,7 @@
^Note that in a textarea (such as Comments), Enter adds a new line. To save, press Control + Enter
-
- XOWA currently has two category systems. This page will explain the rationale / details behind both.
+ As of v3.9.2.1 XOWA has one Category system: v3.
+
+
+ The first part of this page will discuss v3.
+
+
+ The second part of the page is an archived copy of the earlier v1 and v2 explanation
+
- For more information about V2 internals see App/Category/Internals
+ Version 3 was introduced to handle Categories for HTML dumps on PC and Android. The high-level details are as follows:
+
+
+ Uses the Wikimedia categorylinks and page_props dumps: Like v2, v3 downloads separate MediaWiki dumps for categorylinks.sql and page_props.sql . Both files are needed to generate accurate renditions of the Wikipedia Category system.
+
+
+ Generates "*-xtn.category.*.xowa" files: v3 stores all the category info in "*-xtn.category.*.xowa" files. For smaller wikis, the data is stored instead in the "-core.xowa" file
+
+
+ Works with both Wikitext databases and HTML databases: v3 will work with wikis imported by "Import/Online" (Wikitext) as well as "Download Central" (HTML)
+
+
+ Is backwards compatible with v1 and v2: v3.9.2.1 will work with v1, v2, and v3 category systems
+
+
+ Is generated automatically with import: v3 is now generated automatically when importing a wiki. Previously, v2 would require a separate post-processing step under Import/Offline.
+
+
+ Smaller size: v3 makes some database changes to reduce file size. For English Wikipedia, that means a difference between 10 GB and 8 GB. 8 GB may sound like a lot for Categories, but keep in mind there are over 100 million page to category links.
+
+
+ Does not work for text database dumps: XOWA originally started off storing files in text files instead of sqlite files. I switched over to SQLite three years ago and phased out text databases two years ago. It's possible that some users with old wikis (3 years old) may still have these text databases. If so, then the new Category system won't work.
+
+
+
Version 1
@@ -119,13 +148,13 @@
V2 is the official category system and should generate Category pages just like Wikipedia.
+ XOWA uses SQLite database files to store its data.
+
+
+
+ Files have an extension of either ".xowa" or ".sqlite3". Note that ".xowa" files are just plain old sqlite3 databases with a renamed extension.
+
+
+ Files are limited to 2 GB. Note that SQLite can easily handle files greater than 100 GB. The 2 GB limit is imposed for the following two constraints:
+
+
+ 2 GB limit for SQLite files on Android. I've not found any documentation that states this, but opening a file larger than 2 GB file always fails.
+
+
+ 4 GB limit for FAT32 on flash cards. FAT32 is an old file-system format. However, it is the only format that is supported natively by all major operating systems. I install a set of wikis on a 200 GB FAT32-formatted microSD card and carry it around from machine to machine
+
+ SQLite browser. This is a free, open-source GUI that can be useful for those who don't want just a command-line shell. See: http://sqlitebrowser.org/
+
+
+
+ XOWA Database sets: Wikitext vs HTML
+
+
+ There are two types of database sets for XOWA: Wikitext and HTML
+
+
+ Wikitext
+
+
+
+ Wikitext is a custom language by MediaWiki to generate HTML. For example, two apostrophes are used for italics: ''italics''
+
+
+ Wikitext databases will have -text- in the file name. For example, en.wikipedia.org-text-ns.000.xowa
+
+ Wikitext databases have been supported since the very start of XOWA. As such, they have a full set of functionality
+
+
+ However, Wikitext databases are slower. They need to take the Wikitext and convert it to HTML. As such, Wikitext databases don't work for Android, nor for other low-spec machines like Raspberry Pi
+
+
+
+ HTML
+
+
+
+ HTML is the raw text as used by a web browser. For example, the following is used for italics: <i>italics</i>
+
+
+ HTML databases will have -html- in the file name. For example, en.wikipedia.org-html.xowa
+
+ HTML databases have only been supported recently (as of 2016-06). As such, they are missing a small percentage of functionality. The most significant piece of that functionality is Popups, but there may be other features.
+
+
+ HTML databases are fast, as they don't need to convert Wikitext to HTML. As such, HTML databases are used for Android and recommended for Raspberry Pi
+
+
+
diff --git a/home/wiki/App/Import/Download/Aria2.html b/home/wiki/App/Import/Download/Aria2.html
index d0243e4b1..e08a847b2 100644
--- a/home/wiki/App/Import/Download/Aria2.html
+++ b/home/wiki/App/Import/Download/Aria2.html
@@ -48,13 +48,7 @@
Stop it with Ctrl-C, and resume at a later time. Note that resume information is added to the text file (in this case xowa_image_download_https.txt)
-
+
diff --git a/home/wiki/App/Import/Download_Central.html b/home/wiki/App/Import/Download_Central.html
index 6f82d81b2..3b12aa472 100644
--- a/home/wiki/App/Import/Download_Central.html
+++ b/home/wiki/App/Import/Download_Central.html
@@ -306,13 +306,7 @@
^Most download managers will have various tricks to speed up downloads. For example, they can open up multiple connections to a server and download the file in pieces. In contrast, XOWA only opens one connection and downloads. For more technical info, see: http://stackoverflow.com/questions/21981690/why-do-download-managers-work-faster-than-the-default-browser-downloaders
-
-
-Categories
-
-
-
-
+
diff --git a/home/wiki/App/Import/English_Wikipedia.html b/home/wiki/App/Import/English_Wikipedia.html
index 1d14968d6..fb1bc7e80 100644
--- a/home/wiki/App/Import/English_Wikipedia.html
+++ b/home/wiki/App/Import/English_Wikipedia.html
@@ -228,13 +228,7 @@
^Note that when the import completes, it will move the 10 GB file to /xowa/wiki/#dump/done. This file can be deleted safely. Note that XOWA doesn't delete the file, as some users may want to keep the 10 GB file around for archival purposes, and redownoading 10 GB would be time-consuming.
-
+
diff --git a/home/wiki/App/Import/Offline.html b/home/wiki/App/Import/Offline.html
index e780b619e..db54f46d9 100644
--- a/home/wiki/App/Import/Offline.html
+++ b/home/wiki/App/Import/Offline.html
@@ -51,13 +51,7 @@
You will get a prompt about the file already existing. Answer "No" (do not delete the file).
-
-
-Categories
-
-
-
-
+
diff --git a/home/wiki/App/Import/Other_wikis.html b/home/wiki/App/Import/Other_wikis.html
index ca813a3cb..69b52c510 100644
--- a/home/wiki/App/Import/Other_wikis.html
+++ b/home/wiki/App/Import/Other_wikis.html
@@ -133,13 +133,7 @@
XOWA will update images every few months. If you would like an update earlier, please post at https://sourceforge.net/p/xowa/discussion/general/thread/67c4cdb5/
-
For example, if you have viewed the Earth article in Simple Wikipedia you will have downloaded the The Earth seen from Apollo 17.jpg to C:\xowa\file\commons.wikimedia.org not to C:\xowa\file\simple.wikipedia.org. Note that commons is the destination because the actual online url for the image is https://commons.wikimedia.org/wiki/File:The Earth seen from Apollo 17.jpg.
The keys "up" / "down" / "page-up" / "page-down" works like Firefox: The url-bar is meant to emulate Firefox's behavior. This includes:
@@ -92,7 +92,7 @@
Pressing "up" in the url bar goes goes to the last row.
- Pressing "page-up" jumps up by 5 rows. This number can be changed at Options/Search_suggest.
+ Pressing "page-up" jumps up by 5 rows. This number can be changed at Options/Search_suggest.
Pressing "page-up" when there is less than 5 rows will jump to the first row.
@@ -198,13 +198,7 @@
Many words with each word being linked to many pages but resulting in low-scored pages: For example, "page meg*" will only match one page in English Wikipedia: "Megan Page".
This has a low priority for (a) performance reasons and (b) frequency of use reasons. It may be re-evaluated later. Feel free to contact me for further discussion.
+
diff --git a/home/wiki/App/UI/Main_window.html b/home/wiki/App/UI/Main_window.html
index 9705af0c6..6c50f3fae 100644
--- a/home/wiki/App/UI/Main_window.html
+++ b/home/wiki/App/UI/Main_window.html
@@ -304,13 +304,7 @@
The "Browser box" has a right-click context menu. It also responds to the same keyboard shortcuts listed above.
-
+
diff --git a/home/wiki/App/Wiki_types/Wikia.com.html b/home/wiki/App/Wiki_types/Wikia.com.html
index 34e4995a1..69f1ab0cc 100644
--- a/home/wiki/App/Wiki_types/Wikia.com.html
+++ b/home/wiki/App/Wiki_types/Wikia.com.html
@@ -141,13 +141,7 @@
If you create a css that matches the wikia.com one and would like it to be part of XOWA's release, please contact me via Help/Feedback
-
+
diff --git a/home/wiki/App/Xtn/Browser/HTTP_Server.html b/home/wiki/App/Xtn/Browser/HTTP_Server.html
index e8c651e3a..1d92ba22e 100644
--- a/home/wiki/App/Xtn/Browser/HTTP_Server.html
+++ b/home/wiki/App/Xtn/Browser/HTTP_Server.html
@@ -74,13 +74,7 @@
^Note that you will need to run XOWA as root if you choose to pass port 80 for http_server_port.
-
-
-Categories
-
-
-
-
+
diff --git a/home/wiki/App/Xtn/Browser/TCP_Server.html b/home/wiki/App/Xtn/Browser/TCP_Server.html
index 128e92c70..04b789876 100644
--- a/home/wiki/App/Xtn/Browser/TCP_Server.html
+++ b/home/wiki/App/Xtn/Browser/TCP_Server.html
@@ -191,13 +191,7 @@
stops the console
-
-
-Categories
-
-
-
-
+
diff --git a/home/wiki/App/Xtn/Browser/XOWA_viewer.html b/home/wiki/App/Xtn/Browser/XOWA_viewer.html
index 9be0b9ed6..88bb4f444 100644
--- a/home/wiki/App/Xtn/Browser/XOWA_viewer.html
+++ b/home/wiki/App/Xtn/Browser/XOWA_viewer.html
@@ -240,13 +240,7 @@
Open the xowa_viewer.xpi in Firefox
-
-
-Categories
-
-
-
-
+
diff --git a/home/wiki/App/Xtn/Html/Popups.html b/home/wiki/App/Xtn/Html/Popups.html
index 125402bee..a1a0c3404 100644
--- a/home/wiki/App/Xtn/Html/Popups.html
+++ b/home/wiki/App/Xtn/Html/Popups.html
@@ -220,13 +220,7 @@
Textual data only: In keeping with both MediaWiki approaches, only textual data is shown. Tables, boxes (infoboxes, navboxes), and images are removed from the preview.
-
-
-Categories
-
-
-
-
+
diff --git a/home/wiki/App/Xtn/Html/Scripts.html b/home/wiki/App/Xtn/Html/Scripts.html
index 69ba95fce..5ee50c3ee 100644
--- a/home/wiki/App/Xtn/Html/Scripts.html
+++ b/home/wiki/App/Xtn/Html/Scripts.html
@@ -53,13 +53,7 @@
Save the most current contents of XOWA's xowa.ready.js file
-
-
-Categories
-
-
-
-
+
diff --git a/home/wiki/App/Xtn/Import/Dansguardian.html b/home/wiki/App/Xtn/Import/Dansguardian.html
index ca49cee23..c8fd10220 100644
--- a/home/wiki/App/Xtn/Import/Dansguardian.html
+++ b/home/wiki/App/Xtn/Import/Dansguardian.html
@@ -273,13 +273,7 @@ Sun
However, the number of rules will affect the amount of memory required by the computer. For example, 100 rules may take 1 MB. 10,000 rules may take 10 MB.
-
- The math module uses MathJax by default. You can change it to LaTEX by navigating to Options/Math and choosing LaTeX
+ The math module uses MathJax by default. You can change it to LaTEX by navigating to Options/Math and choosing LaTeX
XOWA currently has no instructions for Mac OS X. You can help by installing LaTEX and dvipng on our machine and posting your instructions. Please see Help/Feedback.
- The math module uses MathJax by default. You can change it to LaTEX by navigating to Options/Math and choosing LaTeX
+ The math module uses MathJax by default. You can change it to LaTEX by navigating to Options/Math and choosing LaTeX
Testing
@@ -86,13 +86,7 @@
Note also that this message box may disrupt dynamic math updates. You will need to manually refresh the page in order to see the math image. This manual refresh is only needed for the first page; future pages will automatically update images dynamically.
-
+ Sometimes though you may only want to update one page without:
+
+
+
+ Waiting for the dump to occur
+
+
+ Downloading and importing a whole new wiki
+
+
+
+ The Page sync feature allows you to update selected pages
+
+
+ Issues
+
+
+ The Page sync feature is still a work in progress. The following are known issues:
+
+
+ Math, Helper Buttons (enlarge / more info), Musical scores, and other images don't work
+
+
+ Wikipedia stores some images in a separate location. XOWA still needs code to detect this location, copy it offline, and show it correctly. This should be done in the next few releases, but in the meantime it's heavily recommended that you don't use Automatic sync for all pages. In particular, math pages will lose all equations after a sync. For example, https://en.wikipedia.org/wiki/Pythagorean_theorem
+
+
+ No rollback option
+
+
+ XOWA stores only the latest version of a page. Previous versions of the page will not be available after synchronization. This feature will be added in a future version
+
+
+ If you synchronize and want to rollback, you will need to do it manually for all updates. For example, here's a scenario for English Wikipedia:
+
+
+
+ Backup en.wikipedia.org-core.xowa.
+
+
+ Backup en.wikipedia.org-html.user.xowa and en.wikipedia.org-sync.xowa if they exist.
+
+ A page must exist in the offline wiki in order to be synchronized. New pages cannot be synchronized. Navigating to a new page will just result in a "Page not found" error.
+
+
+ This feature will also be added in a future version
+
+
+ Other issues
+
+
+ Other issues may be present. Please check https://github.com/gnosygnu/xowa/issues/72 for updates. Once the issue is closed, then the Page Sync feature will no longer be marked "Work in progress" and should be fully operational
+
+
+ Usage notes
+
+
+ Manual sync
+
+
+ Manual sync works by doing the following:
+
+
+
+ Enable "manual sync" in the options page
+
+
+ Click the "Sync" link in the left-hand sidebar.
+
+
+
+ Note the following details:
+
+
+
+ online mode required: You must have "Web access enabled" in Options/Security
+
+
+ exclusions: Page sync does not work for the following pages:
+
+
+ home wiki pages: For example, this page. The home wiki is updated offline with every release. The latest version can also be viewed online at http://xowa.org
+
+
+ Special pages: Nearly all special pages have dynamic content and cannot be "mirrored" offline
+
+
+ Wikia / non-Wikimedia pages: Wikia wikis and non-Wikimedia wikis cannot be synchronized. This may be an option for a future release, but there are currently no plans.
+
+
+
+
+
+ Auto sync
+
+
+ Automatic sync works by doing the following:
+
+
+
+ Enable "manual sync" in the options page
+
+
+ Visit the Main_Page for a wiki. The page will automatically sync
+
+
+
+ Note the following details:
+
+
+
+ default page is Main_Page: Due to the issues above, it's recommended that auto-sync only be enabled for the Main Page. Other pages can be added under custom scope
+
+
+ default interval is 1440 minutes (24 hours): By default, XOWA will only synchronize a page if the last synchronization check is at least 24 hours old. This interval can also be adjusted
+
+
+
+ Technical details
+
+
+ Manual sync
+
+
+ This is an overview of what occurs when the Sync link is pressed
+
+
+
+ XOWA calls the Wikipedia api to get the HTML version of the page. For example:
+
As of v2.5.4.1, XOWA stores all files in simple.wikipedia.org-file-user.xowa. Previously it used /xowa/file/#meta to track the metadata.
-
-
-Categories
-
-
-
-
+
diff --git a/home/wiki/Archive/Help/Known_issues.html b/home/wiki/Archive/Help/Known_issues.html
index 9b587ab8c..0a90ec1c3 100644
--- a/home/wiki/Archive/Help/Known_issues.html
+++ b/home/wiki/Archive/Help/Known_issues.html
@@ -44,13 +44,7 @@
<CategoryTree> uses JavaScript to selectively drill-down through a Category Tree
-
-
-Categories
-
-
-
-
+
diff --git a/home/wiki/Archive/Help/Version_compatibility.html b/home/wiki/Archive/Help/Version_compatibility.html
index d9dbb6232..12ecbde99 100644
--- a/home/wiki/Archive/Help/Version_compatibility.html
+++ b/home/wiki/Archive/Help/Version_compatibility.html
@@ -53,13 +53,7 @@
siteInfo.xml and wiki.gfs are no longer used. They may be deleted. The new replacements are wiki_core.gfs and wiki_stats.gfs. For an example, see /xowa/wiki/simple.wikipedia.org/cfg.
-
-
-Categories
-
-
-
-
+
diff --git a/home/wiki/Archive/Help/Version_notes.html b/home/wiki/Archive/Help/Version_notes.html
index 0f52e9d70..c7574c062 100644
--- a/home/wiki/Archive/Help/Version_notes.html
+++ b/home/wiki/Archive/Help/Version_notes.html
@@ -108,13 +108,7 @@
^Specifically, they will be saved in /xowa/user/anonymous/app/data/user_system_cfg.gfs
-
-
-Categories
-
-
-
-
+
diff --git a/home/wiki/Archive/Import/Batch.html b/home/wiki/Archive/Import/Batch.html
index 76b4e2fbb..0d5733a11 100644
--- a/home/wiki/Archive/Import/Batch.html
+++ b/home/wiki/Archive/Import/Batch.html
@@ -91,13 +91,7 @@ app.bldr.run;
Open a terminal and run the following java -jar xowa_windows.jar --cmd_file xowa_batch_wiki.gfs --app_mode cmd
-
+ As per https://github.com/gnosygnu/xowa/issues/81 Category pages were not showing for HTML dumps. I rewrote the Category system to handle this. Here are the high-level points:
+
+
+
+ Works for future HTML dumps: I will be uploading new versions of 2016-09 English, German and French Wikipedia this week that will use the Category system. Unfortunately, existing HTML dumps won't work with them.
+
+
+ Smaller size: The new Category system makes some database changes to reduce file size. For English Wikipedia, that means a difference between 10 GB and 8 GB. 8 GB may sound like a lot for Categories, but keep in mind there are over 100 million page to category links.
+
+
+ Backward compatible: The existing Category system will work with the new code-base.
+
+
+ Does not work for text database dumps: XOWA originally started off storing files in text files instead of sqlite files. I switched over to SQLite three years ago and phased out text databases two years ago. It's possible that some users with old wikis (3 years old) may still have these text databases. If so, then the new Category system won't work.
+
+
+
+ In short, the changes should be transparent to most, except HTML dump users will soon get Categories.
+
+
+ There's a little more info at Category Overview for those interested.
+
+
+ (PC) Basic page synchronization feature to get latest from online
+
+
+ As per https://github.com/gnosygnu/xowa/issues/72 XOWA will now be able to sync pages directly from the online version. This feature is still a work-in-progress, and probably should only be done for selective pages. Certain pages won't sync correctly for a few more releases -- namely, pages with Math. See App/Xtn/Page_sync for more info.
+
+
+ (PC / Android) Download Central changes for dropped connections / Mac OS X
+
+
+ As per https://github.com/gnosygnu/xowa/issues/80 dropped connections in Download Central would appear to hang on Mac OS X. This release should now force Mac OS X to report dropped connections. In addition, it also will try to reconnect for a minute once a connection is dropped, before giving up.
+
+
+ (PC) Various parser changes including Protectiong Banner, stack overflow, and invalid UTF-8
+
+
+ This release has a bunch of parser changes. The Change log has a complete list. The most relevant items are the following:
+
+
+
+ Protection banners now appear: Previously, they didn't show due to a bug in returning Protection data. Currently, they will show whenever the pages says they are protected. Note that no page is ever protected from editing in XOWA: you can always click on the Edit button.
+
+
+ Stack overflow error: A handful of pages would crash the program with stack overflow. These will now just show with omitted content
+
+
+ Xowa/Luaj issue with foreign characters: There was a multi-threading issue with foreign characters. These are random and very low frequency. They will be fixed for future dumps.
+
+
+
+ (PC) Other minor changes including read-only support, torrent-file links and docs for HTML dump
+
+
+ Thanks to multiple users, this build also resolves some minor issues:
+
+
+
+ Read-only Sqlite files are slow (thanks to Ope30): Previously, a sqlite file marked as read-only would be much slower. This build now handles read-only files correctly.
+
+
+ Torrent links for non-Wikipedia wikis were wrong (thanks to malakov1): Download Central was reporting the wrong torrent links for non-Wikipedia wikis. For example, English Wiktionary was being listed as http://archive.org/Xowa_enwiktionary_latest . This build corrects the issue.
+
+
+ Wrong documentation for HTML dumps (thanks to Rüdiger Gleim): The command-line script for generating HTML dumps have changed over the past few releases. I forgot to update the documentation page at Dev/Command-line/Thumbs.
+
+
+
+ Also, at risk of omission, additional thanks to markusmaresch for reporting the Mac OS X issue, and MDE186 for testing the page sync feature.
+
+
+ Next release: v3.9.3
+
+
+
+ Wikis: This is going to be a busy week. I'm going to generate 2016-09 versions of English, German, and French Wikipedia with the new Category System and Random.
+
+
+ Dynamic update of wiki pages: I still have more work to do on this feature. In particular, I'd like to get Math and other images showing correctly.
+
+
+ Image shrink: I'm going to try to shrink the English Wikipedia images on a quarterly basis. I'm hoping to get this integrated directly into Download Central, so Android users can get it to work
+
v2.2.2 will include an update of the German wikis. I'm still making incremental progress on Android, and will hopefully complete most of the database rewrite.
Finally, an apology to those I've promised minor changes. I've spent a lot of time on Android recently, and there's been a backlog of items. I'm hoping to be done with the Android version in the next few weeks and will work on these items as I get time.
English Wikipedia is building now, so hopefully I'll have that ready for next week.
-
-
-Categories
-
-
-
-
+
diff --git a/home/wiki/Blog/2016-07.html b/home/wiki/Blog/2016-07.html
index 0ab7cfd21..8026c545d 100644
--- a/home/wiki/Blog/2016-07.html
+++ b/home/wiki/Blog/2016-07.html
@@ -489,13 +489,7 @@
Wikis: I'm working on French next and adding them to Download Central.
-
-
-Categories
-
-
-
-
+
diff --git a/home/wiki/Blog/2016-08.html b/home/wiki/Blog/2016-08.html
index 4686c9c40..30077f840 100644
--- a/home/wiki/Blog/2016-08.html
+++ b/home/wiki/Blog/2016-08.html
@@ -450,13 +450,7 @@ http://ftp.fi.muni.cz/pub/wikimedia/
mwad integration with the status bar: Another week of slippage here as well.
-
+ As per https://github.com/gnosygnu/xowa/issues/81 Category pages were not showing for HTML dumps. I rewrote the Category system to handle this. Here are the high-level points:
+
+
+
+ Works for future HTML dumps: I will be uploading new versions of 2016-09 English, German and French Wikipedia this week that will use the Category system. Unfortunately, existing HTML dumps won't work with them.
+
+
+ Smaller size: The new Category system makes some database changes to reduce file size. For English Wikipedia, that means a difference between 10 GB and 8 GB. 8 GB may sound like a lot for Categories, but keep in mind there are over 100 million page to category links.
+
+
+ Backward compatible: The existing Category system will work with the new code-base.
+
+
+ Does not work for text database dumps: XOWA originally started off storing files in text files instead of sqlite files. I switched over to SQLite three years ago and phased out text databases two years ago. It's possible that some users with old wikis (3 years old) may still have these text databases. If so, then the new Category system won't work.
+
+
+
+ In short, the changes should be transparent to most, except HTML dump users will soon get Categories.
+
+
+ There's a little more info at Category Overview for those interested.
+
+
+ (PC) Basic page synchronization feature to get latest from online
+
+
+ As per https://github.com/gnosygnu/xowa/issues/72 XOWA will now be able to sync pages directly from the online version. This feature is still a work-in-progress, and probably should only be done for selective pages. Certain pages won't sync correctly for a few more releases -- namely, pages with Math. See App/Xtn/Page_sync for more info.
+
+
+ (PC / Android) Download Central changes for dropped connections / Mac OS X
+
+
+ As per https://github.com/gnosygnu/xowa/issues/80 dropped connections in Download Central would appear to hang on Mac OS X. This release should now force Mac OS X to report dropped connections. In addition, it also will try to reconnect for a minute once a connection is dropped, before giving up.
+
+
+ (PC) Various parser changes including Protectiong Banner, stack overflow, and invalid UTF-8
+
+
+ This release has a bunch of parser changes. The Change log has a complete list. The most relevant items are the following:
+
+
+
+ Protection banners now appear: Previously, they didn't show due to a bug in returning Protection data. Currently, they will show whenever the pages says they are protected. Note that no page is ever protected from editing in XOWA: you can always click on the Edit button.
+
+
+ Stack overflow error: A handful of pages would crash the program with stack overflow. These will now just show with omitted content
+
+
+ Xowa/Luaj issue with foreign characters: There was a multi-threading issue with foreign characters. These are random and very low frequency. They will be fixed for future dumps.
+
+
+
+ (PC) Other minor changes including read-only support, torrent-file links and docs for HTML dump
+
+
+ Thanks to multiple users, this build also resolves some minor issues:
+
+
+
+ Read-only Sqlite files are slow (thanks to Ope30): Previously, a sqlite file marked as read-only would be much slower. This build now handles read-only files correctly.
+
+
+ Torrent links for non-Wikipedia wikis were wrong (thanks to malakov1): Download Central was reporting the wrong torrent links for non-Wikipedia wikis. For example, English Wiktionary was being listed as http://archive.org/Xowa_enwiktionary_latest . This build corrects the issue.
+
+
+ Wrong documentation for HTML dumps (thanks to Rüdiger Gleim): The command-line script for generating HTML dumps have changed over the past few releases. I forgot to update the documentation page at Dev/Command-line/Thumbs.
+
+
+
+ Also, at risk of omission, additional thanks to markusmaresch for reporting the Mac OS X issue, and MDE186 for testing the page sync feature.
+
+
+ Next release: v3.9.3
+
+
+
+ Wikis: This is going to be a busy week. I'm going to generate 2016-09 versions of English, German, and French Wikipedia with the new Category System and Random.
+
+
+ Dynamic update of wiki pages: I still have more work to do on this feature. In particular, I'd like to get Math and other images showing correctly.
+
+
+ Image shrink: I'm going to try to shrink the English Wikipedia images on a quarterly basis. I'm hoping to get this integrated directly into Download Central, so Android users can get it to work
+
The most important category to review is major. The other categories may be skipped due to lower importance (if they were more important, they would be bumped up to major).
- (add) Search: Added search suggest v2 for more accurate results {requested by Schnark}. See: Options/Search_suggest
+ (add) Search: Added search suggest v2 for more accurate results {requested by Schnark}. See: Options/Search_suggest
(fix) [broken since:v0.7.3] Import: Import fails if data_storage_format is not .gz.
@@ -1167,7 +1167,7 @@
(add) Addon: Wikidata JSON formatter works for addon.
- (add) Search: Search now has auto-wildcard option; EX:Earth -> Earth* {requested by hidp123}. See: Options/Search_suggest
+ (add) Search: Search now has auto-wildcard option; EX:Earth -> Earth* {requested by hidp123}. See: Options/Search_suggest
(add) Import: Option to not automatically delete xml file after import {requested by hidp123}. See: Options/Import
@@ -1688,7 +1688,7 @@
(fix) Search: Search broken when not using search suggestions {detected by hidp123;anonymous7}.
- (add) Search: Search now supports AllPages mode {requested by Schnark}. See: Options/Search_suggest
+ (add) Search: Search now supports AllPages mode {requested by Schnark}. See: Options/Search_suggest
(add) Import: Import/Script redesigned to be more intuitive {contributed by Schnark}. See: Dashboard/Import/Offline
@@ -1734,7 +1734,7 @@
- (add) Search: Dynamic search suggestions; EX:Go to search box and start typing "e" "a" "r" "t" "h" {contributed by Schnark}. See: Options/Search_suggest
+ (add) Search: Dynamic search suggestions; EX:Go to search box and start typing "e" "a" "r" "t" "h" {contributed by Schnark}. See: Options/Search_suggest
(fix) Xtn/Lst: Lst handles recursive {{#lst}} calls; EX:http://it.wikisource.org {detected by Joe Webber}. See: http://it.wikisource.org
@@ -3002,13 +3002,7 @@
inital release
-
-
-Categories
-
-
-
-
+
diff --git a/home/wiki/Change_log/2014.html b/home/wiki/Change_log/2014.html
index f3a8c0714..b99c08c7d 100644
--- a/home/wiki/Change_log/2014.html
+++ b/home/wiki/Change_log/2014.html
@@ -1845,7 +1845,7 @@
Module.Popups: Increase hover show time from 400 to 600 ms (affects number of accidental hovers while scrolling through page). See: Options/Popups
@@ -4933,13 +4933,7 @@
(change) Lang: Simplewiki sometimes defaults to "Home" as Main_page b/c {{int}} messages should be created in wiki messages, not lang messages {detected by anonymous}. See: http://wikimediafoundation.org
-
-
-Categories
-
-
-
-
+
diff --git a/home/wiki/Change_log/2015.html b/home/wiki/Change_log/2015.html
index fb0cc4f3d..331a50963 100644
--- a/home/wiki/Change_log/2015.html
+++ b/home/wiki/Change_log/2015.html
@@ -2930,13 +2930,7 @@
Parser.Template: Ignore safesubst for IsSubsting (affects 1 en.w page failing to load with stack overflow error). See: https://en.wikipedia.org/wiki/Wikipedia:WikiProject_Celtic_history_and_culture
-
-
-Categories
-
-
-
-
+
diff --git a/home/wiki/Change_log/2016.html b/home/wiki/Change_log/2016.html
index a6975b1c8..946e4c9ef 100644
--- a/home/wiki/Change_log/2016.html
+++ b/home/wiki/Change_log/2016.html
@@ -1054,13 +1054,7 @@
Html: Do not output alt in caption if alt only has whitespace (affects useless <hr> when alt is just blank space); EX:[[File:A.png|thumb|alt= ]]. See: https://en.wikipedia.org/wiki/Bird
-
-
-Categories
-
-
-
-
+
diff --git a/home/wiki/Change_log/v3.4.4.1.html b/home/wiki/Change_log/v3.4.4.1.html
index 28313d7bd..2863afa11 100644
--- a/home/wiki/Change_log/v3.4.4.1.html
+++ b/home/wiki/Change_log/v3.4.4.1.html
@@ -156,13 +156,7 @@
Resolved by: Convert byte[] to string for call to xowa__html__add__head.
-
+
+
\ No newline at end of file
diff --git a/home/wiki/Dashboard/Files/Deletion.html b/home/wiki/Dashboard/Files/Deletion.html
index ced30e8c3..758bf26e2 100644
--- a/home/wiki/Dashboard/Files/Deletion.html
+++ b/home/wiki/Dashboard/Files/Deletion.html
@@ -110,13 +110,7 @@
-
-
-Categories
-
-
-
-
+
diff --git a/home/wiki/Dashboard/Image_databases.html b/home/wiki/Dashboard/Image_databases.html
index 5eb24a5b0..b2859d599 100644
--- a/home/wiki/Dashboard/Image_databases.html
+++ b/home/wiki/Dashboard/Image_databases.html
@@ -2988,13 +2988,7 @@
The file folder is the folder that contains the file databases for the sqlite3 version. For example, for English Wikipedia on the same Windows system, the wiki folder would be C:\xowa\file\en.wikipedia.org. If you're using German Wikisource, then the wiki folder would be C:\xowa\file\de.wikisource.org
-
V2 is a category system that reproduces the exact category system in Wikipedia. However it requires additional files ("categorylinks") and takes up more space (as much as 10 GB for English Wikipedia)
@@ -1260,7 +838,7 @@
^Data storage for wiki. Can be either xdat or sqlite.
- ^Current category level of wiki. Can be either 1 or 2. See App/Category for more info
+ ^Current category level of wiki. Can be either 1 or 2. See App/Category for more info
^Current search level of wiki. Can be either 1 or 2. See App/Search for more info
@@ -1275,13 +853,7 @@
^Generate script to upgrade title search for wiki
-
-
-Categories
-
-
-
-
+
diff --git a/home/wiki/Dev/Code.html b/home/wiki/Dev/Code.html
index 0fe2ac992..49bdb05b3 100644
--- a/home/wiki/Dev/Code.html
+++ b/home/wiki/Dev/Code.html
@@ -335,13 +335,7 @@
Even with the above, the style is admittedly eccentric. For this, I will try to standardize it wherever it is inconsistent or whenever it becomes a point of contention.
-
+
diff --git a/home/wiki/Dev/Config_files.html b/home/wiki/Dev/Config_files.html
index 3c3b30753..9f3d8f69e 100644
--- a/home/wiki/Dev/Config_files.html
+++ b/home/wiki/Dev/Config_files.html
@@ -77,13 +77,7 @@
If it fails, then a message box will display an error. Note that the log will also have the same information: /xowa/user/anonymous/app/tmp/log/current.
-
Integers can not include commas or decimal places. For example, 1,234 is not valid. Use 1234 instead.
-
-
-Categories
-
-
-
-
+
diff --git a/home/wiki/Dev/Config_files/Examples.html b/home/wiki/Dev/Config_files/Examples.html
index 3b5f02533..fee00d059 100644
--- a/home/wiki/Dev/Config_files/Examples.html
+++ b/home/wiki/Dev/Config_files/Examples.html
@@ -119,13 +119,7 @@
Run XOWA. Note the "Hello World!" text at the top of the page.
-
The CSS files are downloaded automatically from Wikimedia servers. xowa_common.css is the main file. xowa_wiki.css is the contents of the css at simple.wikipedia.org/wiki/MediaWiki:Commons.css
-
-
-Categories
-
-
-
-
+
diff --git a/home/wiki/Dev/Design/Data_dump_format.html b/home/wiki/Dev/Design/Data_dump_format.html
index a4dd5b831..2fcf7e627 100644
--- a/home/wiki/Dev/Design/Data_dump_format.html
+++ b/home/wiki/Dev/Design/Data_dump_format.html
@@ -134,13 +134,7 @@
^This seems to be a result of Java's lack of support for an unsigned byte data-type, as well as other performance gains from a native C++/C application. (7-zip on Windows; bzip2 on Linux)
-
XOWA distributes all language files in the xowa_app_*.zip package. They can be found at /xowa/user/anonymous/lang/xowa/ under the appropriate language code. For example, the language definition for English is in en.gfs, French is fr.gfs, German is de.gfs, etc..
+
diff --git a/home/wiki/Dev/File/Setup/Index.html b/home/wiki/Dev/File/Setup/Index.html
index b6af4abf3..cb571dce9 100644
--- a/home/wiki/Dev/File/Setup/Index.html
+++ b/home/wiki/Dev/File/Setup/Index.html
@@ -28,13 +28,7 @@
Mac OS X: Dev/File/Setup/Mac OS X
-
-
-Categories
-
-
-
-
+
diff --git a/home/wiki/Dev/File/Setup/Linux.html b/home/wiki/Dev/File/Setup/Linux.html
index 30be95fcc..b4f805f1e 100644
--- a/home/wiki/Dev/File/Setup/Linux.html
+++ b/home/wiki/Dev/File/Setup/Linux.html
@@ -148,13 +148,7 @@ make
sudo make install
sudo ldconfig /usr/local/lib
-
+
diff --git a/home/wiki/Dev/SWT.html b/home/wiki/Dev/SWT.html
index ad3e77e82..2bb56a684 100644
--- a/home/wiki/Dev/SWT.html
+++ b/home/wiki/Dev/SWT.html
@@ -102,13 +102,7 @@
A very small minority of pages would not render the same as a browser. For example, https://en.wikisource.org/wiki/Lubbers_Don't_Live!
-
-
-Categories
-
-
-
-
+
diff --git a/home/wiki/Dev/Sqlite/Tips.html b/home/wiki/Dev/Sqlite/Tips.html
index 4a272cc6d..3c2572a3a 100644
--- a/home/wiki/Dev/Sqlite/Tips.html
+++ b/home/wiki/Dev/Sqlite/Tips.html
@@ -194,13 +194,7 @@ FROM tbl1 t1
+
diff --git a/home/wiki/Diagnostics/Xtns/Score/ABC.html b/home/wiki/Diagnostics/Xtns/Score/ABC.html
index 8cafc7acc..46b222a10 100644
--- a/home/wiki/Diagnostics/Xtns/Score/ABC.html
+++ b/home/wiki/Diagnostics/Xtns/Score/ABC.html
@@ -28,13 +28,7 @@ K:C
C, D, E, F,|G, A, B, C|D E F G|A B c d|
e f g a|b c' d' e'|f' g' a' b'|]
-
-
-Categories
-
-
-
-
+
diff --git a/home/wiki/Diagnostics/Xtns/Score/All.html b/home/wiki/Diagnostics/Xtns/Score/All.html
index ca171314f..bc7f662ac 100644
--- a/home/wiki/Diagnostics/Xtns/Score/All.html
+++ b/home/wiki/Diagnostics/Xtns/Score/All.html
@@ -73,13 +73,7 @@ K:C
C, D, E, F,|G, A, B, C|D E F G|A B c d|
e f g a|b c' d' e'|f' g' a' b'|]
-
Note that most users will only need to do steps 1 and 2. Steps 3 and 4 are only for those who want to generate HTML dumps / image updates on their own.
- Html_dump: Show video thumbnails when underlying video is actually an .ogg file.
+ Html_dump: Add left-hand sidebar for 2016-09 dumps. {detected by Ope30}
- DownloadCentral: Add md5 and size. {suggested by malakov1}
+ DownloadCentral: Fix Download Central "hanging" on Mac OS X. {detected by markusmaresch}
- Parser.Xml: Fix broken infobox images inside a few dozen en.w pages.
+ DownloadCentral: Fix torrent links for non-Wikipedia wikis. {detected by malakov1}
- Resolved by: Do not pop table stack if <tr> prematurely inside lnki.
+ Example: -> Xowa_enwiki_latest_archive.torrent x> Xowa_enwiktionary_latest_archive.torrent.
+
diff --git a/home/wiki/Help/Features/Search/Build.html b/home/wiki/Help/Features/Search/Build.html
index 078def2c2..366e9cfd9 100644
--- a/home/wiki/Help/Features/Search/Build.html
+++ b/home/wiki/Help/Features/Search/Build.html
@@ -213,13 +213,7 @@
Recreate through Dashboard/Wiki_maintenance: A search index is built when a wiki is first created. You can also recreate it at Dashboard/Wiki_maintenance
-
-
-Categories
-
-
-
-
+
diff --git a/home/wiki/Help/Feedback.html b/home/wiki/Help/Feedback.html
index f4a58afa3..e0df47a6e 100644
--- a/home/wiki/Help/Feedback.html
+++ b/home/wiki/Help/Feedback.html
@@ -31,13 +31,7 @@
Send an email to gnosygnu at gmail dot com
-
-
-Categories
-
-
-
-
+
diff --git a/home/wiki/Help/License/Code.html b/home/wiki/Help/License/Code.html
index 1555e2288..aaecff64e 100644
--- a/home/wiki/Help/License/Code.html
+++ b/home/wiki/Help/License/Code.html
@@ -37,13 +37,7 @@
You should have received a copy of the GNU Affero General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
-
XOWA values user privacy. It will never collect and upload personal information. It is designed to be run in a standalone environment, without any connection to the internet. See Help/Privacy/Desktop for more information.
For any questions or clarifications, please contact gnosygnu at gmail dot com.
-
-
-Categories
-
-
-
-
+
diff --git a/home/wiki/Help/Privacy/Desktop.html b/home/wiki/Help/Privacy/Desktop.html
index 7f7aa1808..eeb18c760 100644
--- a/home/wiki/Help/Privacy/Desktop.html
+++ b/home/wiki/Help/Privacy/Desktop.html
@@ -183,7 +183,7 @@
If you choose to use LaTEX then temporary files will be generated. All files are created in the following directory: /xowa/file/name_of_wiki/math.
- You can always disable LaTEX again at Options/Math
+ You can always disable LaTEX again at Options/Math
Content applications
@@ -194,13 +194,7 @@
These third-party viewer applications can be disabled at Options/Apps by setting all values to blank.
+
diff --git a/home/wiki/Help/Roadmap.html b/home/wiki/Help/Roadmap.html
index 98b9f0bd4..0cf881e60 100644
--- a/home/wiki/Help/Roadmap.html
+++ b/home/wiki/Help/Roadmap.html
@@ -142,13 +142,7 @@
This is scheduled for v2.1.0, but much depends on progress in other areas
-
- v3.8.5.1 features the following:
+ v3.9.2.1 features the following:
- (Wiki) Publish 2016-08 English Wikipedia and German Wikipedia
+ (PC) Fix for Category pages not working
- (PC / Android) Better html database support for packed galleries, video thumbnails, and image maps
+ (PC) New Category system to handle HTML dumps
- (PC) Support the lsth parser function
+ (PC) Basic page synchronization feature to get latest from online wiki
- (PC) Add torrent link / MD5 info to the Download Central Info page
+ (PC / Android) Download Central changes for dropped connections / Mac OS X
+
+
+ (PC) Various parser changes including Protectiong Banner, stack overflow, and invalid UTF-8
+
+
+ (PC) Other minor changes including read-only support, torrent-file links and docs for HTML dump.
@@ -182,14 +188,17 @@
Upgrade users
- If you are upgrading from any version before v3.8.1, you should download a xowa_app_v3.8.1+ file.
+ If you are upgrading from any version before v3.9.1, you should download a xowa_app_v3.9.1+ file.
- v3.3.4 features an auto-complete address bar. It also has a faster Search system. See App/Search
+ v3.6.3 features a Download page. See Download Central
+
+
+ v3.3.4 features an auto-complete address bar. It also has a faster Search system. See App/Search
@@ -205,13 +214,13 @@
v2.8.1 has a language converter dropdown for the Chinese wikis.
- v2.6.3 uses WebKit for MacOSX. For more info, see Blog
+ v2.6.3 uses WebKit for MacOSX. For more info, see Blog
@@ -221,13 +230,13 @@
- v2.5.4 has changed online file downloads. Among other things, ImageMagick and Inkscape are no longer required. For more info, see Files
+ v2.5.4 has changed online file downloads. Among other things, ImageMagick and Inkscape are no longer required. For more info, see Files
- v2.4.2 has improved Search performance and v2.4.4 has added multi-wiki Search. For more info, see App/Search
+ v2.4.2 has improved Search performance and v2.4.4 has added multi-wiki Search. For more info, see App/Search
- v2.4.1 has introduced major database layout changes. For more info, see Database
+ v2.4.1 has introduced major database layout changes. For more info, see Database
@@ -247,15 +256,15 @@
If the font size is too small, right-click and choose Font Increase
- Find box: Press Ctrl+F on your keyboard. This will show the Find box (in the lower left-hand corner). Type "tips". Notice that the word "Tips" is highlighted above. Note that Alt+P (Previous) and Alt+N (Next) will change search direction. See Find Box for more info.
+ Find box: Press Ctrl+F on your keyboard. This will show the Find box (in the lower left-hand corner). Type "tips". Notice that the word "Tips" is highlighted above. Note that Alt+P (Previous) and Alt+N (Next) will change search direction. See Find Box for more info.
- Address bar: Press Alt+D or Ctrl+L on your keyboard. This will take you to the Address bar at the top of the screen. Type "s.w:Earth" (no quotes) and press Enter. This will open the Simple Wikipedia page for Earth. See Address bar/Shortcuts for more shortcuts.
+ Address bar: Press Alt+D or Ctrl+L on your keyboard. This will take you to the Address bar at the top of the screen. Type "s.w:Earth" (no quotes) and press Enter. This will open the Simple Wikipedia page for Earth. See Address bar/Shortcuts for more shortcuts.
Middle-click in address bar: Copy and paste urls directly into the address bar with the middle mouse button to open the page. For example:
@@ -276,14 +285,14 @@
- Now you can copy and paste any Wikipedia url directly into XOWA and its page will open (and all its images will download.) See Address_bar/URLs for more info
+ Now you can copy and paste any Wikipedia url directly into XOWA and its page will open (and all its images will download.) See Address_bar/URLs for more info
- Middle-click on link: Open a tab in a new link by middle-clicking on it. Try middle-clicking the following: Options
+ Middle-click on link: Open a tab in a new link by middle-clicking on it. Try middle-clicking the following: Options
Right-click: Right click on an image and click save as file
@@ -294,7 +303,7 @@
- Wikidata: If you're seeing {{#property}} in your wikis you should also set up Wikidata. See App/Wiki_types/Wikidata
+ Wikidata: If you're seeing {{#property}} in your wikis you should also set up Wikidata. See App/Wiki_types/Wikidata
- 1 : search uses page title data. searches are less accurate, but less space is used
-
-
- 2 : (default) search uses title word data. searches are more accurate, but more space is used and setup is longer. For English Wikipedia, another 1.8 GB and 38 min of time are needed.
-
-
-
-
- ^Choose one of the following: (default is .gz)
+
text: fastest for reading but has no compression. Simple Wikipedia will be 300 MB
@@ -524,8 +501,8 @@ http://ftp.fi.muni.cz/pub/wikimedia/
-
- ^Enter a list of server urls separated by a comma and newline.
+
+ ^Enter a list of server urls separated by a comma and newline.
The default value is:
@@ -547,8 +524,8 @@ http://ftp.fi.muni.cz/pub/wikimedia/
-
- ^NOTE 1: this option only applies if the "Custom wiki commands" option is wiki.download,wiki.import (wiki.unzip must be removed)
+
+ ^NOTE 1: this option only applies if the "Custom wiki commands" option is wiki.download,wiki.import (wiki.unzip must be removed)
Select the method for importing a wiki dump bz2 file. (default is checked)
- ^Affects the xowa_common.css in /xowa/user/anonymous/wiki/wiki_name/html/. Occurs when importing a wiki. (default is checked)
+
+ ^Affects the xowa_common.css in /xowa/user/anonymous/wiki/wiki_name/html/. Occurs when importing a wiki. (default is checked)
checked : downloads xowa_common.css from the Wikimedia servers. Note that this stylesheet will be the latest copy but it may cause unexpected formatting in XOWA.
@@ -659,8 +636,8 @@ http://ftp.fi.muni.cz/pub/wikimedia/
-
- ^(Only relevant for wiki.unzip) Choose one of the following: (default is checked)
+
+ ^(Only relevant for wiki.unzip) Choose one of the following: (default is checked)
checked : (default) the .xml file is automatically deleted once the import process completes
@@ -670,8 +647,8 @@ http://ftp.fi.muni.cz/pub/wikimedia/
- ^ abcEnter a number in MB to represent the cutoff for generating sets of page databases as one file or many files (default is 1500)
+
+ ^ abcEnter a number in MB to represent the cutoff for generating sets of page databases as one file or many files (default is 1500)
XOWA generates three types of page databases:
+ ^Choose one of the following: (default is unchecked)
+
+
+ checked: Show "Sync" in sidebar
+
+
+ unchecked: (default) Do not show "Sync" in sidebar
+
+
Other notes:
+
+
+ Clicking "Sync" will manually synchronize current page
+
+
+ "Sync" will never show in the home wiki
+
+
+
+
+ ^Choose one of the following: (default is unchecked)
+
+
+ checked: Automatically synchronize page when viewing it
+
+
+ unchecked: (default) Do not automatically synchronize page
+
+
Other notes:
+
+
+ Pages are automatically synchronized based on options for time and scope. The default behavior is to synchronize when first visiting the page, and then every 24 hours afterwards
+
+
+ Automatic synchronization will never occur in the home wiki
+
+
+
+
+ ^Enter the number of minutes to wait before checking for a new update: (default is 1440) [4]
+
+
+ ^Specify specific page titles to sync: (default is Main_Page)
+
+
+ Main_Page means sync all Main_Pages in all wikis
+
+
+ * is a wildcard and means sync all pages in all wikis
+
+
+ Individual pages can be specified on each line.
+
+
For example, the following would only auto-synchronize two pages: the Main_Page and Earth in English Wikipedia:
+
+en.w:Main_Page
+en.w:Earth
+
+
+ Note that all lines must be separated by a UNIX line ending (ASCII 10) not a Windows line ending (ASCII 13 + ASCII 10)
+
+
diff --git a/home/wiki/Options/Scribunto.html b/home/wiki/Options/Scribunto.html
index ccea458bf..48f956aae 100644
--- a/home/wiki/Options/Scribunto.html
+++ b/home/wiki/Options/Scribunto.html
@@ -403,7 +403,7 @@ li.active a, li.active a:hover
executable permissions: Linux / Mac OS X users may need to grant execute permission to the Lua binary. To ensure proper setup, please do the following: