Blog

Release: v3.9.2.1 (2016-09-11 19:00 Sun)

The PC version is a major release. It has a fix for broken Category pages, a new Category system, a new page sync feature, and other changes.

The Android version is a minor release. It has a fix for dropped connections in Download Central.

(PC) Fix for Category pages not working

v3.8.5.1 had a bug that broke Category pages. For example, https://simple.wikipedia.org/wiki/Category:Earth . This would only retrieve one random category. v3.9.2.1 fixes the issue.

(PC) New Category system to handle HTML dumps

As per https://github.com/gnosygnu/xowa/issues/81 Category pages were not showing for HTML dumps. I rewrote the Category system to handle this. Here are the high-level points:

• Works for future HTML dumps: I will be uploading new versions of 2016-09 English, German and French Wikipedia this week that will use the Category system. Unfortunately, existing HTML dumps won't work with them.
• Smaller size: The new Category system makes some database changes to reduce file size. For English Wikipedia, that means a difference between 10 GB and 8 GB. 8 GB may sound like a lot for Categories, but keep in mind there are over 100 million page to category links.
• Backward compatible: The existing Category system will work with the new code-base.
• Does not work for text database dumps: XOWA originally started off storing files in text files instead of sqlite files. I switched over to SQLite three years ago and phased out text databases two years ago. It's possible that some users with old wikis (3 years old) may still have these text databases. If so, then the new Category system won't work.

In short, the changes should be transparent to most, except HTML dump users will soon get Categories.

There's a little more info at Category Overview for those interested.

(PC) Basic page synchronization feature to get latest from online

As per https://github.com/gnosygnu/xowa/issues/72 XOWA will now be able to sync pages directly from the online version. This feature is still a work-in-progress, and probably should only be done for selective pages. Certain pages won't sync correctly for a few more releases -- namely, pages with Math. See App/Xtn/Page_sync for more info.

(PC / Android) Download Central changes for dropped connections / Mac OS X

As per https://github.com/gnosygnu/xowa/issues/80 dropped connections in Download Central would appear to hang on Mac OS X. This release should now force Mac OS X to report dropped connections. In addition, it also will try to reconnect for a minute once a connection is dropped, before giving up.

(PC) Various parser changes including Protectiong Banner, stack overflow, and invalid UTF-8

This release has a bunch of parser changes. The Change log has a complete list. The most relevant items are the following:

• Protection banners now appear: Previously, they didn't show due to a bug in returning Protection data. Currently, they will show whenever the pages says they are protected. Note that no page is ever protected from editing in XOWA: you can always click on the Edit button.
• Stack overflow error: A handful of pages would crash the program with stack overflow. These will now just show with omitted content
• Xowa/Luaj issue with foreign characters: There was a multi-threading issue with foreign characters. These are random and very low frequency. They will be fixed for future dumps.

(PC) Other minor changes including read-only support, torrent-file links and docs for HTML dump

Thanks to multiple users, this build also resolves some minor issues:

• Read-only Sqlite files are slow (thanks to Ope30): Previously, a sqlite file marked as read-only would be much slower. This build now handles read-only files correctly.
• Torrent links for non-Wikipedia wikis were wrong (thanks to malakov1): Download Central was reporting the wrong torrent links for non-Wikipedia wikis. For example, English Wiktionary was being listed as http://archive.org/Xowa_enwiktionary_latest . This build corrects the issue.
• Wrong documentation for HTML dumps (thanks to Rüdiger Gleim): The command-line script for generating HTML dumps have changed over the past few releases. I forgot to update the documentation page at Dev/Command-line/Thumbs.

Also, at risk of omission, additional thanks to markusmaresch for reporting the Mac OS X issue, and MDE186 for testing the page sync feature.

Next release: v3.9.3

• Wikis: This is going to be a busy week. I'm going to generate 2016-09 versions of English, German, and French Wikipedia with the new Category System and Random.
• Dynamic update of wiki pages: I still have more work to do on this feature. In particular, I'd like to get Math and other images showing correctly.
• Image shrink: I'm going to try to shrink the English Wikipedia images on a quarterly basis. I'm hoping to get this integrated directly into Download Central, so Android users can get it to work

Wikis: Czech Wikis (2016-09) (2016-09-05 23:00 Mon)

The Czech wikis are available in Download Central. To get them, do the following:

• Go to home/wiki/Special:XowaDownloadCentral
• Click the "Download" icon to the right of the "Available" table
• The Czech wikis will appear at the bottom of the "Available" table.

Release: NONE (2016-09-04 22:00 Sun)

This week has no release. I'm working on a variety of items based on issues at https://github.com/gnosygnu/xowa/issues Thanks to Ope30, markusmarech, and some anonymous others for all their input!

The Czech wikis are done and being uploaded now. They should be ready by tomorrow in Download Central (click the Download icon to get a new list of wikis)

For wikis, I'm going to try to update 2016-09 English Wikipedia, but I want to work on adding Categories as well as a few other items.