Blog: Items and events from XOWA's development

From XOWA: a free, open-source, offline wiki application

Release: v2.11.3.1 (2015-11-15 20:00 Sun)

v2.11.3 is a fairly minor release. It addresses one minor bug with "1. REDIRECT" showing up on pages.

Fix for "1. REDIRECT" showing up on some pages

This was a bug I introduced while fixing another issue in v2.11.2. Some pages would now show "1. REDIRECT", most notably en.w:Earth

Wiki bundles for Slovak, Slovenian, Galician, and Waray-Waray wikis

These have been uploaded to archive.org

Next release: v2.11.4

I fixed the one minor Wikibase issue, but am deferring a few other possible issues till later. I'm still chugging away on the HTML dump. My aim is to get a simple wiki dump for the next release.

I generated the English Wiki dump but there were only about 3 thousand new images, and I didn't feel that it merited an update. For wikis, I'm going to update French, and then go through as many as I can of Lithuanian, Limburgish, Estonian, Latvian and Nynorsk.

Release: v2.11.2.1 (2015-11-08 22:00 Sun)

v2.11.2 is a significant release. It addresses minor, but noticeable issues with gallery sections, location maps, and references. It also has some Wikibase fixes.

Fix for Gallery rendering vertically instead of horizontally

Wikimedia added a change a few weeks ago to force <gallery> tags to use a css. See https://phabricator.wikimedia.org/T110709 This resulted in galleries being shown vertically, instead of horizontally.

v2.11.2 now includes this css.

Fix for Location maps not working on non-English wikis

This has been a bug for a few months, but I had misdiagnosed it as bad Module code. Basically, non-English wikis would copy-paste the same Module:Location_map code in to their wikis. This failed to work in XOWA, because:

The Module code has a hard-coded reference to Template
XOWA actually generates the non-English name for Template for the wiki. For Croatian Wikipedia, this is Šablon not Template

This basically broke location maps in non-English wikis. I should have caught this earlier, but the first few I looked at were all bad usages (no location coordinates passed).

v2.11.2 now uses the canonical name just like MediaWiki.

Fix for extraneous pre-sections in References

I think this is also a recent bug, but I can't track down the actual cause. MediaWiki has inconsistent logic for trimming whitespace around template and link arguments. The one at issue was a piece of code like {{TemplateName| B\n }}. On my local version of MediaWiki (1.25.2), it doesn't trim the "\n " after the B and creates a <pre> section. However, on Wikipedia, it does trim the whitespace and does not create a <pre> section.

I think this behavior changed recently, but couldn't find the actual line of code in MediaWiki. For now, XOWA will automatically trim end whitespace for all template arguments. In my opinion this is more intuitive behavior anyway.

Minor Wikibase fixes

These are a handful of signficant fixes that only affect some several dozen pages. They are in brief:

in Lua code, entity.formatPropertyValues('P123') would not work. Note that entity.formatPropertyValues('property_name') did work
Lua code expects a "datatype" property on entity and a value of "globe-coordinate".
Lua code required actual numbers for quantity ("99"), not the literal Wikidata value ("+99")

Minor fixes for Redirect

These are two minor items worth more explanation:

Redirects would not work if they had an argument. For example, #REDIRECT [[Target_page|link=collapsed]]
Clicking on "Redirected from" links at the top of the page would not work. For example, try https://en.wikipedia.org/wiki/UK and click on "Redirected from UK"

Minor JTidy fix to always enclose body text

This was actually not working from the start. Basically tidy needs to take something like ...

<div class="floatleft">a</div>a1<div class="floatleft">b</div>b1

... and add some <p> tags to produce ...

<div class="floatleft">a</div><p>a1</p><div class="floatleft">b</div><p>b1</p>

This has a noticeable visual effect for some wikis (Croatian Wiktionary) which have HTML like the above.

I modified the jtidy_xowa.jar to handle this behavior.

Wiki bundles for Danish, Serbo-Croatian, Croatian, and Estonian wikis

These have been uploaded to archive.org

Next release: v2.11.3

I made some good progress with the rewritten HTML dump feature this week, but am still not done. I'm hoping to get ready for v2.11.3 but I need to work on a few other minor issues that have spilled over from this week. (namely in Wikibase).

For wikis, I'm generating 2015-11 English Wikipedia. I'm also going to continue through the minor wikis and try to do Slovak, Waray-Waray, Slovanian and Galician

Release: v2.11.1.1 (2015-11-01 19:30 Sun)

v2.11.1 is a minor release. There are a few minor fixes for Hebrew wikis.

Minor fixes for Hebrew wikis

v2.11.1 now supports Hebrew dates for the time function like {{#time:xhxjj xjx xhxjY|28-08-1608 + 341 days}} generating ד' ה'שס\"ט". In addition there is basic support for Hebrew grammar like {{grammar:prefixed|וabc}} generating ווabc.

Image packages for English Wikipedia, Hebrew, Norwegian and Vietnamese wikis

The English Wikipedia update for 2015-10 is finally up. The rest of the wikis are up at archive.org

Next release: v2.11.2

I've been working on a new UI for the Android alpha. I've also spent time rewriting the HTML dump. For the next few weeks, I don't expect to do anything with the desktop app outside of bug fixing.

For wikis, I'll move on to Danish, Serbo-Croatian, Hungarian and Esperanto wikis

Release: NONE (2015-10-25 20:00 Sun)

I'm skipping the release this week. I spent most of the time this week on the HTML dump as well as Android. I'm looking at doing the regular monthly release next week but don't expect any major issues.

I'm also working on the wikis below and will post during the week.

Release: v2.10.3.1 (2015-10-18 20:15 Sun)

v2.10.3 is a minor release. There are a few minor fixes for issues related to v2.10.2.

Minor fixes

There are two worth mentioning:

Commons wiki would not download files
Mediawiki would not handle the translate function

The rest are more trivial in comparison. You can check the [[Help:Change log]] for more detail.

Image packages for Serbian, Malay and Bulgarian wikis

I'm uploading Bulgarian now, but the rest should be ready at archive.org. See: https://archive.org/details/Xowa_srwiki_latest https://archive.org/details/Xowa_mswiki_latest and https://archive.org/details/Xowa_bgwiki_latest

Next release: v2.10.4

I'm still waiting on commons / wikidata before doing an English Wikipedia update. I'll work on Hebrew this week and Norwegian / Vietnamese as well.

For code, I'm going to change tack again and table the article update function. I'm working on a different XOWA Android UI with hopes of updating the alpha in the next few weeks.

Release: v2.10.2.1 (2015-10-11 22:45 Sun)

v2.10.2 is a major release. It has a critical fix for the 2015-10 English Wikipedia dump. It also features better functionality for Chinese / Serbian language wikis. Finally, it also has some significant under-the-hood changes

Fix for broken references in 2015-10 English Wikipedia dump

XOWA was showing broken references on 2.6 millions pages for the 2015-10 English Wikipedia dump. This was due to some recent changes in Module:Citation/CS1. Specifically, the number of local variables in the citation0 function was greater than the default 200 limit for Luaj.

I recompiled a new version of luaj_xowa.jar to fix this issue.

Language support for Chinese / Serbian languages

This item basically involves the following changes:

Serbian languages now have language dropdowns. For example, sr.wikipedia.org now allows you to switch between Cyrillic and Latin variants
Chinese and Serbian wikis now fully implement the "-{}-" syntax. This includes some significant functions like recursive conversion, word-interchangeability, and adhoc rule-definition. See: https://www.mediawiki.org/wiki/Writing_systems/Syntax

I'm really hoping that the new functionality matches MediaWiki. If you're a Chinese / Serbian user, please take a moment to spot-check a few pages and let me know if there is anything amiss.

Better support for <source> and <syntaxhighlight>

MediaWiki recently changed the <syntaxhighlight> extension away from geshi to Pygments. In the process, it looks like they changed a few of the general parsing rules as well. v2.10.4 has better support for these blocks, though actual syntax coloring is still some time away.

Full support for interwiki links

v2.10.2 now incorporates online calls to Wikimedia APIs. The first use was to download a full list of interwiki aliases for each wiki. For those users in offline mode, you can manually generate the list. For more info, see [[Help:Site config]]

Support for Dansguardian import filter

XOWA supports custom creation of wikis by matching articles against words. For example, you can create an English Wikipedia that doesn't have any Pokemon articles. Or you can create an English Wikipedia that just has Pokemon articles. The system is intricate but should be quite powerful. For more info, see [[Help:Import/Dansguardian]]

More source code cleanup

There was a lot of source code cleanup. The process is still ongoing as I'm working on removing old code, as well as rewriting some knottier sections.

Next release: v2.10.3

I didn't have time to generate any wikis this week. I'm going to do English, Serbian and one of Malay, Hebrew, Bulgarian for v2.10.2

For code, I'm going to start working on an article update function (update the offline article with the latest version from online Wikipedia).

Release: NONE (2015-10-04 22:00 Sun)

I'm skipping the release this week. I ran into some issues with Chinese / Serbian translations, though I think I'm pretty close to being done. In addition, I have some other significant changes in queue which I didn't want to introduce for a "first-week-of-the-month" release.

If you want a recent stable release, use v2.9.4.1. I'll keep that release up for a month.

Otherwise, next week's v2.10.2 will have more changes. I'll also resume generating wikis from the list below.