1
0
mirror of https://github.com/gnosygnu/xowa.git synced 2025-05-30 14:04:56 +00:00

$version_number

This commit is contained in:
gnosygnu 2016-09-02 15:39:48 -04:00
parent 9253976235
commit 8daf05754d
9 changed files with 257 additions and 29 deletions

View File

@ -399,7 +399,7 @@
The Android app is a major release. It has a Random feature, shows more images, and adds CSS-tweaks.
</p>
<h3>
<span class="mw-headline" id="Documentation_for_html-dump_script._See_Dev.2FCommand-line.2FDumps">Documentation for html-dump script. See <a href="http://xowa.org/home/wiki/Dev/Command-line/Dumps.html" id="xolnki_11" title="Dev/Command-line/Dumps">Dev/Command-line/Dumps</a></span>
<span class="mw-headline" id="Documentation_for_html-dump_script._See_Dev.2FCommand-line.2FDumps">Documentation for html-dump script. See <a href="http://xowa.org/home/wiki/Dev/Command-line/Dumps.html" id="xolnki_11" title="Dev/Command-line/Dumps" class="xowa-visited">Dev/Command-line/Dumps</a></span>
</h3>
<p>
This item is self-explanatory. The XOWA Android app is getting more stable, so I felt it would be time to document the generation of the HTML databases.

View File

@ -420,7 +420,7 @@
</dl>
<ul>
<li>
<b>Requires separate post-processing generation step</b>: The wikitext dumps were automatically generated by downloading an XML dump. The HTML dumps requires another post-processing step that is not simple to run (See: <a href="http://xowa.org/home/wiki/Dev/Command-line/Dumps.html" id="xolnki_16" title="Dev/Command-line/Dumps">Dev/Command-line/Dumps</a>)
<b>Requires separate post-processing generation step</b>: The wikitext dumps were automatically generated by downloading an XML dump. The HTML dumps requires another post-processing step that is not simple to run (See: <a href="http://xowa.org/home/wiki/Dev/Command-line/Dumps.html" id="xolnki_16" title="Dev/Command-line/Dumps" class="xowa-visited">Dev/Command-line/Dumps</a>)
</li>
</ul>
<dl>

View File

@ -908,7 +908,7 @@
</p>
<ul>
<li>
Command-line: Expand instructions for generating HTML dumps. See: <a href="http://xowa.org/home/wiki/Dev/Command-line/Dumps.html" id="xolnki_13" title="Dev/Command-line/Dumps">Dev/Command-line/Dumps</a>
Command-line: Expand instructions for generating HTML dumps. See: <a href="http://xowa.org/home/wiki/Dev/Command-line/Dumps.html" id="xolnki_13" title="Dev/Command-line/Dumps" class="xowa-visited">Dev/Command-line/Dumps</a>
</li>
</ul>
<p>

View File

@ -150,6 +150,76 @@
<button class='options_button' id='xowa_srh_upgrade' onclick='upgrade_search("commons.wikimedia.org")'>Search</button>
</td>
</tr>
<tr>
<td>
cs.wikibooks.org
</td>
<td class='cell_center'>
n
</td>
<td class='cell_center'>
2016-08-29
</td>
<td class='cell_center'>
2015-08-05
</td>
<td class='cell_center'>
Dump complete
</td>
<td class='cell_center'>
sqlite3
</td>
<td class='cell_center'>
v2
</td>
<td class='cell_center'>
v1
</td>
<td class='cell_center'>
<button class='options_button' id='xowa_wiki_import' onclick='import_wiki("cs.wikibooks.org")'>Import</button>
</td>
<td class='cell_center'>
<button class='options_button' id='xowa_ctg_upgrade' onclick='upgrade_category("cs.wikibooks.org")'>Category</button>
</td>
<td class='cell_center'>
<button class='options_button' id='xowa_srh_upgrade' onclick='upgrade_search("cs.wikibooks.org")'>Search</button>
</td>
</tr>
<tr>
<td>
cs.wikinews.org
</td>
<td class='cell_center'>
n
</td>
<td class='cell_center'>
2016-09-01
</td>
<td class='cell_center'>
2015-08-06
</td>
<td class='cell_center'>
Dump complete
</td>
<td class='cell_center'>
sqlite3
</td>
<td class='cell_center'>
v2
</td>
<td class='cell_center'>
v1
</td>
<td class='cell_center'>
<button class='options_button' id='xowa_wiki_import' onclick='import_wiki("cs.wikinews.org")'>Import</button>
</td>
<td class='cell_center'>
<button class='options_button' id='xowa_ctg_upgrade' onclick='upgrade_category("cs.wikinews.org")'>Category</button>
</td>
<td class='cell_center'>
<button class='options_button' id='xowa_srh_upgrade' onclick='upgrade_search("cs.wikinews.org")'>Search</button>
</td>
</tr>
<tr>
<td>
cs.wikipedia.org
@ -158,7 +228,7 @@
n
</td>
<td class='cell_center'>
2016-08-01
2016-09-01
</td>
<td class='cell_center'>
2015-08-06
@ -187,36 +257,142 @@
</tr>
<tr>
<td>
cs.wikipedia_old.org
cs.wikiquote.org
</td>
<td class='cell_center'>
n
</td>
<td class='cell_center'>
1969-12-31
2016-08-31
</td>
<td class='cell_center'>
2015-08-05
</td>
<td class='cell_center'>
&lt;&lt;NULL&gt;&gt;
Dump complete
</td>
<td class='cell_center'>
xdat
</td>
<td class='cell_center'>
v1
sqlite3
</td>
<td class='cell_center'>
v2
</td>
<td class='cell_center'>
<button class='options_button' id='xowa_wiki_import' onclick='import_wiki("cs.wikipedia_old.org")'>Import</button>
v1
</td>
<td class='cell_center'>
<button class='options_button' id='xowa_ctg_upgrade' onclick='upgrade_category("cs.wikipedia_old.org")'>Category</button>
<button class='options_button' id='xowa_wiki_import' onclick='import_wiki("cs.wikiquote.org")'>Import</button>
</td>
<td class='cell_center'>
<button class='options_button' id='xowa_srh_upgrade' onclick='upgrade_search("cs.wikipedia_old.org")'>Search</button>
<button class='options_button' id='xowa_ctg_upgrade' onclick='upgrade_category("cs.wikiquote.org")'>Category</button>
</td>
<td class='cell_center'>
<button class='options_button' id='xowa_srh_upgrade' onclick='upgrade_search("cs.wikiquote.org")'>Search</button>
</td>
</tr>
<tr>
<td>
cs.wikisource.org
</td>
<td class='cell_center'>
n
</td>
<td class='cell_center'>
2016-09-01
</td>
<td class='cell_center'>
2015-08-05
</td>
<td class='cell_center'>
Dump complete
</td>
<td class='cell_center'>
sqlite3
</td>
<td class='cell_center'>
v2
</td>
<td class='cell_center'>
v1
</td>
<td class='cell_center'>
<button class='options_button' id='xowa_wiki_import' onclick='import_wiki("cs.wikisource.org")'>Import</button>
</td>
<td class='cell_center'>
<button class='options_button' id='xowa_ctg_upgrade' onclick='upgrade_category("cs.wikisource.org")'>Category</button>
</td>
<td class='cell_center'>
<button class='options_button' id='xowa_srh_upgrade' onclick='upgrade_search("cs.wikisource.org")'>Search</button>
</td>
</tr>
<tr>
<td>
cs.wikiversity.org
</td>
<td class='cell_center'>
n
</td>
<td class='cell_center'>
2016-08-31
</td>
<td class='cell_center'>
2015-08-06
</td>
<td class='cell_center'>
Dump complete
</td>
<td class='cell_center'>
sqlite3
</td>
<td class='cell_center'>
v2
</td>
<td class='cell_center'>
v1
</td>
<td class='cell_center'>
<button class='options_button' id='xowa_wiki_import' onclick='import_wiki("cs.wikiversity.org")'>Import</button>
</td>
<td class='cell_center'>
<button class='options_button' id='xowa_ctg_upgrade' onclick='upgrade_category("cs.wikiversity.org")'>Category</button>
</td>
<td class='cell_center'>
<button class='options_button' id='xowa_srh_upgrade' onclick='upgrade_search("cs.wikiversity.org")'>Search</button>
</td>
</tr>
<tr>
<td>
cs.wiktionary.org
</td>
<td class='cell_center'>
n
</td>
<td class='cell_center'>
2016-09-01
</td>
<td class='cell_center'>
2015-08-05
</td>
<td class='cell_center'>
Dump complete
</td>
<td class='cell_center'>
sqlite3
</td>
<td class='cell_center'>
v2
</td>
<td class='cell_center'>
v1
</td>
<td class='cell_center'>
<button class='options_button' id='xowa_wiki_import' onclick='import_wiki("cs.wiktionary.org")'>Import</button>
</td>
<td class='cell_center'>
<button class='options_button' id='xowa_ctg_upgrade' onclick='upgrade_category("cs.wiktionary.org")'>Category</button>
</td>
<td class='cell_center'>
<button class='options_button' id='xowa_srh_upgrade' onclick='upgrade_search("cs.wiktionary.org")'>Search</button>
</td>
</tr>
<tr>
@ -716,7 +892,7 @@
n
</td>
<td class='cell_center'>
2016-06-01
2016-09-01
</td>
<td class='cell_center'>
2015-08-06
@ -925,7 +1101,7 @@
n
</td>
<td class='cell_center'>
2016-08-02
2016-08-21
</td>
<td class='cell_center'>
2015-08-05

View File

@ -343,6 +343,7 @@ app.bldr.cmds {
add ('simple.wikipedia.org' , 'util.download') {dump_type = 'page_props';}
add ('simple.wikipedia.org' , 'util.download') {dump_type = 'image';}
add ('simple.wikipedia.org' , 'util.download') {dump_type = 'pagelinks';}
add ('simple.wikipedia.org' , 'util.download') {dump_type = 'imagelinks';}
add ('simple.wikipedia.org' , 'text.init');
add ('simple.wikipedia.org' , 'text.page') {
// calculate redirect_id for #REDIRECT pages. needed for html databases
@ -370,6 +371,9 @@ app.bldr.cmds {
// create an "image" table to get the metadata for all files in the current wiki
add ('simple.wikipedia.org' , 'wiki.image');
// create an "imagelinks" table to find out which images are used for the wiki (performance optimization)
add ('simple.wikipedia.org' , 'wiki.imagelinks');
// parse all page-to-page links
add ('simple.wikipedia.org' , 'wiki.page_link');
@ -384,7 +388,8 @@ app.bldr.cmds {
// cleanup all downloaded files as well as temporary files
add ('simple.wikipedia.org' , 'util.cleanup') {delete_tmp = 'y'; delete_by_match('*.xml|*.sql|*.bz2|*.gz');}
// v1 html generator
// parse every page in the listed namespace and gather data on their lnkis.
// this step will take the longest amount of time.
add ('simple.wikipedia.org' , 'file.lnki_temp') {
@ -419,6 +424,24 @@ app.bldr.cmds {
hzip_diff = 'y';
}
}
// v2 html generator; allows for multi-threaded / multi-machine builds
/*
add ('simple.wikipedia.org' , 'wiki.mass_parse.init') {cfg {ns_ids = '0|4|14';}}
add ('simple.wikipedia.org' , 'wiki.mass_parse.exec') {
cfg {
num_wkrs = 8; load_all_templates = 'y'; cleanup_interval = 50; hzip_enabled = 'y'; hdiff_enabled ='y'; manual_now = '2016-08-01 01:02:03';
// uncomment the following 3 lines if using the build script as a "worker" helping a "server"
// num_pages_in_pool = 32000;
// mgr_url = '\\server_machine_name\xowa\wiki\en.wikipedia.org\tmp\xomp\';
// wkr_machine_name = 'worker_machine_1'
}
}
// note that if multi-machine mode is enabled, all worker directories must be manually copied to the server directory (a build command will be added later)
add ('simple.wikipedia.org' , 'wiki.mass_parse.make');
*/
// aggregate the lnkis
add ('simple.wikipedia.org' , 'file.lnki_regy');
@ -511,6 +534,7 @@ app.bldr.cmds {
add ('en.wikipedia.org' , 'util.download') {dump_type = 'page_props';}
add ('en.wikipedia.org' , 'util.download') {dump_type = 'image';}
add ('en.wikipedia.org' , 'util.download') {dump_type = 'pagelinks';}
add ('en.wikipedia.org' , 'util.download') {dump_type = 'imagelinks';}
*/
/*
add ('en.wikipedia.org' , 'text.init');
@ -525,6 +549,7 @@ app.bldr.cmds {
add ('en.wikipedia.org' , 'wiki.image');
add ('en.wikipedia.org' , 'wiki.page_dump.make');
add ('en.wikipedia.org' , 'wiki.page_link');
add ('en.wikipedia.org' , 'wiki.imagelinks');
add ('en.wikipedia.org' , 'search.page__page_score') {iteration_max = 100;}
add ('en.wikipedia.org' , 'search.link__link_score') {page_rank_enabled = 'y';
score_adjustment_mgr {

View File

@ -343,6 +343,7 @@ app.bldr.cmds {
add ('simple.wikipedia.org' , 'util.download') {dump_type = 'page_props';}
add ('simple.wikipedia.org' , 'util.download') {dump_type = 'image';}
add ('simple.wikipedia.org' , 'util.download') {dump_type = 'pagelinks';}
add ('simple.wikipedia.org' , 'util.download') {dump_type = 'imagelinks';}
add ('simple.wikipedia.org' , 'text.init');
add ('simple.wikipedia.org' , 'text.page') {
// calculate redirect_id for #REDIRECT pages. needed for html databases
@ -370,6 +371,9 @@ app.bldr.cmds {
// create an "image" table to get the metadata for all files in the current wiki
add ('simple.wikipedia.org' , 'wiki.image');
// create an "imagelinks" table to find out which images are used for the wiki (performance optimization)
add ('simple.wikipedia.org' , 'wiki.imagelinks');
// parse all page-to-page links
add ('simple.wikipedia.org' , 'wiki.page_link');
@ -384,7 +388,8 @@ app.bldr.cmds {
// cleanup all downloaded files as well as temporary files
add ('simple.wikipedia.org' , 'util.cleanup') {delete_tmp = 'y'; delete_by_match('*.xml|*.sql|*.bz2|*.gz');}
// v1 html generator
// parse every page in the listed namespace and gather data on their lnkis.
// this step will take the longest amount of time.
add ('simple.wikipedia.org' , 'file.lnki_temp') {
@ -419,6 +424,24 @@ app.bldr.cmds {
hzip_diff = 'y';
}
}
// v2 html generator; allows for multi-threaded / multi-machine builds
/*
add ('simple.wikipedia.org' , 'wiki.mass_parse.init') {cfg {ns_ids = '0|4|14';}}
add ('simple.wikipedia.org' , 'wiki.mass_parse.exec') {
cfg {
num_wkrs = 8; load_all_templates = 'y'; cleanup_interval = 50; hzip_enabled = 'y'; hdiff_enabled ='y'; manual_now = '2016-08-01 01:02:03';
// uncomment the following 3 lines if using the build script as a "worker" helping a "server"
// num_pages_in_pool = 32000;
// mgr_url = '\\server_machine_name\xowa\wiki\en.wikipedia.org\tmp\xomp\';
// wkr_machine_name = 'worker_machine_1'
}
}
// note that if multi-machine mode is enabled, all worker directories must be manually copied to the server directory (a build command will be added later)
add ('simple.wikipedia.org' , 'wiki.mass_parse.make');
*/
// aggregate the lnkis
add ('simple.wikipedia.org' , 'file.lnki_regy');
@ -511,6 +534,7 @@ app.bldr.cmds {
add ('en.wikipedia.org' , 'util.download') {dump_type = 'page_props';}
add ('en.wikipedia.org' , 'util.download') {dump_type = 'image';}
add ('en.wikipedia.org' , 'util.download') {dump_type = 'pagelinks';}
add ('en.wikipedia.org' , 'util.download') {dump_type = 'imagelinks';}
*/
/*
add ('en.wikipedia.org' , 'text.init');
@ -525,6 +549,7 @@ app.bldr.cmds {
add ('en.wikipedia.org' , 'wiki.image');
add ('en.wikipedia.org' , 'wiki.page_dump.make');
add ('en.wikipedia.org' , 'wiki.page_link');
add ('en.wikipedia.org' , 'wiki.imagelinks');
add ('en.wikipedia.org' , 'search.page__page_score') {iteration_max = 100;}
add ('en.wikipedia.org' , 'search.link__link_score') {page_rank_enabled = 'y';
score_adjustment_mgr {

View File

@ -181,7 +181,7 @@
<a href="http://xowa.org/home/wiki/Dev/Command-line.html" id="xolnki_24" title="Dev/Command-line">Overview</a>
</li>
<li>
<a href="http://xowa.org/home/wiki/Dev/Command-line/Dumps.html" id="xolnki_25" title="Dev/Command-line/Dumps">Image dumps</a>
<a href="http://xowa.org/home/wiki/Dev/Command-line/Dumps.html" id="xolnki_25" title="Dev/Command-line/Dumps" class="xowa-visited">Image dumps</a>
</li>
<li>
<a href="http://xowa.org/home/wiki/Dev/Command-line/Wikidata.html" id="xolnki_26" title="Dev/Command-line/Wikidata">Wikidata</a>
@ -203,7 +203,7 @@
<a href="http://xowa.org/home/wiki/Dev/Design/Language_files.html" id="xolnki_29" title="Dev/Design/Language files">Language files</a>
</li>
<li>
<a href="http://xowa.org/home/wiki/App/UI/Menus.html" id="xolnki_30" title="App/UI/Menus">Menu configuration</a>
<a href="http://xowa.org/home/wiki/App/UI/Menus.html" id="xolnki_30" title="App/UI/Menus" class="xowa-visited">Menu configuration</a>
</li>
</ul>
</td>

View File

@ -360,9 +360,9 @@ li.active a, li.active a:hover
<td>
<textarea xowa_prop="xowa.api.usr.cache.info" class="options_textbox options_readonly" readonly='readonly' style="width: 600px; height: 100px;" id='xowa_prop_3'>
cache folder: C:\xowa\file
space used: 518.310 KB
file count: 105
oldest file: 2016-08-27 11:41:54
space used: 2.327 MB
file count: 306
oldest file: 2016-09-01 21:21:00
</textarea>
</td>
</tr>

View File

@ -379,6 +379,8 @@ add_grp_default('xowa.gui.menus.group.tools') {
add_spr;
add_btn_default('xowa.nav.setup.maintenance');
add_btn_default('xowa.nav.setup.download');
add_spr;
add_btn('gnosygnu.random', 'Random Page', 'e', '', 'app.api.nav.goto("Special:Random");');
}
add_grp_default('xowa.gui.menus.group.help') {
add_btn_default('xowa.nav.help.help');
@ -811,7 +813,7 @@ add_btn_default('xowa.gui.browser.prog_log.show');
<li>
<span class="reference-text">At the top of the window.</span>
</li>
</ul><span class="reference-text">For more information, see <a href="http://xowa.org/home/wiki/App/UI/Menus.html" id="xolnki_2" title="App/UI/Menus">App/UI/Menus</a></span>
</ul><span class="reference-text">For more information, see <a href="http://xowa.org/home/wiki/App/UI/Menus.html" id="xolnki_2" title="App/UI/Menus" class="xowa-visited">App/UI/Menus</a></span>
</li>
<li id="cite_note-browser.tabs.btns.source-2">
<span class="mw-cite-backlink"><a href="#cite_ref-browser.tabs.btns.source_2-0">^</a></span> <span class="reference-text">Specify the source for the tab button popup menu. This menu shows when:</span>
@ -819,7 +821,7 @@ add_btn_default('xowa.gui.browser.prog_log.show');
<li>
<span class="reference-text">Right-clicking on a tab button or in the blank area in the tab bar</span>
</li>
</ul><span class="reference-text">For more information, see <a href="http://xowa.org/home/wiki/App/UI/Menus.html" id="xolnki_3" title="App/UI/Menus">App/UI/Menus</a></span>
</ul><span class="reference-text">For more information, see <a href="http://xowa.org/home/wiki/App/UI/Menus.html" id="xolnki_3" title="App/UI/Menus" class="xowa-visited">App/UI/Menus</a></span>
</li>
<li id="cite_note-browser.html.page.source-3">
<span class="mw-cite-backlink"><a href="#cite_ref-browser.html.page.source_3-0">^</a></span> <span class="reference-text">Specify the source for the HTML box's popup menu. This menu shows when:</span>
@ -827,7 +829,7 @@ add_btn_default('xowa.gui.browser.prog_log.show');
<li>
<span class="reference-text">Right-clicking anywhere in the HTML area and nothing is selected</span>
</li>
</ul><span class="reference-text">For more information, see <a href="http://xowa.org/home/wiki/App/UI/Menus.html" id="xolnki_4" title="App/UI/Menus">App/UI/Menus</a></span>
</ul><span class="reference-text">For more information, see <a href="http://xowa.org/home/wiki/App/UI/Menus.html" id="xolnki_4" title="App/UI/Menus" class="xowa-visited">App/UI/Menus</a></span>
</li>
<li id="cite_note-browser.html.link.source-4">
<span class="mw-cite-backlink"><a href="#cite_ref-browser.html.link.source_4-0">^</a></span> <span class="reference-text">Specify the source for the HTML box's link menu. This menu shows when:</span>
@ -835,7 +837,7 @@ add_btn_default('xowa.gui.browser.prog_log.show');
<li>
<span class="reference-text">Right-clicking on a link or selected text in the HTML area</span>
</li>
</ul><span class="reference-text">For more information, see <a href="http://xowa.org/home/wiki/App/UI/Menus.html" id="xolnki_5" title="App/UI/Menus">App/UI/Menus</a></span>
</ul><span class="reference-text">For more information, see <a href="http://xowa.org/home/wiki/App/UI/Menus.html" id="xolnki_5" title="App/UI/Menus" class="xowa-visited">App/UI/Menus</a></span>
</li>
<li id="cite_note-browser.html.file.source-5">
<span class="mw-cite-backlink"><a href="#cite_ref-browser.html.file.source_5-0">^</a></span> <span class="reference-text">Specify the source for the HTML box's file menu. This menu shows when:</span>
@ -843,7 +845,7 @@ add_btn_default('xowa.gui.browser.prog_log.show');
<li>
<span class="reference-text">Right-clicking on an image or any other file in the HTML area</span>
</li>
</ul><span class="reference-text">For more information, see <a href="http://xowa.org/home/wiki/App/UI/Menus.html" id="xolnki_6" title="App/UI/Menus">App/UI/Menus</a></span>
</ul><span class="reference-text">For more information, see <a href="http://xowa.org/home/wiki/App/UI/Menus.html" id="xolnki_6" title="App/UI/Menus" class="xowa-visited">App/UI/Menus</a></span>
</li>
<li id="cite_note-browser.prog-6">
<span class="mw-cite-backlink"><a href="#cite_ref-browser.prog_6-0">^</a></span> <span class="reference-text">Specify the source for the status bar popup menu. This menu shows when:</span>
@ -851,7 +853,7 @@ add_btn_default('xowa.gui.browser.prog_log.show');
<li>
<span class="reference-text">Right-clicking on the status bar at the bottom of the screen</span>
</li>
</ul><span class="reference-text">For more information, see <a href="http://xowa.org/home/wiki/App/UI/Menus.html" id="xolnki_7" title="App/UI/Menus">App/UI/Menus</a></span>
</ul><span class="reference-text">For more information, see <a href="http://xowa.org/home/wiki/App/UI/Menus.html" id="xolnki_7" title="App/UI/Menus" class="xowa-visited">App/UI/Menus</a></span>
</li>
</ol>
</div>