2016-04-17 15:48:03 +00:00
<!DOCTYPE html>
< html dir = "ltr" >
< head >
< meta http-equiv = "content-type" content = "text/html;charset=UTF-8" / >
< title > Dev/Command-line/Wikidata - XOWA< / title >
< link rel = "shortcut icon" href = "https://gnosygnu.github.io/xowa/xowa_logo.png" / >
< link rel = "stylesheet" href = "https://gnosygnu.github.io/xowa/xowa_common.css" type = "text/css" >
2016-06-13 03:17:17 +00:00
< style data-source = "xowa" type = "text/css" >
2016-04-17 15:48:03 +00:00
.console {font-family: monospace; color: #EEEEEE ; background-color: black ; border: medium solid black;}
.code
,.path
,.url {font-family: monospace; color: black ; background-color: #f9f9f9 ; border: medium solid #f9f9f9;}
.bold {font-weight: 900;}
< / style >
2016-06-13 03:17:17 +00:00
< style data-source = "xowa" type = "text/css" >
2016-04-17 15:48:03 +00:00
.console {font-family: monospace; color: #EEEEEE ; background-color: black ; border: medium solid black;}
.code
,.path
,.url {font-family: monospace; color: black ; background-color: #f9f9f9 ; border: medium solid #f9f9f9;}
.bold {font-weight: 900;}
< / style >
< / head >
< body class = "mediawiki ltr sitedir-ltr ns-0 ns-subject skin-vector action-submit vector-animateLayout" spellcheck = "false" >
< div id = "mw-page-base" class = "noprint" > < / div >
< div id = "mw-head-base" class = "noprint" > < / div >
< div id = "content" class = "mw-body" >
< h1 id = "firstHeading" class = "firstHeading" > < span > Dev/Command-line/Wikidata< / span > < / h1 >
< div id = "bodyContent" class = "mw-body-content" >
< div id = "siteSub" > From XOWA: the free, open-source, offline wiki application< / div >
< div id = "contentSub" > < / div >
< div id = "mw-content-text" lang = "en" dir = "ltr" class = "mw-content-ltr" >
< div id = "toc" class = "toc" >
2017-06-26 01:14:55 +00:00
< div id = "toctitle" class = "toctitle" >
2016-04-17 15:48:03 +00:00
< h2 >
Contents
< / h2 >
< / div >
< ul >
< li class = "toclevel-1 tocsection-1" >
< a href = "#Import_using_the_XML_dump" > < span class = "tocnumber" > 1< / span > < span class = "toctext" > Import using the XML dump< / span > < / a >
< / li >
< li class = "toclevel-1 tocsection-2" >
< a href = "#Import_using_the_JSON_dump" > < span class = "tocnumber" > 2< / span > < span class = "toctext" > Import using the JSON dump< / span > < / a >
< / li >
< / ul >
< / div >
< p >
XOWA can import Wikidata through the command-line
< / p >
< h2 >
2017-04-28 01:02:09 +00:00
< span class = "mw-headline" id = "Import_using_the_XML_dump" > Import using the XML dump< / span >
2016-04-17 15:48:03 +00:00
< / h2 >
< p >
XOWA can build wikidata using the XML dump at www.mediwa/wikidatawiki/. This import is basically the same as an import of any other wiki.
< / p >
< p >
The script for the XML import follows.
< / p >
< pre class = 'code' >
// build wikidata database; this only needs to be done once, whenever wikidata is updated
add ('www.wikidata.org' , 'util.cleanup') {delete_all = 'y';}
add ('www.wikidata.org' , 'util.download') {dump_type = 'pages-articles';}
add ('www.wikidata.org' , 'util.download') {dump_type = 'categorylinks';}
add ('www.wikidata.org' , 'util.download') {dump_type = 'page_props';}
add ('www.wikidata.org' , 'util.download') {dump_type = 'image';}
add ('www.wikidata.org' , 'text.init');
add ('www.wikidata.org' , 'text.page');
add ('www.wikidata.org' , 'text.cat.core');
add ('www.wikidata.org' , 'text.cat.link');
add ('www.wikidata.org' , 'text.cat.hidden');
add ('www.wikidata.org' , 'text.term');
add ('www.wikidata.org' , 'text.css');
add ('www.wikidata.org' , 'util.cleanup') {delete_tmp = 'y'; delete_by_match('*.xml|*.sql|*.bz2|*.gz');}
< / pre >
< h2 >
2017-04-28 01:02:09 +00:00
< span class = "mw-headline" id = "Import_using_the_JSON_dump" > Import using the JSON dump< / span >
2016-04-17 15:48:03 +00:00
< / h2 >
< p >
As of v2.6.3, XOWA also provides basic support for building wikidata from the JSON dump. This support was added for the following reasons:
< / p >
< ul >
< li >
< b > Current delay in XML dumps< / b > : The last good wikidata XML dump was 2+ months old due to problems with dump generation. See: < a href = "https://phabricator.wikimedia.org/T98585" rel = "nofollow" class = "external free" > https://phabricator.wikimedia.org/T98585< / a >
< / li >
< li >
< b > JSON dumps recommended< / b > : Wikidata seems to prefer using the JSON dump over the XML dump. See: < a href = "http://www.wikidata.org/wiki/Wikidata:Database_download" rel = "nofollow" class = "external free" > http://www.wikidata.org/wiki/Wikidata:Database_download< / a >
< / li >
< li >
< b > JSON dumps are more frequent< / b > : The JSON dumps have been dumping regularly on a weekly basis. In contrast the XML dumps take 3 - 4 weeks.
< / li >
< / ul >
< p >
Despite these reasons, there are limitations to the JSON dump.
< / p >
< ul >
< li >
< b > Non-JSON pages not available< / b > : The JSON dump doesn't provide other pages, such as the Main Page or MediaWiki pages. Only pages in the main and property namespaces are available. This is by design. See: < a href = "https://lists.wikimedia.org/pipermail/wikidata/2015-June/006441.html" rel = "nofollow" class = "external free" > https://lists.wikimedia.org/pipermail/wikidata/2015-June/006441.html< / a >
< / li >
< li >
< b > Page metadata not available< / b > : Certain properties are not available, such as page_id and last_modified. XOWA provides substitutes for these values, but they will not match the Wikimedia version
< / li >
< / ul >
< p >
The script for the JSON import follows.
< / p >
< pre class = 'code' >
add ('www.wikidata.org' , 'util.cleanup') {delete_all = 'y';}
// TODO: add ('www.wikidata.org' , 'util.download') {dump_type = 'wikidata-json';}
add ('www.wikidata.org' , 'wbase.json_dump');
add ('www.wikidata.org' , 'text.term');
add ('www.wikidata.org' , 'text.css');
add ('www.wikidata.org' , 'util.cleanup') {delete_tmp = 'y'; delete_by_match('*.xml|*.sql|*.bz2|*.gz|*.json');}
< / pre >
2016-09-12 01:53:06 +00:00
2016-04-17 15:48:03 +00:00
< / div >
< / div >
< / div >
< div id = "mw-head" class = "noprint" >
< div id = "left-navigation" >
< div id = "p-namespaces" class = "vectorTabs" >
< h3 > Namespaces< / h3 >
< ul >
< li id = "ca-nstab-main" class = "selected" > < span > < a id = "ca-nstab-main-href" href = "index.html" > Page< / a > < / span > < / li >
< / ul >
< / div >
< / div >
< / div >
< div id = 'mw-panel' class = 'noprint' >
< div id = 'p-logo' >
< a style = "background-image: url(https://gnosygnu.github.io/xowa/xowa_logo.png);" href = "http://xowa.org/" title = "Visit the main page" > < / a >
< / div >
< div class = "portal" id = 'xowa-portal-home' >
< h3 > XOWA< / h3 >
< div class = "body" >
< ul >
< li > < a href = "http://xowa.org/index.html" title = 'Visit the main page' > Main page< / a > < / li >
< li > < a href = "http://xowa.org/screenshots.html" title = 'See screenshots of XOWA' > Screenshots< / a > < / li >
2016-06-26 06:10:12 +00:00
< li > < a href = "https://www.youtube.com/watch?v=q0qbXYXEH6M" title = "See a video of XOWA Desktop in action" > Video< / a > < / li >
2016-04-17 15:48:03 +00:00
< li > < a href = "http://xowa.org/home/wiki/Help/Download_XOWA.html" title = 'Download the XOWA application' > Download XOWA< / a > < / li >
< li > < a href = "http://xowa.org/home/wiki/Dashboard/Image_databases.html" title = 'Download offline wikis and image databases' > Download wikis< / a > < / li >
< / ul >
< / div >
< / div >
< div class = "portal" id = 'xowa-portal-started' >
< h3 > Getting started< / h3 >
< div class = "body" >
< ul >
< li > < a href = "http://xowa.org/home/wiki/App/Setup/System_requirements.html" title = 'Get XOWA's system requirements' > Requirements< / a > < / li >
< li > < a href = "http://xowa.org/home/wiki/App/Setup/Installation.html" title = 'Get instructions for installing XOWA' > Installation< / a > < / li >
< li > < a href = "http://xowa.org/home/wiki/App/Import/Simple_Wikipedia.html" title = 'Learn how to set up Simple Wikipedia' > Simple Wikipedia< / a > < / li >
< li > < a href = "http://xowa.org/home/wiki/App/Import/English_Wikipedia.html" title = 'Learn how to set up English Wikipedia' > English Wikipedia< / a > < / li >
< li > < a href = "http://xowa.org/home/wiki/App/Import/Other_wikis.html" title = 'Learn how to set up other Wikipedias' > Other Wikipedias< / a > < / li >
< / ul >
< / div >
< / div >
< div class = "portal" id = 'xowa-portal-android' >
< h3 > Android< / h3 >
< div class = "body" >
< ul >
< li > < a href = "http://xowa.org/home/wiki/Android/Setup.html" title = 'Setup XOWA on your Android device' > Setup< / a > < / li >
2016-06-26 06:10:12 +00:00
< li > < a href = "https://www.youtube.com/watch?v=jsMTBxGweUw" title = "See a video of XOWA Android in action" > Video< / a > < / li >
2016-04-17 15:48:03 +00:00
< / ul >
< / div >
< / div >
< div class = "portal" id = 'xowa-portal-help' >
< h3 > Help< / h3 >
< div class = "body" >
< ul >
< li > < a href = "http://xowa.org/home/wiki/Help/About.html" title = 'Get more information about XOWA' > About< / a > < / li >
< li > < a href = "http://xowa.org/home/wiki/Help/Contents.html" title = 'View a list of help topics' > Contents< / a > < / li >
< li > < a href = "http://xowa.org/home/wiki/Help/Media.html" title = 'Read what others have written about XOWA' > Media< / a > < / li >
< li > < a href = "http://xowa.org/home/wiki/Help/Feedback.html" title = 'Questions? Comments? Leave feedback for XOWA' > Feedback< / a > < / li >
< / ul >
< / div >
< / div >
< div class = "portal" id = 'xowa-portal-blog' >
< h3 > Blog< / h3 >
< div class = "body" >
< ul >
< li > < a href = "http://xowa.org/home/wiki/Blog.html" title = 'Follow XOWA' ' s development process ' > Current< / a > < / li >
< / ul >
< / div >
< / div >
< div class = "portal" id = 'xowa-portal-links' >
< h3 > Links< / h3 >
< div class = "body" >
< ul >
< li > < a href = "http://dumps.wikimedia.org/backup-index.html" title = "Get wiki datababase dumps directly from Wikimedia" > Wikimedia dumps< / a > < / li >
< li > < a href = "https://archive.org/search.php?query=xowa" title = "Search archive.org for XOWA files" > XOWA @ archive.org< / a > < / li >
< li > < a href = "http://en.wikipedia.org" title = "Visit Wikipedia (and compare to XOWA!)" > English Wikipedia< / a > < / li >
< / ul >
< / div >
< / div >
< div class = "portal" id = 'xowa-portal-donate' >
< h3 > Donate< / h3 >
< div class = "body" >
< ul >
< li > < a href = "https://archive.org/donate/index.php" title = "Support archive.org!" > archive.org< / a > < / li > <!-- listed first due to recent fire damages: http://blog.archive.org/2013/11/06/scanning - center - fire - please - help - rebuild/ -->
< li > < a href = "https://donate.wikimedia.org/wiki/Special:FundraiserRedirector" title = "Support Wikipedia!" > Wikipedia< / a > < / li >
2017-02-22 02:46:24 +00:00
< li > < a href = "http://xowa.org/home/wiki/Help/Donate.html" title = "Support XOWA!" > XOWA< / a > < / li >
2016-04-17 15:48:03 +00:00
< / ul >
< / div >
< / div >
2016-04-17 18:00:49 +00:00
2016-04-17 15:48:03 +00:00
< / div >
< / body >
< / html >
