From 0d79cdb55785c8c305684eb5e8c10528df6703e3 Mon Sep 17 00:00:00 2001 From: gnosygnu Date: Thu, 6 Oct 2016 02:45:30 -0400 Subject: [PATCH] $version_number --- home/wiki/App/Import/English_Wikipedia.html | 596 ++------------------ home/wiki/Wiki_setup/Arabic_wikis.html | 12 +- home/wiki/Wiki_setup/Czech_wikis.html | 12 +- home/wiki/Wiki_setup/English_wikis.html | 12 +- home/wiki/Wiki_setup/French_wikis.html | 12 +- home/wiki/Wiki_setup/German_wikis.html | 12 +- home/wiki/Wiki_setup/Haitian_wikis.html | 12 +- home/wiki/Wiki_setup/Polish_wikis.html | 12 +- home/wiki/Wiki_setup/Simple_wikis.html | 12 +- 9 files changed, 106 insertions(+), 586 deletions(-) diff --git a/home/wiki/App/Import/English_Wikipedia.html b/home/wiki/App/Import/English_Wikipedia.html index ca31f16c1..4e3ab24d7 100644 --- a/home/wiki/App/Import/English_Wikipedia.html +++ b/home/wiki/App/Import/English_Wikipedia.html @@ -25,92 +25,37 @@

- Overview + Background

- English Wikipedia has a lot of data. There are 16.0+ million pages with 22.0+ GB of text, as well as 4.25+ million thumbnails. + English Wikipedia has a lot of data. There are over 16 million pages with 25 GB of text, as well as 5 million images.

Setting all this up on your computer will not be a quick process. As a general estimate, you will need at least 30 GB and 5 hours processing time. If you want images as well, the numbers increase to 100 GB of disk space and 30+ hours of processing time. However, when you are done, you will have a complete, recent copy of English Wikipedia with images that can fit on a 128 GB SD card. @@ -119,507 +64,82 @@ Although the process itself is not hard, I strongly recommend that you try Simple Wikipedia first. Simple Wikipedia has 180,000 pages and 90,000 images. The text version uses 200 MB and sets up in 5 minutes. With images, this expands to 2 GB and 30 minutes of downloading time. Simple Wikipedia is a reasonably accurate simulation of English Wikipedia -- just much smaller. It'll also give you a pretty good idea of what XOWA can do.

- Part 1: Set up the wiki + Quick start

+

+ Articles +

- The first part is to set up the wiki. You have two options for this part: + There are two approaches:

+

+ Download pre-built wikis from archive.org +

+ +

+ Build wikis using the database dumps at wikimedia.org +

+

- Option 1: Import the wiki with XOWA + Images

-

- Overview -

-

- Steps -

- -

- That's it. The import process has now started. This part takes at least 5 hours so you may want to let it run for a while. When it's done, it will automatically load the Main Page. -

-

- Option 2: Download the wiki from archive.org -

-

- Overview -

- -

- Steps -

- -

- Option 2: Download the wiki from archive.org -

-

- Overview -

- -

- Steps -

-


- Part 2: Download the images + Detailed start

- This part takes much longer to complete. It will require at least 70 GB of disk space and 24+ hours of download time. You'll be downloading compressed files from archive.org. + See Wiki_setup/English_wikis

-

- Steps -

- -

- Updating English Wikipedia -

-

- Wikipedia is constantly updating. New pages are added, and existing pages are changed to include different images. The above steps will give you a complete set of images for 2015-04-03. However, if you want to stay up to date with Wikipedia, then you may also want to download the monthly updates. -

-

- Monthly updates will be posted at the same url: https://archive.org/details/Xowa_enwiki_latest There will be a new link with the name of the wiki dump: for example: 2016-08. They will have new images introduced in the Wikipedia dump for that month. Note that these updates should be downloaded and unzipped in order (i.e.: first 2015-05-02, then 2015-06-02, etc). There are some files that appear in multiple sets: the most recent copy of the file should always replace the earlier version. -

-

- Note that if you update your wiki, you do not have to update the images. The two are independent of each other. In other words, you can use the 2017-01-01 English Wikipedia xml dump with the 2015-04-03 English Wikipedia images. Note that new images in the 2017-01-01 dump will not show up until you download the appropriate monthly updates. -

-

- Disk space usage -

-

- Some may wonder why XOWA needs so much disk space, especially when compared to other apps. The following is a brief list of reasons: -

- -

- English Wikipedia -

-

- HTML dbs as of 2016-08 -

- -

- English Wikipedia -

-

- Main set as of 2016-06 -

- -

- Updates for 2016-07 -

- -

- Updates for 2016-08 -

- -

- Notes -

-
    -
  1. - ^ Note that when the import completes, it will move the 10 GB file to /xowa/wiki/#dump/done. This file can be deleted safely. Note that XOWA doesn't delete the file, as some users may want to keep the 10 GB file around for archival purposes, and redownoading 10 GB would be time-consuming. -
    2. -
diff --git a/home/wiki/Wiki_setup/Arabic_wikis.html b/home/wiki/Wiki_setup/Arabic_wikis.html index 45908bd80..297d79e91 100644 --- a/home/wiki/Wiki_setup/Arabic_wikis.html +++ b/home/wiki/Wiki_setup/Arabic_wikis.html @@ -38,7 +38,7 @@

- Build wikis using the Wikimedia database dumps at wikimedia.org + Build wikis using the database dumps at wikimedia.org

@@ -84,7 +84,7 @@
  • - 3 Build wikis using the Wikimedia database dumps at wikimedia.org + 3 Build wikis using the database dumps at wikimedia.org

    - Build wikis using the Wikimedia database dumps at wikimedia.org + Build wikis using the database dumps at wikimedia.org

    Automatic @@ -365,10 +365,10 @@ Find your wiki in the list

  • - Click the "download" link. + Click the download link.
  • - Wait a few hours for the wiki to build + Wait for the wiki to build. When it is done, it will automatically load the Main Page

    • diff --git a/home/wiki/Wiki_setup/Czech_wikis.html b/home/wiki/Wiki_setup/Czech_wikis.html index ffbdafb2c..ab5d53990 100644 --- a/home/wiki/Wiki_setup/Czech_wikis.html +++ b/home/wiki/Wiki_setup/Czech_wikis.html @@ -38,7 +38,7 @@

    - Build wikis using the Wikimedia database dumps at wikimedia.org + Build wikis using the database dumps at wikimedia.org

    @@ -84,7 +84,7 @@
  • - 3 Build wikis using the Wikimedia database dumps at wikimedia.org + 3 Build wikis using the database dumps at wikimedia.org

    - Build wikis using the Wikimedia database dumps at wikimedia.org + Build wikis using the database dumps at wikimedia.org

    Automatic @@ -365,10 +365,10 @@ Find your wiki in the list

  • - Click the "download" link. + Click the download link.
  • - Wait a few hours for the wiki to build + Wait for the wiki to build. When it is done, it will automatically load the Main Page

    • diff --git a/home/wiki/Wiki_setup/English_wikis.html b/home/wiki/Wiki_setup/English_wikis.html index bc6156081..056fc7714 100644 --- a/home/wiki/Wiki_setup/English_wikis.html +++ b/home/wiki/Wiki_setup/English_wikis.html @@ -38,7 +38,7 @@

    - Build wikis using the Wikimedia database dumps at wikimedia.org + Build wikis using the database dumps at wikimedia.org

    @@ -84,7 +84,7 @@
  • - 3 Build wikis using the Wikimedia database dumps at wikimedia.org + 3 Build wikis using the database dumps at wikimedia.org

    - Build wikis using the Wikimedia database dumps at wikimedia.org + Build wikis using the database dumps at wikimedia.org

    Automatic @@ -430,10 +430,10 @@ Find your wiki in the list

  • - Click the "download" link. + Click the download link.
  • - Wait a few hours for the wiki to build + Wait for the wiki to build. When it is done, it will automatically load the Main Page

    • diff --git a/home/wiki/Wiki_setup/French_wikis.html b/home/wiki/Wiki_setup/French_wikis.html index 0d753d73e..41478be07 100644 --- a/home/wiki/Wiki_setup/French_wikis.html +++ b/home/wiki/Wiki_setup/French_wikis.html @@ -38,7 +38,7 @@

    - Build wikis using the Wikimedia database dumps at wikimedia.org + Build wikis using the database dumps at wikimedia.org

    @@ -84,7 +84,7 @@
  • - 3 Build wikis using the Wikimedia database dumps at wikimedia.org + 3 Build wikis using the database dumps at wikimedia.org

    - Build wikis using the Wikimedia database dumps at wikimedia.org + Build wikis using the database dumps at wikimedia.org

    Automatic @@ -406,10 +406,10 @@ Find your wiki in the list

  • - Click the "download" link. + Click the download link.
  • - Wait a few hours for the wiki to build + Wait for the wiki to build. When it is done, it will automatically load the Main Page

    • diff --git a/home/wiki/Wiki_setup/German_wikis.html b/home/wiki/Wiki_setup/German_wikis.html index ef5e0a18c..4b8580f59 100644 --- a/home/wiki/Wiki_setup/German_wikis.html +++ b/home/wiki/Wiki_setup/German_wikis.html @@ -38,7 +38,7 @@

    - Build wikis using the Wikimedia database dumps at wikimedia.org + Build wikis using the database dumps at wikimedia.org

    @@ -84,7 +84,7 @@
  • - 3 Build wikis using the Wikimedia database dumps at wikimedia.org + 3 Build wikis using the database dumps at wikimedia.org

    - Build wikis using the Wikimedia database dumps at wikimedia.org + Build wikis using the database dumps at wikimedia.org

    Automatic @@ -418,10 +418,10 @@ Find your wiki in the list

  • - Click the "download" link. + Click the download link.
  • - Wait a few hours for the wiki to build + Wait for the wiki to build. When it is done, it will automatically load the Main Page

    • diff --git a/home/wiki/Wiki_setup/Haitian_wikis.html b/home/wiki/Wiki_setup/Haitian_wikis.html index 918685d69..f8e4009e8 100644 --- a/home/wiki/Wiki_setup/Haitian_wikis.html +++ b/home/wiki/Wiki_setup/Haitian_wikis.html @@ -38,7 +38,7 @@

    - Build wikis using the Wikimedia database dumps at wikimedia.org + Build wikis using the database dumps at wikimedia.org

    @@ -84,7 +84,7 @@
  • - 3 Build wikis using the Wikimedia database dumps at wikimedia.org + 3 Build wikis using the database dumps at wikimedia.org

    - Build wikis using the Wikimedia database dumps at wikimedia.org + Build wikis using the database dumps at wikimedia.org

    Automatic @@ -191,10 +191,10 @@ Find your wiki in the list

  • - Click the "download" link. + Click the download link.
  • - Wait a few hours for the wiki to build + Wait for the wiki to build. When it is done, it will automatically load the Main Page

    • diff --git a/home/wiki/Wiki_setup/Polish_wikis.html b/home/wiki/Wiki_setup/Polish_wikis.html index fdaefab2f..45ed8cdfb 100644 --- a/home/wiki/Wiki_setup/Polish_wikis.html +++ b/home/wiki/Wiki_setup/Polish_wikis.html @@ -38,7 +38,7 @@

    - Build wikis using the Wikimedia database dumps at wikimedia.org + Build wikis using the database dumps at wikimedia.org

    @@ -84,7 +84,7 @@
  • - 3 Build wikis using the Wikimedia database dumps at wikimedia.org + 3 Build wikis using the database dumps at wikimedia.org

    - Build wikis using the Wikimedia database dumps at wikimedia.org + Build wikis using the database dumps at wikimedia.org

    Automatic @@ -368,10 +368,10 @@ Find your wiki in the list

  • - Click the "download" link. + Click the download link.
  • - Wait a few hours for the wiki to build + Wait for the wiki to build. When it is done, it will automatically load the Main Page

    • diff --git a/home/wiki/Wiki_setup/Simple_wikis.html b/home/wiki/Wiki_setup/Simple_wikis.html index 4a8e79ac4..2e0283917 100644 --- a/home/wiki/Wiki_setup/Simple_wikis.html +++ b/home/wiki/Wiki_setup/Simple_wikis.html @@ -38,7 +38,7 @@

    - Build wikis using the Wikimedia database dumps at wikimedia.org + Build wikis using the database dumps at wikimedia.org

    @@ -84,7 +84,7 @@
  • - 3 Build wikis using the Wikimedia database dumps at wikimedia.org + 3 Build wikis using the database dumps at wikimedia.org

    - Build wikis using the Wikimedia database dumps at wikimedia.org + Build wikis using the database dumps at wikimedia.org

    Automatic @@ -191,10 +191,10 @@ Find your wiki in the list

  • - Click the "download" link. + Click the download link.
  • - Wait a few hours for the wiki to build + Wait for the wiki to build. When it is done, it will automatically load the Main Page