Comments on: How to mirror Wikipedia http://www.igeek.co.za/2009/10/16/how-to-mirror-wikipedia/ A Capetonian geek living the life Thu, 02 Feb 2012 05:51:41 +0000 hourly 1 http://wordpress.org/?v=3.3.1 By: pdh http://www.igeek.co.za/2009/10/16/how-to-mirror-wikipedia/comment-page-1/#comment-285045 pdh Tue, 17 Jan 2012 09:06:35 +0000 http://www.igeek.co.za/?p=476#comment-285045 Is anybody hosting a wikipedia mirror that is accessible on-line? Is anybody hosting a wikipedia mirror that is accessible on-line?

]]> By: Nick http://www.igeek.co.za/2009/10/16/how-to-mirror-wikipedia/comment-page-1/#comment-284664 Nick Tue, 17 Jan 2012 01:59:23 +0000 http://www.igeek.co.za/?p=476#comment-284664 For the two users above me, the images aren't available. Longer explanation here: http://en.wikipedia.org/wiki/Wikipedia:Database_download#Where_are_images_and_uploaded_files For the two users above me, the images aren’t available. Longer explanation here:
http://en.wikipedia.org/wiki/Wikipedia:Database_download#Where_are_images_and_uploaded_files

]]> By: Matt http://www.igeek.co.za/2009/10/16/how-to-mirror-wikipedia/comment-page-1/#comment-272919 Matt Mon, 02 Jan 2012 07:19:10 +0000 http://www.igeek.co.za/?p=476#comment-272919 unfortunately no.. i looked around for an answer to this problem but couldn't find one.. if you find a way to get all the images, please do share! 1 unfortunately no.. i looked around for an answer to this problem but couldn’t find one.. if you find a way to get all the images, please do share!

1

]]> By: Zach http://www.igeek.co.za/2009/10/16/how-to-mirror-wikipedia/comment-page-1/#comment-272735 Zach Mon, 02 Jan 2012 03:04:21 +0000 http://www.igeek.co.za/?p=476#comment-272735 Does this include all the images as well? Does this include all the images as well?

]]> By: Matt http://www.igeek.co.za/2009/10/16/how-to-mirror-wikipedia/comment-page-1/#comment-233475 Matt Mon, 14 Nov 2011 22:34:20 +0000 http://www.igeek.co.za/?p=476#comment-233475 excellent walk through, and the only definitive guide that i could find. thank you very much. a pretty consistant link to download the latest wikipedia pages would be: http://dumps.wikimedia.org/enwiki/latest/enwiki-latest-pages-articles.xml.bz2 in the end i had a little bit of trouble trying to get mwimport to be recognized/found. make sure to use the whole file name "mwimport.sh" excellent walk through, and the only definitive guide that i could find. thank you very much.

a pretty consistant link to download the latest wikipedia pages would be:
http://dumps.wikimedia.org/enwiki/latest/enwiki-latest-pages-articles.xml.bz2

in the end i had a little bit of trouble trying to get mwimport to be recognized/found. make sure to use the whole file name “mwimport.sh”

]]> By: Ken D'Ambrosio http://www.igeek.co.za/2009/10/16/how-to-mirror-wikipedia/comment-page-1/#comment-52461 Ken D'Ambrosio Wed, 23 Feb 2011 21:30:28 +0000 http://www.igeek.co.za/?p=476#comment-52461 Couple things: 1) Yes, a tweak to /etc/mysql/my.cnf was required; I changed max_packet_size to 128 MB. (It was 16 MB. 128 was probably overkill... but hey -- better safe than sorry.) 2) No need to de-compress the .bz2 -- and I don't know if you could even *do* that with tar, since a .bz2 is a bzip'd file, and not a tar archive. Instead, I used the following: bzcat enwiki-[...]-pages-articles.xml.bz2 | mwimport | mysql -p -f -u 3) Note that you can't just do a "wget" on the mwimport link above -- that's a link to a mediawiki page that, in turn, has text you need to stuff into an executable, and then chmod +x on. Couple things:
1) Yes, a tweak to /etc/mysql/my.cnf was required; I changed max_packet_size to 128 MB. (It was 16 MB. 128 was probably overkill… but hey — better safe than sorry.)
2) No need to de-compress the .bz2 — and I don’t know if you could even *do* that with tar, since a .bz2 is a bzip’d file, and not a tar archive. Instead, I used the following:
bzcat enwiki-[...]-pages-articles.xml.bz2 | mwimport | mysql -p -f -u
3) Note that you can’t just do a “wget” on the mwimport link above — that’s a link to a mediawiki page that, in turn, has text you need to stuff into an executable, and then chmod +x on.

]]> By: Dan http://www.igeek.co.za/2009/10/16/how-to-mirror-wikipedia/comment-page-1/#comment-567 Dan Mon, 14 Dec 2009 04:30:17 +0000 http://www.igeek.co.za/?p=476#comment-567 Just curious, how much memory was on the system that you did this on, and did you modify any of the settings for mySQL? Just curious, how much memory was on the system that you did this on, and did you modify any of the settings for mySQL?

]]>