[IIAB] gutenberg files

[Joel Steres]

The gutenberg db and index have been removed from the repository.

On closer inspection it does look like the cache/generated content is
the content the index shows under cache/epub.  The problems I
encountered are probably just due to the incomplete mirroring.

Hope you're doing well.

-Joel


On Tue, Mar 12, 2013 at 8:04 PM, Braddock <braddock at braddock.com> wrote:
> Hi Joel,
>
> Thanks for your activity.  I haven't been able to keep completely up the
> last few days.
>
> I mirrored some of cache/generated to another server using:
> rsync -avHS --delete --delete-after ftp.ibiblio.org::gutenberg-epub
> generated
>
> I've copied that incomplete download (only 5.7 GB) to zhen in
> /knowledge/data/gutenberg/cached now.
>
> If you want a symlink from within static/ that is fine with me.
>
> I've seen no sign of a cache/epub/ directory.
>
> I've been trying to keep the path /knowledge universal across devices (zhen,
> the Satellite, the GoFlex Home, and my personal server) so links into it
> should work anywhere.
>
> On a side note, the 100MB gutenberg.db should probably not be in the git
> repo.  I'd prefer if it lived under /knowledge/processed/, which is where
> I'm keeping all processed data.
>
> I hope to have some time to get back into IIAB in the next couple days.  We
> had the funeral today, so things should begin to return to normal.
>
> -braddock
>
>
>
>
> On 03/12/2013 09:58 AM, Joel Steres wrote:
>>
>> Hi Braddock,
>>
>>> I am also mirroring cache/generated - the gutenberg mirrors seem to block
>>> access to it via ftp etc, but I can get it via rsync. Maybe those files
>>> will
>>> be more consistent.
>>
>> Thanks for mirroring cache/generated. In the current catalog all files
>> referencing 'cache' point to cache/epub/... rather than
>> cache/generated/ and the contents of the two paths differ. I looked at
>> the rsync script from git but it does not seem to include the addition
>> for gutenberg.org/cache mirroring.  Could you either make the
>> adjustment or show me where to do so?
>>
>> Also, I found that html files include images.  It might be easier to
>> put the gutenberg files into the flask static directory and permit the
>> existing paths to work.  No objections if I symlink to
>> /knowledge/data/gutenberg/gutenberg/ from iiab/static/gutenberg/data/?
>>
>> -Joel
>
>
>