[IIAB] autocompletion search

[Joel Steres]

Hi Braddock,

I haven't had a chance to do testing with a smartphone.

I updated the autocomplete to sort titles based on downloads. (Rebuild
with gutenberg_build.sh required.)  It was a good suggestion. While
titles are sorted, creators and contributors are not. I suppose we
could compute a total download count for each author with a clever sql
query, but I haven't done that yet. This still leaves the question of
whether to interleave title/creator/contributor or leave them
clustered. I am still handling the autocomplete on the client.
Performance improved locally after I made some changes to the includes
a while back. I need to reevaluate the performance running from zhen
and on mobile devices.

I also fixed the title sorting so that ordering is not affected by
punctuation and case (at least for English words).

I noticed that some book entries remain for audio books that have no
audio files available (excluded) but were not removed because the
audio book has a text readme or two that kept me from deleting the
entry.

Finally, I'll be out of town until Thursday.

Best,
Joel

On Fri, Mar 15, 2013 at 3:37 PM, Braddock <braddock at braddock.com> wrote:
> Hi Joel,
> The Gutenberg search site looks great!
>
> Does it work on smartphones?  (haven't tried it yet)
>
> I think you can greatly enhance your autocomplete and search result rankings
> by using the pgterms:downloads counter in the catalog as a popularity
> measure.  By eyeball, it looks like a pretty reliable indicator.
>
> Obviously the autocomplete needs to be done server-side.  I have server-side
> autocomplete jqueryUI/Flask/python code snippets below if they are helpful.
> Basically you just return a JSON list.  See the jquery UI autocomplete demo
> for the javascript side - it isn't hard to set up.
>
> thanks,
> braddock
>
>
> @blueprint.route('wikititles_autocomplete')
> def wikititles_autocomplete_view():
>     partial = request.args.get('term')
>     p = wikititles_mongo.autocomplete(database(), partial)
>     j = json_dumps(list(p))
>     return Response(j, mimetype='application/json')
>
> [...snip...]
>
> def autocomplete(db, prefix):
>     upperPrefix = prefix.upper()
>     regex = get_prefix_regex(upperPrefix)
>     matching = db.wikititles.find(
>         {'name_in_upper_case': regex}
>     ).sort(
>         [('reverse_links', -1)]
>     )
>     return (title['name'] for title in matching)
>
>
>
>
> On 03/09/2013 03:25 PM, Joel Steres wrote:
>>
>> Based on suggestions at HAK I added prototype of a search autocomplete
>> feature.  To offload the server I pass all the possible matches to the
>> client at once rather than interactively getting autocomplete
>> suggestions from the server on each keypress.  The result is it is too
>> big/slow and only matches single strings.  Still, would someone please
>> test it and tell me if it is worth pursuing. (Note may require
>> patience.  When I loaded it from zhen across several network hops it
>> took 11 seconds before it was loaded.  Not sure why especially when
>> the response code was 304, not modified.)
>>
>> Autocomplete is only attached to the search box at
>> http://127.0.0.1:25000/iiab/books/
>
>
>