Comments on: ThinkingSphinx Rails plugin fork

By: Ed

Ed — Sat, 12 Dec 2009 17:02:13 +0000

Canvas,
I haven’t updated the code but I know that the latest TS gem has rolled in the timestamp delta feature. I’m not sure it’s implemented the exact same way as I did it, but it’s likely better :) Checkout the Github page.

By: Canvas

Canvas — Sat, 12 Dec 2009 01:21:50 +0000

Hi Ed,

I have been using your fork (version 0.9.9, rails 2.0.2, sphinx 0.9.8.1) with timestamp support in delta index. And it has been working really well. Thank you a lot for the hard work.

I am now upgrading rails to version 2.3.4, sphinx to 0.9.9 and thinking-sphinx to 1.3.9. I am wondering whether you have a similar fork for the latest thinking-sphinx.

Best wishes,

Canvas

By: Ed

Ed — Wed, 17 Jun 2009 13:34:57 +0000

I haven’t gotten that error. That looks like something outside of ThinkingSphinx. Can you tell me how to reproduce it?

By: baldrailers

baldrailers — Wed, 17 Jun 2009 09:37:46 +0000

ActionView::TemplateError (undefined method `per_page’ for #)

How did you resolve this one?

By: Ed

Ed — Sat, 03 Jan 2009 19:09:35 +0000

Canvas,
I don’t know why Sphinx would be removing the deltas upon a second run of the delta rake task. I run the same task hourly on our system and the deltas just keep getting recreated. The deltas should only disappear after a merge action (thinking_sphinx:merge) or a complete re-index (thinking_sphinx:index).

Checkout the docs for more details on the ideas behind indexing:
http://www.sphinxsearch.com/docs/current.html#indexing

By: Canvas

Canvas — Fri, 02 Jan 2009 19:08:19 +0000

Hi Ed,

To walk around the delta index problem, I am trying to modify thinking_sphinx_tasks.rb by
adding function call “ts_merge(…)” into rake task
“thinking_sphinx:index:delta”. But I am not sure whether this is the right approach or not.

One more question, should I just run “rake Shingking_sphinx:index” once when the
searchd is started and schedule “rake thingking_sphinx:index:delta” and “rake thingking_sphinx:index:merge” in background daemons to run repeatedly?

Thanks in advance. Any suggestion is appreciated.

Cheers

By: Canvas

Canvas — Wed, 31 Dec 2008 01:37:14 +0000

Hi there Ed,

Thank you very much for your information. I tried daemon gem, it works fine.

In the daemon I run “rake thinking_sphinx:index:delta” every 10 seconds. And I then ran into another interesting problem. The delta index built on the first call of “rake thinking_sphinx:index:delta” will be gone if the rake command is called a second time after some time interval. I tried it manually and got the same result. If the rake command can not be called repeatedly, what’s the point of having it? What else do I have to do besides calling the rake command in the Daemon to make it right?

This problem really conflicts with the delta index concept in my mind. I thought the delta index keeps accumulating when “rake thinking_sphinx:index:delta” is repeatedly called untill a “rake thingking_sphinx:index” or “rake thinking_sphinx:index:merge” is called.

One more question, do “rake thingking_sphinx:index” clear delta index? What about “rake thinking_sphinx:index:merge”?

Thanks in advance.

Cheers

By: Ed

Ed — Tue, 23 Dec 2008 01:15:03 +0000

Canvas,
For your cron issue - there could be many reasons it’s not working. Assuming this is running under root: can you check root’s email on that box? That’s where the output of all crons will go unless redirected somewhere else. Seeing the output of the task should point you in the right direction. The issue could be path-related. I don’t use .cron files myself - I do everything via crontab, so I’m not sure if the globals you set will work or not - but that’d be what I’d check. As for getting it to run more than once a minute. Cron can’t do that (to my knowledge). Furthermore, you probably don’t actually want to re-index more than once a minute. Unless you have really tiny indexes, the process will likely step on one another. I’d benchmark the runtime of the indexing process to see. If it only takes a couple seconds and you have to have it run every 10 seconds. Write a daemonized script to do so. Check out the daemon gem - has everything you’d need.

For your second question: The thinking_sphinx:index:all task just runs thinking_sphinx:index. It’s there just to be thorough. thinking_sphinx:merge merges in the delta indexes into the main indexes. According to the Sphinx docs, you should do this every once in awhile to keep up performance. We do it once a week on our ~2GB of indexes.

By: Canvas

Canvas — Fri, 19 Dec 2008 21:55:53 +0000

Hi there Ed,

One more question, it seems to me that “rake thinking_sphinx:configure”, “rake thinking_sphinx:delta”, “rake thinking_sphinx:index” are enough to build index. Why do we need “rake thinking_sphinx:index:all” and “rake thinking_sphinx:merge”? When should I use them and for what purpose? Thank you very much.

Cheers
–
Canvas

By: Canvas

Canvas — Fri, 19 Dec 2008 21:48:07 +0000

Hi there,

I set ThinkingSphinx.offline_indexing = true in my environment.rb.

The data in my system updates quite frequently, so I plan to perform “rake thinking_sphinx:index:delta” every 10 seconds and “rake thinking_sphinx:index” every 30 minutes. But I have two problems in doing so.

First of all, I have problem to create a cron job to run every 10 seconds. It seems that cron jobs can only be scheduled in minutes.

Secondly, do I need to do anything to trigger the cron? It seems not working. I created the cron file as “/etc/cron.d/sphinx_index.cron”. And the following is the content of the cron file:

SHELL=/bin/bash
PATH=//workspace/CA/BETA_3/EconveyancePro
MAILTO=
HOME=/

# build sphinx index every 30 minutes
*/30 * * * * rake thinking_sphinx:index.

Thank you very much in advance.

Cheers
–
Canvas