Ideas for Search functionality

[Mike Grigsby]

My clients are finally yelling at me about search capability in my web apps. For Windows apps I've always used Mike Hanson's QBE. Rebuilding a similar system isn't feasible considering my time constraints.

I need to search on four fields, two of which are Strings of 1,000 and 10,000 bytes each.

Does anyone have any advice or suggestions? These are TPS files and I'm concerned about performance.

I'm thinking of a loop where I go through each field doing an INSTRING() sort of search and adding the record to a queue or something. I'm also concerned about the time overhead to deal with the queue development. Does anyone have an easier approach or code snippets they might share?

BTW, the 4.30 stuff looks great! Thanks Bruce. It fixed a lot of things that I thought were due to my lack of experience<g>.

[Wolfgang Orth]

Mike,

John Hickey did such:

http://www.nettalkcentral.com/index.php?option=com_smf&Itemid=36&topic=3.0

You can search for Customers Firstname, just type a single e to get all firstnames containing an e, just to have some stuff to be retrieved.

[Big Ian]

Mike

What you could try is doing it the same way that the search engines do it

Basically you parse out every word from the text and save it in a related file.  Then you parse out the search term, compare it to your list and prioritise on the number of hits.

There's more you can do but that should get you going.

HTH

[CyberFerret]

We recently had to build a text intensive search engine for a proprietary product database.

The way we did it was to dump the relevant text fields into a SQLite database, and then use the built in Full Text Searching engine in SQLite to do the searching.

Check out the excellent article on ClarionMag with respect to integrating SQLite into Clarion.  In essence, we pass a query to the SQLite engine which returns the search dataset into a queue.  In time trials, a search through 50,000 product lines returns a dataset in average 0.2 seconds.  That is the total time to build the query string, pass it to SQLite and for SQLite to return the results and build the queue in Clarion.

And we are talking complex queries such as "red OR blue widgets size 10 NOT wingtip".

Let me know if you want more info.

Cheers,
Devan