How To Structure Data For Searchability

I am writing a search application specifically for music playlists. The genre and file format differs from playlist to playlist, and sometimes within the playlist there are differe

Solution 1:

If you try to think too hard on how to structure your data for searching, there is a good chance you will miss an important search that you could have really used in your app.

Alternatively (and this is from experience) you end up re-inventing all sorts of indexing techniques.

I have some experience with lucene (there is java and .net version, there was a C port but I am not sure how alive it is these days) - and it can do amazing things with data that is stored in any structure.

I like the look of couch db, just depends how much you want to experiment with something new and powerful, or go for something which is (currently) fairly battle hardened: lucene.

Solution 2:

A fulltext index will serve you best if your users are going to be the ones defining the queries. Just create a custom text field that describes each attribute you want to be searchable e.g. "urban filetype:pdf gospel" and search that.

Solution 3:

OK, just brainstorming here --

Perhaps using octal or binary to store your "format" types as a bitmask?

http://www.nitrogen.za.org/viewtutorial.asp?id=17

RandB: 1 HipHop:2 Gospel:4 Urban: 8

Now, these things are additive. You know that if something is tagged Urban, you're not going to store "8" in the flag field, but you'll store 11...Urban && HipHop && RandB. This is just a bit of "business intelligence" you'll have to have spelled out somewhere.

You can then use binary comparisons to figure out which flags you're looking for.

Solution 4:

I don't see how database software would play a role in your solution.

If I were to be the one implementing this, I would first ensure all related data is captured in a normalized way. This would include things like category, artwork, lyrics, etc.

The main advantage of this is your idea of 'complex' searches actually become quite simple.