Database Within A Database (table Design)

Possible Duplicate: Database design to create tables on the fly I have a need to create a database within a database. This question is related to this. I will attempt to explain

Solution 1:

You may want to look at an Entity Attribute Value model (EAV) design for your data then.

Basically you can have a table with table names, and some other meta-data on tables.

Then, you can create a table for each of those rows to have the column data, such as datatype and name.

Then, you have a table where you put the values for each column, in a long table.

This allows you to dynamically create tables, or add/remove rows dynamically.

For a comparison on relational and EAV you can look at this question:

Entity Attribute Value Database vs. strict Relational Model Ecommerce

If you want to have a relational view of this data though, then you will need to create triggers to help keep views up-to-date, and this can be a great deal of work to get that to work well. If you don't need a relational view then you should be fine.

Another way to do this is with a NoSQL database (http://en.wikipedia.org/wiki/NoSQL), as the schema doesn't have to be set, and so you can just store the columns you need for that row.

At this point I would go the NoSQL way as there are many databases that can work, and the reinventing you need to do is minimal.

Solution 2:

For the last part of your question, you're asking how to do a cross-tabulation query against an EAV schema. Some databases support this via extensions to the SQL standard, others don't support it at all. For portability you have to do it in your app. PostgreSQL offers the crosstab function in the tablefunc extension for this.

If you go down the EAV path you will sooner or later regret it. It's useful in certain limited circumstances, but it's a bad fit for the relational model and causes lots of pain and problems, not the least of which is awful performance.

Consider instead:

• If at all possible, re-design so you don't need dynamic schema. Probably not possible in your case since your express requirement is a user-editable schema for a web-based database app, but in most cases this is the right choice.

• Dynamically create/drop schema with ALTER TABLE, CREATE TABLE, etc. Some databases are much better at this than others. PostgreSQL's transactional DDL can help a lot. Caution is required to avoid this becoming a performance and maintenance nightmare, but it's probably the sanest option if you're trying to model a relational database with dynamic structure.

• Key/value stores that are optimised for EAV-like querying; see Key/Value stores. Be careful, many of these systems don't offer full ACID semantics and may have limited query languages, so you can land up doing lots more work in the application.

• Storing XML or JSON in the database. You can do that with a relational DB, but you're likely to be better off with a document database. Same caveats as for K/V stores apply. This approach works ok if you're doing all the querying logic in your app and your data sizes aren't too big.

• Using database-specific features like PostgreSQL's hstore to support arbitrary key/value storage where required, and use standard relational design where k/v isn't required. If you want relations as output it's still a major PITA that involves inefficient crosstab queries and joining.

Chris makes a good point: your whole design is very dubious. See: The inner platform effect and TDWTF's take on it. Seriously, don't go there.