Tuesday, November 20, 2012

Don't Mix Model with Implementation

(This is a rewrite of a 11/20/12 post to bring it in line with the correct interpretation of Codd's true RDM as formalized by McGoveran.)

Here's what's wrong with the last week's database picture, namely:

"When you design your database tables there are some important things to think of:
- Normalize to remove redundant data
- Use the smallest datatype possible
- Create as few indexes as possible, but not too few
- Avoid redundant indexes
- Every table must have clustered index
This is important in a normal database but it is even more important in SQL Azure because you have limited space for every database, your connections may be terminated due to heavy use of resources, you pay for what you use and the data that you transfer. You can use the SQL Azure management portal do design your tables or write the T-SQL statement yourself. The syntax to create a table in SQL azure is the same as in SQL server 2005/2008, but not all options are supported in SQL Azure.
CREATE TABLE [dbo].[table1]
 ([id] [int] IDENTITY(1,1) NOT NULL,
  [column1] [NVARCHAR](50) NOT NULL,
  [column2] [NVARCHAR](15) NOT NULL,
  [column3] [TINYNT] NULL,
            IGNORE_DUP_KEY = OFF,
         -- FILLFACTOR=80,
         -- ALLOW_ROW_LOCKS = ON,
         -- ALLOW_PAGE_LOCKS = ON,
--HÃ¥kan Winther, A SQL Azure tip a day – Create a table

Logical-Physical Confusion

Failure to keep the three levels of representation
  1. Conceptual modeling: Informal representation of a reality of interest as business rules;
  2. Database (logical) design: Formal representation of the conceptual model as integrity constraints on relations;
  3. Physical implementation: Representation and access of relation data in storage.
distinct and and avoid conceptual-logical conflation (CLC)--mixing 1 and 2--and logical-physical confusion (LPC)--mixing 2 and 3--has been a constant theme here.

LPC is reflected in the five recommendations for database design: the last four are actually implementation, not database design recommendations--model. I know this may be considered pedantic, but LPC is so rampant and costly in the industry that this is a necessity, not pedantry.

Note: Incidentally, when you design databases you should think of relations, not tables--R-tables are representation of relations on some physical medium for visualization.

I don't know how an Azure SQL database is different from a "normal database" (whatever that means) and while I understand product-specific implementation recommendations, why the four above, let alone normalization, are "more important" for Azure, escapes me.

Physical Independence

LPC is one of the many fundamental misconceptions common in the industry[1] is that the RDM is not practical because it does not "handle" implementation, as reflected in the following:
"I have a single, if not simplistic, question: You constantly remind us that the relational model is a logical model having no connection to any physical model (so I infer). You also indicate how no commercial product fully implements the relational model. Therefore, how do we make use of the relational model when dealing with the physical constructs of a commercial database program (Oracle, Access, DB2, etc.)?
DBMS designers have not been immune to it: SQL products induce and reinforce it. Consider the above table definition statement in the context of Codd's 1969 proscription:
"Let us denote the data sublanguage by R and the host language by H. R permits the declaration of relations and their [constrained] domains [and] identifies the primary key for that relation. R permits the specification for retrieval of any subset of data from the data bank ... H permits supporting declarations which indicate, perhaps less permanently, how these relations are represented in storage."[2] (emphasis ours).
The section that starts with WITH violates the proscriptions by contaminating the model with implementation details.

It is in response to such violations that Codd issued his famous 12 rules to distinguish RDBMSs from non-relational ones[3]. Rule 8 states:

"Interactive applications and application programs should not have to be modified whenever [physical] changes in internal storage structures and access methods are made to the database".
So physical independence (PI) is a major relational advantage, not a weakness, because it simplifies data sub-languages and gives DBMS designers complete freedom of implementation and changes thereof at will without disrupting queries and applications, relegating performance optimization to the DBMS. This enormous value of PI should be considered in the context of the overwhelming variety and complexity of the myriad of DBMS configurations, settings and options in the context of different OSs, hardware, storage and access methods, data growth, concurrent access, network loads and so on, each of which has configurable settings of its own. Without RDM's PI application developers would have hopelessly continued to optimize all of this and re-optimize when anything changed. Here's a glimpse at it:
"Wait a minute! Some important options of the clustered index can't be controlled, like FILLFACTOR, ALLOW_xxx_LOCKS and DATA_COMPRESSION. What does that mean? Well, if you execute the following code, you can see that FILLFACTOR=0, ALLOW_xxx_LOCKS are 1 SELECT name, fill_factor, allow_row_locks, allow_page_locks FROM sys.indexes. That means that every page of the index will be filled to 100%, leaving no room for new records. Well, thats fine for indexes that are always increasing, but if not, you'll get extra IO operations caused by page splits and you'll get fragmented indexes, causing performance issues. To remove the fragmentation you need to rebuild your indexes regulary to increase the performance, but that is a subject for later blog posts. In the SQL Azure platform, you shouldn't need to care about the infrastructure, like files or file groups and it isn't possible to place tables or indexes in different file groups."
Contaminating logical design--the model--with such implementation considerations is unproductive and does costly damage to both . Which demonstrates Codd's genius in targeting the RDM at PI. Unfortunately, the industry has yet to fully appreciate it. That practitioners still insist on blaming the RDM for poor performance is testament to the sorry state of foundation knowledge.

A Note on Normalization and Redundancy

Codd probably borrowed the term normal form for relations from logic, where relation specifications can be expressed as first order predicates in prenex normal form[4]. A relation is in its normal form by definition--i.e., has attributes defined on simple domains with atomic values, otherwise it is not a relation, and first order predicate logic (FOPL)--half of the formal foundation of the RDM--is not sufficient, which robs databases of all relational benefits. For  designs with attributes defined on non-simple relation-valued domains (RVD) Codd devised a method for eliminating them without loss of information that he called normalization. Note very carefully that it is only a design repair method--relational databases consisting of properly designed relations do not require it--and it involves no redundancy!

During the development of the relational algebra (RA)--whose operations derive relations from other relations--it was discovered that certain relation designs produced certain types of redundancy that cause complications. Although those designs had to do with attribute dependencies, not with the normal form, they were labeled "higher" normal forms (2NF-5NF), the normal form was re-labeled first normal form (1NF) and the methods devised to eliminate those redundancies were labeled "further normalization"[5]. The redundancy in each of those forms could be eliminated by further normalizing it--splitting an affected relation into multiple relations in a higher form--up to 5NF, a form devoid of (those types of) redundancy, at which point relations are fully normalized.

This historical sequence of events has inhibited the realization that relations should be really in 5NF--not just in 1NF--by definition, which is what the Principle of Full Normalization (POFN) really means. Adherence to it obviates  any explicit normalization altogether and guarantees not just logical, but also semantic correctness to boot.

In other words, strictly speaking, normalization to 1NF does not eliminate redundancy, only further normalization up to 5NF does. But neither is explicitly necessary if databases are properly designed such that they are in 5NF[6].

Note: We ignore, for the purposes of this discussion, that the initial normal form and the subsequent 1NF are not exactly the same; and 6NF.



[2] Codd, E. F., Derivability, Redundancy and Consistency of Relations Stored in Large Data Banks, IBM Research Report, San Jose, California RJ599 (1969)

[3] Pascal, F., Interpreting Codd: The 12 Rules

[4] Pascal, F., The Interpretation and Representation of Database Relations

[5] Codd, E. F., Further Normalization of the Data Base Relational Model, IBM Research Report, San Jose, California RJ909 (1971)

[6] Pascal, F., The Costly Illusion: Normalization, Integrity and Performance

Do you like this post? Please link back to this article by copying one of the codes below.

URL: HTML link code: BB (forum) link code:

No comments:

Post a Comment