My June post @All Analytics:
The data management industry operates like the fashion industry. Its most persistent characteristic is migration from fad to fad. Every few years -- the number keeps getting smaller -- some "new" problem is discovered, for which the solution is so magical, that it is extended everywhere to everything, whether it is applicable or not. But many of these problems are old and fundamental and some of the “solutions” bring them back, rather than solve them. ...
Read it all. (Please comment there, not here)
Maybe All Analytics is broken or maybe I'm too thick to figure out how to comment on anything but the last commenter's comment, so I'm commenting here instead.
ReplyDeleteYou ask "Shouldn't we strive to avoid complexity [...]?" What I think you probably meant was shouldn't we strive not to add complexity?
So-called database designers who strive to conceal real compexity (by minimizing the number of tables they define) drive me bonkers. Clearly revealing genuine complexity that actually exists in the enterprise of interest is simplification.
I have not been able to reply @All Analytics since Friday. My editor was away since then but I emailed him and he'll reply on Monday.
DeleteTo your comment: Those who avoid modeling and design upfront, or "denormalize for performance" are deluding themselves: they trade structure complexity for integrity and manipulation complexity. Guess which is more costly?
The other core problem is that a conceptual/logical model the context of the universe of discourse must be stable for the life of the database. If reality changes faster, it must be redone IF LOGICAL AND SEMANTIC CORRECTNESS IS TO BE GUARANTEED. But with the "startup culture" of "asap monetizing" who cares?