Critique #3 The Database is NOT a detail
Critique #3 The Database is NOT a detail
Uncle Bob makes the proposition that business logic should only be in Use Cases and Entities, and that the DB is just an I/O mechanism. Universally applying this leads to suboptimal system quality.
Uncle Bob says the database is a detail, not part of the architecture. He says that our system should be independent of the database, that the database should be swappable, and that the database is just a data dump. It is advocated that behavior is only in use cases and entities, that the database is just for I/O, that we should defer decisions about the database as much as possible, and that we can unit test business rules without the database.
In the previous article, Critique #2 "The database is a detail", I outlined the original perspective of Uncle Bob. In this article, we we will ask the key question: Is it optimal to implement business logic only in Use Cases and Entities, and never in the database?
This article is part of the A Critique of Clean Architecture and TDD series. If you’re a technical leader and software architect who has already tried Clean Architecture and seen the problems that came along with it, or if you’re looking to implement Clean Architecture, I’m writing this for you. You’ve already spent countless hours, days, and months searching for solutions to questions; you just see polarized answers and people arguing with each other. It’s a jungle out there - scrawling through Stack Overflow and Reddit. I had spent several months researching Clean Architecture—and years implementing it in practice. I faced so many stumbling blocks and gray areas. I wish I had read a book back then to reconcile the “contractions” and provide me with guidance. And this is why we’re here, welcome. Let’s go on this journey, one step at a time.
Where should business logic be? App vs DB
By Clean Architecture, BL should be fully on the application side.
How strictly should we apply this rule? Is it optimal?
Why we should avoid logic in DB/SP?
Many applications avoid logic in stored procedures but generally move the logic into the application layer.
As Khalid Elsawaf commented (this is something I, too, had experienced - sounds very familiar!):
Anyone who has ever had to debug a 3000 line stored procedure will agree that business logic should be as far away from the database as possible!
As Nozim Mehrubonov commented (in response to that Vedran B. provided example of writing tests and refactoring since many applications he writes consist of SPs):
Well , how would you approach refactoring system that has most of it's business logic in stored procedures in Oracle or MS SQL?)
As Nuno Anjos commented:
yes database is a detail and you should never put your business logic on it
Systems that are DB agnostic are much easier to Migrate!
The downside of zero logic in the database
A while ago, Vedran B. wrote a post criticizing Uncle Bob’s perspective that “all logic” should be outside the databases is “one of the most damaging ideas“. The example he gave was the situation of a database with millions of records and pulling out all the logic from the database into the application side. (As a side note, Vedran writes many articles about databases and performance on LinkedIn).
Furthermore, Slobodan Tanasić had commented on older posts (sorry, I can’t remember the comment links, but I’ll add the links if I find them) on the performance inefficiency (and development time, maintenance issues) if we were to do reporting logic with hashmaps compared to just using the database JOIN functionality. Would anyone do this? Uncle Bob’s code samples on GitHub show loading data from multiple repositories and performing in-memory joins (in production, this would cause bad performance!)
Slobodan Tanasić also commented:
In addition to performance, there are also other -ilities solved best by using what relational databases have to offer: concise and maintainable declarative code, advanced data processing features already implemented (joins, different index types, aggregations, analytic/window functions, roll ups), auditing, text searching and so on.
What happens when we scale?
Richard Smith commented also about two pitfalls. One pitfall, where he noted the “breakdown“ was the need for transactionality and working at scale - “Where I find it breaks down a bit is when you want to take advantage of transactions“ & “The database also becomes more than just a detail when you start to work at scale.“
The database also becomes more than just a detail when you start to work at scale. I always felt that the Clean Code/Clean Architecture guys are talking about small systems working with small data, where you can just ignore issues of scale. But in reality there are times when you need to structure the queries you make to your data store, or the relationships between domain objects, a certain way for performance reasons.
Can we just simply swap databases?
Val Tórtola Luis commented that databases are not interchangeable:
Databases are not interchangeable, the query access patters of your application matters and a pattern that is optimal in one database, could be quite unacceptable in other.
It depends on “write” side versus “read“ side
In the article Introduction to CQRS, the author distinguishes between the “write“ (command) side, versus the “read“ (query side). In the illustration, we can see:
The command side is decoupled from the database, via the domain model and the repositories
The query side is (implementation-wise) NOT decoupled from the database, because it just has a “thin data layer“ and the majority of the work is done by the DB
Michaël Azerhad also explained the distinction between the “write” side and the “read” side:
In the write side, favor putting logic with your programming language but when it makes sense.
If you lose at RAM and Performance points, favor DB, especially if they are made to compute that case.
In the read side, favor database, they can run queries 1000 times faster than your programming language.
Read queries are complex by nature.
A query of 50 lines should not shock.
Now, I’ll share my perspectives with you
Above I’ve written a spectrum of perspectives. Now I’ll write my thoughts:
Firstly, I will show that the stated goal in Clean Architecture (that the goal of architecture is to minimize maintenance costs) is too narrow. Instead, I propose a multi-factor model which takes a weighted sum of quality characteristics and seeks to optimize that “total quality“, subject to constraints. Maintainability is one of the quality characteristics, but there are several others.
Secondly, I will show that the premise that “the database is a detail” (and not part of the architecture) is faulty! The argument is not whether or not we should or shouldn’t put business logic in the database. Instead, the argument is that the choice of database is one of the important structural decisions which affects total system quality; hence it is an architectural decision - it is part of architecture!
Thirdly, I will address a key proposition in Clean Architecture - that all behavior should go only into Use Cases and Entities, and never in the Database. I will demonstrate that this can result in SUBOPTIMAL quality (from both performance and maintenance perspectives). I will refer to Uncle Bob’s GitHub code samples to demonstrate the suboptimality.
Fourthly, I will examine Clean Architecture concerning a subset of quality characteristics defined by ISO/IEC 25010. I will outline where the principle of treating the DB as a detail leads to optimal and suboptimal results. I will also examine the downsides of deferring the database decision. I argue that it is neither realistic in many applications nor desirable.
Lastly, I will present my current decision-making framework and the general guidelines I use to make decisions when, where, and why I choose to put business logic in Use Cases and Entities versus Databases. I will also explain why we shouldn’t defer the DB decision.
But if we conclude that it’s suboptimal to place all behavior on the application side, then that behavior can’t be unit tested. So then what happens to testability and refactorability? We touch upon that topic here, but we’ll expand in an upcoming article.