In the last few posts [3][4][5], we learn what Isolation and Consistency guarantees are. In this post, we will go one step further and understand what is the difference between them and how they are used together.
Review what is Isolation guarantee
The Isolation guarantee defines how the states of concurrent transactions are visible to others. The goal of Isolation guarantee is to prevent a transaction to read and write data from other concurrent transactions that may be temporary or uncompleted, or maybe aborted.
The most strict Isolation level — Serializability — guarantees that concurrent transactions are executed in a way that the outcome (the database states) is equivalent to that as if they are executed serially.
In order to trade off for a better performance, some weaker Isolation levels than Serializability have been introduced. The weaker the Isolation level is, the more concurrent anomalies the system will be vulnerable to.
We learnt that the traditional Isolation level definitions are problematic when it is applied to replicated database systems. Concurrent anomalies such as stale read, immortal write, and causal reverse are possible in a replicated database system that only guarantees One-Copy Serializability.
Isolation levels that are higher than Serializability have been introduced. In this post, however, we will revisit these higher Isolation levels and showcase its relationship with Consistency levels.
Keep reading!
Review what is Consistency guarantee
The notion of Consistency has been mentioned in the context of database transactions (ACID properties) and distributed systems (CAP theorem). In the last post[5], we learnt that the Consistency in ACID is application semantic specific, not really a guarantee that can be provided by the database systems. When discussing Consistency levels, we are really referring to the Consistency guarantee of a distributed system, in the context of CAP theorem.
The Consistency in CAP defines how predictable when the clients read or write data from the system. The goal of Consistency levels is to set up the expectation between distributed systems and their clients.
The most strict Consistency level — Strict Consistency — guarantees that every read will receive the most recent (in real) write or an error. For example, we say a system is not consistent if two clients execute read operations at the same point of time but receive different states, or the states do not reflect the most recent writes.
In the example, the criteria of calling a system is consistent is very strict. In practice, we define many weaker consistency levels such as Sequential Consistency and Eventual Consistency to trade off for a better performance.
When Isolation meets Consistency
As you may be aware of, the contexts are slightly different when we describe Isolation and Consistency. Isolation is described in the context of transactions which consists of one or more read and write operations. On the other hand, Consistency is described in the context of atomic read or write operations.
Why are they different? It is because the fundamental problem sets of two concepts are different. The problem set of Isolation is concurrent transactions, on the other hand, the problem set of Consistency is any read and write operations of any distributed systems.
Historically, the Consistency guarantee is researched in the multi-thread shared memory model, and the model can be easily extended to distributed systems such as distributed databases or distributed cache, by mapping the clients as threads, and the distributed system as the shared memory.
For a single server database system that supports transactions, Consistency is a trivial guarantee, our focus is mostly on Isolation. For a distributed system (not necessarily a database system) that provides no transaction support, we only need to consider Consistency. However, for a database system that is both distributed and transactional, we need to consider both the Consistency and Isolation.
In our existing model for Consistency, a unit of work is a read or write operation. In this post, we extend it further and consider a unit of work to be a transaction that consists of one or more read or write operations. From diagrams, a transaction is annotated as an “outer” rectangle that encapsulates one or more “inner” rectangles that represent read or write operations.
Difference of Consistency and Isolation
What is the difference between Consistency and Isolation? Let’s first discuss the extreme case where a system provides only Consistency or only Isolation guarantee.
Consistency without Isolation
Firstly, a system that provides only Consistency guarantee but no Isolation guarantee.
For example, in the following diagram, we have two clients who execute 3 transactions concurrently. Both transaction T1 and T2 read the value of X and increase the value by 1 and write back. Transaction T3 reads the value of X.
If the system guarantees Consistency but no Isolation, we may see an execution that follows this read time order:
- T2 read X = 0
- T1 read X = 0
- T1 write X = 1
- T2 write X = 1 (lost update)
- T3 read X = 1
The execution doesn’t violate the Consistency guarantee at all, since every read receives the most recent writes.
However, it breaks the Isolation guarantee of transactions because there’s a lost update anomaly — the write from T1 is overwritten by the write from T2.
Isolation without Consistency
Secondly, a system that provides only Isolation guarantee but no Consistency guarantee.
If we execute the same example in such a system. One execution could be:
- T1 read X = 0
- T1 write X = 1
- T2 read X = 1
- T2 write X = 2
- T3 read X = 0 (stale read)
The execution doesn’t violate the Isolation guarantee at all, since concurrent transactions T1, T2 and T3 are executed as if they are executed serially.
However, it breaks the Consistency guarantee because T3 still receives an old value of X = 0 even if T2 and T3 are executed serially from the same client. If you still recall our previous post [4], it is an instance of stale read anomaly — the execution of T3 “time-traveled” back to a time point before the execution of T1.
The combination of Consistency and Isolation
Only when the distributed database system provides both Consistency and Isolation guarantee, the correctness of the execution is guaranteed.
As you can see from the following diagram, there’s no lost update anomaly nor stale read anomaly.
Difference of Consistency and Isolation
So what is the difference between Consistency and Isolation?
The bottomline is:
- Isolation puts limitations on when and how writes of one transaction are visible by other concurrently executed transactions.
- Consistency puts limitations on when and how writes are visible by any clients of a distributed system.
- The problem sets are different. One is for distributed systems, one is for database transactions.
- The problem sets can be overlapped — i.e. distributed database systems that support transactions. In that case, if write operations are executed in the context of transactions, when and how it would be visible is determined by both Isolation and Consistency guarantees.
What can make things more complicated is — there are a few Isolation and Consistency levels. Different combinations of them provide different expected behaviors. These differences may be very subtle to distinguish.
The combination of different Isolation and Consistency levels
In fact, in one of our previous posts, we have already discussed a subset of the problem under this section.
In the context of a single server database system, we claimed that Serializability is the highest level of Isolation guarantee. However, in the context of distributed database systems, we discussed some “higher” Isolation levels, i.e. One-Copy Serializability, Strict Serializability and some others in between.
Indeed the Serializability is the highest Isolation level. These “higher” Isolation levels are nothing but a mix of Isolation and Consistency guarantees. They all provide the same level of Isolation guarantee — Serializability. In addition to that, they provide different levels of Consistency guarantees — it is a must have to extend database transactions to distributed database systems.
More specifically, a system that provides “One-Copy Serializability” provides sequential Consistency. If you still recall, the “One-Copy Serializability” is built on top of Serializability. It guarantees that any reads will see the most recent writes from the previous transactions in the equivalent serial order. This definition exactly satisfied the requirement of sequential Consistency: all writes appear to take place in some order, and all threads agree on the order.
A system that provides “Strict Serializability” provides Linearizable Consistency, as it guarantees that if transaction X is executed before transaction Y in real time, transaction X will be placed before transaction Y in the equivalent serial order. This satisfied the real time constraint that is enforced by the Linearizability.
Other “higher” levels of Isolation can be mapped to this model as well. A system that provides “Strong Session Serializability” provides Linearizability for transactions in the same session, but Sequential Consistency for transactions across different sessions. A system that provides “Strong Write Serializability” provides Linearizability for write transactions, and Sequential Consistency for read transactions. A system that provides “Strong Partition Serializability” provides Linearizability for transactions within a single partition, but Sequential Consistency for transactions across different partitions.
We can further extend the idea to other Isolation levels. The Snapshot Isolation is a weaker Isolation level than Serializability as it is vulnerable to write skew anomaly. In this paper [2], Isolation levels such as “Strong Snapshot Isolation”, “Strong Session Snapshot Isolation” have been described. If you understand what we have discussed so far, can you guess what level of Isolation and Consistency these terms are suggesting?
As the terms suggested, the “Strong Snapshot Isolation” provides the combination of Linearizable Consistency and Snapshot Isolation. And the “Strong Session Snapshot Isolation” provides a Snapshot Isolation, and on top of that, Linearizable Consistency for transactions in the same session but Sequential Consistency for transactions across sessions.
We can keep extending them and build tens of different terms for different combinations. But you get the idea. In practice, we shouldn’t need so many different combinations. But conceptually, you know it is possible, since you understand the difference between the two guarantees.
Summary
It is not easy for an application developer or even a database developer to fully understand the difference between Isolation and Consistency and make wise decisions when choosing database implementation and the proper correctness levels.
The complexity is due to many reasons. First of all, the two correctness guarantees are conceptually overlapped in the context of concurrent transactions. Secondly, the word “Consistency” has been used in both ACID properties of transactions as well as CAP theorems. But the existing publications failed to provide the clear distinction between them. Lastly, database vendors often mix terminologies of Isolation and Consistency, e.g. “Strong Snapshot Isolation”, and these terms are not consistent across documents from different vendors!
As an application developer, what can we do? The recommendation is that you should equip yourselves with the knowledge that you learn from here, the next time you read the vendor documents, try to break apart the terms into Consistency level and Isolation level — just like how we did for “Strong Snapshot Isolation”. Only after that, trade off between the performance and correctness (both Isolation and Consistency) requirements of your application.
Reference
- http://dbmsmusings.blogspot.com/2019/08/an-explanation-of-difference-between.html
- http://www.vldb.org/conf/2006/p715-daudjee.pdf
- https://seanhu93.medium.com/revisit-database-isolation-863b3ca06f5f
- https://seanhu93.medium.com/review-isolation-levels-in-replicated-database-systems-431fc27b0644
- https://seanhu93.medium.com/understand-consistency-levels-in-distributed-systems-d63633e41a1e
