Preface

On 2022-09-22, JDK19 was released. Its biggest highlight was support for virtual threads, adding another powerful member to the lightweight thread family. Virtual threads free the JVM from relying on the operating system to schedule threads, allowing the JVM itself to schedule them. In fact, Sun implemented JVM-scheduled threads in its virtual machine on the Solaris operating system early on, but considerations of complexity and maintainability eventually led back to the operating-system-scheduled model.

“长安归来锦衣客，昨日城南起新宅。” (A traveler returns from Chang'an in splendid robes; yesterday, a new home rose south of the city.) Looking back over this journey, the concepts surrounding multithreading are dazzlingly numerous, and explanations online are countless, yet I have always felt that an overall perspective was missing. To address this, I have systematically traced the evolution of multithreading in Java, hoping it will help you master the subject.

What this article does not cover:

1 No detailed explanation of how particular technologies work, no source code analysis, and no diagrams. If you find concepts or technologies here that interest you, you can search for them yourself 2 No coverage of libraries and frameworks that support concurrency, such as Quasar, Akka, and Guava

What This Article Covers

1 Discuss the history of multithreading in the JDK 2 Explain the functions, principles, and background of certain technologies in that evolution, and the problems they solved 3 Share the author's views on certain technologies; readers with different views are welcome to discuss them in the comments

Milestones

As usual, let's start with a summary table. It outlines the core threading concepts across successive JDK versions. Here is a perhaps imperfect analogy: the evolution of multithreading can be compared to cars, progressing through the manual transmission era (JDK1.4 and earlier), the automatic transmission era (JDK5–JDK18), and the autonomous driving era (JDK19 and later). This analogy is simply meant to show readers that multithreading became easier to manage after JDK5. After JDK19, the improvement went beyond ease of use, bringing a qualitative leap in performance for I/O-intensive services.

The Manual Transmission Era

I call JDK1.4 and earlier the “manual transmission” era of multithreading, also known as the native multithreading era. Thread operations were still relatively low-level, and no thread pools were available. Developers had to write their own utilities to avoid wasting resources through frequent thread creation, and manually lock shared resources. This era also gave rise to many excellent multithreading frameworks, the most famous of which was adopted by JDK5.0.

JDK 1.0 Thread and Runnable

JDK1.0, released in month 1 of 1996, established Java's fundamental threading model from the outset. This model underwent no substantive changes through subsequent fixes and refinements, making it a sound design that has stood the test of time. Preemptive and cooperative scheduling are two common ways to schedule processes/threads. Preemptive scheduling is well suited to operating systems: the operating system allocates time slices to different processes and can reclaim resources from a process that remains unresponsive for a long time, or even forcibly stop it. Cooperative scheduling requires processes to release resources voluntarily and proactively. Under this approach, a long-running thread may cause all other threads that need the CPU to ”starve”. The Java HotSpot virtual machine uses preemptive scheduling, so Java adopted preemptive thread scheduling from the beginning. In JDK 1.0, threads were mainly created by extending the Thread class or implementing the Runnable interface. A thread was started by calling the object's start method, with the code requiring parallel execution placed in the run method. Coordination and communication between threads relied on blunt methods such as stop/resume/suspend.

How should we explain the concepts of stop/resume/suspend? They mean that the main thread can directly call a child thread's methods to stop, pause or resume it. If you used a portable cassette player as a child, it had three buttons: stop, pause and resume. Imagine listening to 3 portable cassette players at once. The three players are three child threads, and you are the main thread, free to control when these three devices start and stop.

This mechanism has a fatal problem: it is prone to deadlock. This happens when thread A locks a resource and is suspended by the main thread before releasing it (the suspend method does not release locks). If thread B then wants to acquire that resource, it must wait for thread A to resume execution (resume) and release the resource; otherwise, it will never acquire it, resulting in deadlock.

JDK 1.2

The heavy-handed stop/resume/suspend mechanism was prohibited in this version, giving way to approaches such as wait/notify/sleep that allow multiple threads to coordinate their actions. It is worth mentioning that atomic objects, AtomicityXXX, had already been designed in this version, mainly to address the non-atomic nature of i++. The addition of ThreadLocal and Collections expanded the ways multithreading could be used. Because of these two technologies, I call this Java's turbocharged era.

ThreadLocal

ThreadLocal is a lock-free approach that allows multiple threads to share objects that are not thread-safe. It cannot solve problems such as “increasing or decreasing bank balances or inventory.” It excels at safely executing the methods of utility classes by using separate copies. Typical examples include SimpleDateFormat and database connections. Its design is particularly ingenious. Imagine designing it yourself: a straightforward approach would be to maintain a global, thread-safe Map inside ThreadLocal, with threads as keys and shared objects as values. This design has a drawback: memory leaks. As more threads are added, the Map grows without bound. To resolve the memory leak, the Map would have to be cleaned up when threads terminate, which would require stronger GC capabilities. Clearly, the cost would outweigh the benefit. ThreadLocal was therefore designed so that the Map is held by Thread itself, rather than by ThreadLocal. The keys are ThreadLocal variables, and the values are their values. Each Thread stores all the ThreadLocal variables it uses in this Map (this design also gives rise to other issues that are not discussed here; interested readers can search for them).

Collections

The Collections utility class was designed in this version. It wraps thread-safe collections such as SynchronizedList. In an era when the only thread-safe collections were Hashtable, Vector, Stack, and the like, its arrival was significant for its time. The basic idea behind Collections is: I wrap your non-thread-safe collections to make them thread-safe, so upgrading and adapting your existing code does not take much time; you only need to initialize collections using the methods I provide when you create them. This is rather like automotive turbocharging, which increases an engine’s power and torque without changing its displacement. Java’s turbocharging era had arrived ^\_^

The Automatic Transmission Era

JDK 5.0

Import the Concurrency Package

Doug Lea, whose Chinese name is 道格·利, is a university teacher in the United States and a legendary figure. J.U.C is his creation. Before JDK1.5, synchronized was our only option for synchronizing concurrent access to code. At the time, synchronized had not yet been well optimized and performed poorly, and thread control was limited to Object's wait and notify methods. Doug Lea then submitted the JSR-166 proposal to the JCP. Before submitting JSR-166, he had already been using code with functionality similar to the J.U.C package for over three years. That code was the prototype for J.U.C.

J.U.C provides several broad categories of tools, including atomic objects, locks and utilities, thread pools, and thread-safe containers. Developers can flexibly use any of these capabilities to build their products, whether using ReentrantLock to construct underlying frameworks or directly using ready-made tools and containers to write business code. From a historical perspective, J.U.C could indisputably be called a “cutting-edge technology product” in 2004. It made an enormous contribution to Java’s adoption. Java’s automatic-transmission era had arrived: just as automatic cars lower the barrier to driving, J.U.C greatly lowered the barrier to using multithreading for programmers. This was a product that ushered in an era.

Of course, J.U.C also has some flaws:

High CPU overhead：If spinning CAS attempts remain unsuccessful for a long time, they impose very high overhead on the CPU.

Solution: Some parts of JUC limit the number of CAS spins, such as SynchronousQueue in BlockingQueue.

The ABA Problem: If a value starts as A, changes to B, and then changes back to A, a CAS check will find no change even though it has actually changed. This is the ABA problem. In most cases, the ABA problem does not affect the correctness of concurrent program execution.

Solution: Add a version number to each variable and increment it by 1 each time it changes, so A —> B —> A becomes 1A —> 2B —> 3A. Java provides AtomicStampedReference to address this. AtomicStampedReference marks objects with a version stamp by wrapping an [E,Integer] tuple, thereby avoiding the ABA problem.

Can only guarantee atomic operations on a single shared variable：The CAS mechanism guarantees only the atomicity of operations on a single variable; it cannot guarantee the atomicity of an entire code block.

Solution: For example, if you need to ensure that 3 variables are updated together atomically, you have to use Synchronized. You can also consider wrapping multiple variables in AtomicReference to handle situations involving multiple shared variables.

Clarifying the Java Memory Model

This version of the JDK redefined the Java Memory Model. Before this, common memory models included the sequential consistency model and the happens-before model. Under sequential consistency, the order of program execution exactly matches the order shown in the code. This is difficult to guarantee on modern multicore CPUs that optimize instruction execution. Moreover, guaranteeing sequential consistency severely restricts the JVM's runtime code optimizations.

However, the happens-before relationship specified by this version of JSR 133 allows flexibility in instruction execution order:

Within the same thread, in code execution order (that is, the order defined by code semantics), an earlier operation happens-before a later operation. Unlocking a monitor object happens-before a subsequent lock on the same monitor object. A write to a volatile field happens-before a subsequent read of that field. Starting a thread (calling the thread object's start() method) happens-before any other operation in that thread. All operations in a thread happen-before any other thread calls join() on that thread. If operation A happens-before B, and operation B happens-before C, then operation A happens-before C.

In terms of memory allocation, separating each thread’s working memory from main memory gives the JVM considerable room to optimize instruction execution within threads. Variables in main memory can be copied into a thread’s working memory for independent execution, and once execution finishes, the results can be flushed back to main memory at some point: But how can data consistency between threads be ensured? The approach provided by the JLS (Java Language Specification) is that, by default, data consistency cannot be guaranteed at every instant, but it can be achieved by using the keywords synchronized, volatile, and final, whose semantics have been strengthened.

JDK 6.0 synchronized optimizations

As the “eldest son of the Republic,” the synchronized keyword was overshadowed by ReentrantLock in version 5.0. It had to win a round back in this version, so JDK 6.0 introduced some lock optimizations, such as lock spinning, lock elimination, lock coarsening, lightweight locking and biased locking. These optimizations illustrate the idea of “fine-grained management.” Before optimization, an object locked with synchronized had only two states: unlocked and locked (with a heavyweight lock). After optimization, locks have 4 states, ordered from lowest to highest: unlocked, biased, lightweight and heavyweight. These states gradually escalate as contention increases, but cannot be downgraded, with the aim of improving the efficiency of acquiring and releasing locks (personally, I think it was actually too complicated, and the JVM developers were daunted by it).

This optimization finally let synchronized hold its head high; from then on, no one was allowed to say it performed worse than ReentrantLock. But the best was yet to come: biased locking was deprecated in JDK 15 (─.─||). In my view, synchronized is at a disadvantage because it is merely a keyword, with the JVM responsible for its underlying operations. The JVM has to “guess” which locking approach best suits the application. ReentrantLock is different: it hands that decision directly to the programmer. If you want fairness, use a fair lock; if you want unfairness, use a nonfair lock. At the design level, this is something of a shortcut, but it also provides flexibility.

JDK 7.0 Fork/Join Framework

Fork/Join was also a relatively advanced offering when it emerged. Its core strength lies in supporting recursive task decomposition and then combining the results of the individual tasks. Yet it is a technology that feels both familiar and unfamiliar: familiar because it is used in all sorts of places, such as CompletableFuture and Stream, which we will discuss next in JDK8.0; unfamiliar because we seem to use it rarely in business application development.

Some even consider it of little practical use. In my view, if you develop software for business requirements, it offers little value because you will hardly ever use it: data warehouse tools handle large data volumes, and thread pools can serve in other scenarios; if you develop middleware frameworks, it does have value and may come in handy. Few people on the Chinese internet question this technology, but people abroad are already discussing it. Those interested can follow the link to read more Is the Fork-Join framework in Java broken?

JDK 8.0

The release of this version was a milestone for Java, so much so that it now accounts for more than half of all Java programs running worldwide. However, its multithreading updates were not as revolutionary as those in JDK5.0. Besides adding some atomic objects, this version's most notable changes were the following two updates.

CompletableFuture

There are many introductions to CompletableFuture online, mostly explaining how it works and how to use it. But one question kept puzzling me: with so many thread pool tools already available, why was CompletableFuture introduced, and what pain point does it address? What exactly is its core advantage? If you have thought about it, I’m sure you have asked the same question. Don’t worry—I’ve found the answer for you.

Conclusion: CompletableFuture's core strength is Task orchestration。CompletableFuture inherits the features of the Future interface, including the ability to execute tasks concurrently; alternatives exist for all these capabilities. Its task orchestration capabilities, however, are irreplaceable. Its core APIs, including those for building task chains and combining task results, are designed for task orchestration. Thus, the JDK introduced this framework in this version primarily to address the increasingly pressing need for task orchestration in business application development.

One final note: CompletableFuture is implemented using the Fork/Join framework under the hood.

Stream

Clean Architecture mentions three programming paradigms: structured programming (procedural programming), object-oriented programming, and functional programming. Stream is one manifestation of functional programming in Java. In my view, mastering Stream is an essential step for junior programmers advancing to the intermediate level. Stream will certainly feel unfamiliar at first, but once you become comfortable with it, it will open up a new way of thinking about programming. As developers, we often confuse three concepts—functional programming, Stream, and lambda expressions—assuming they all refer to the same thing. Here is my understanding:

•Functional programming is a programming paradigm practiced in various programming languages

•Stream is a new feature in JDK8.0. It can also be understood as a newly introduced concept intended to accommodate functional programming, enabling functional programming on collections through streams

•A lambda expression is an anonymous function that lets you express functional programming more concisely and efficiently

After all that, what is the relationship between Stream and multithreading? The parallel methods in Stream are implemented using the Fork/Join framework underneath. Effective Java offers the related advice to “use caution when making streams parallel,” because all parallel operations run in a common Fork/Join pool. Abnormal execution of one pipeline may impair the performance of other, unrelated parts.

JDK 9.0

Improve the Lock Contention Mechanism

Lock contention limits the performance of many multithreaded Java applications. The new lock contention mechanism improves Java object monitor performance and has been validated by various benchmarks (such as Volano), which can estimate the JVM's maximum throughput. In practice, the new lock contention mechanism achieved excellent results across 22 different benchmarks. If the new mechanism can be adopted in Java 9, application performance will improve greatly. Simply put, when multiple threads contend for a lock, before optimization: all late-arriving threads follow the same standard procedure to wait for the lock. After optimization: when the JVM identifies certain optimizable scenarios, it directly lets late-arriving threads take a “VIP lane” to compete for the lock.

For a detailed explanation, please refer to: Contended locks explained – a performance approach

Reactive Streams

Reactive Streams is a standard for processing asynchronous data streams with non-blocking backpressure, providing a minimal set of interfaces, methods, and protocols to describe the necessary operations and entities.

What is non-blocking backpressure? 背压 is shorthand for back pressure. Put simply, a producer pushes data to a consumer; when the consumer cannot keep up, it informs the producer, which then reduces its production rate. The simplest way to implement this mechanism is through blocking, returning pressure information directly when data is pushed. A non-blocking implementation makes the design more complex while improving performance. PS: I feel 背压 is not a good translation; you cannot infer its meaning from the characters alone. Would 反压 (“counterpressure”) be better? ^\_^

To address the risk of consumers crashing under immense resource pressure (pressure), the rate of the data stream must be controlled through flow control (flow control), preventing a fast stream from overwhelming its destination. This requires back pressure (back pressure): producers and consumers need to interoperate through a backpressure mechanism. This mechanism must be asynchronous and non-blocking. If it is synchronous and blocking, the producer must wait while the consumer processes data, causing performance problems.

Reactive Streams defines a set of entities, interfaces, and interoperability methods, providing a standard for implementing nonblocking backpressure. Third parties follow this standard to implement specific solutions, with common examples including Reactor, RxJava, Akka Streams, and Ratpack.

JDK 10 Thread-Local Control

Safepoints and Their Limitations:

A Safepoint is a mechanism in the HotSpot JVM that makes all application threads stop. To perform certain low-level tasks, the JVM must Stop The World, pausing all application threads. But it cannot simply stop them abruptly; instead, it sends application threads a signal telling them that they need to stop. When an application thread reaches a Safepoint, it follows the instruction and responds. That is why it is called a Safepoint. The word “safe” emphasizes that the JVM is about to perform certain global operations safely, hence the name.

Global safepoint operations include the following: 1, garbage collection pauses 2, code deoptimization. 3, flushing the code cache. 4, class redefinition (such as hot updates or instrumentation). 5, biased lock revocation. 6, various debugging operations (such as: deadlock detection or stack trace dumps).

However, getting all threads to stop at their nearest safepoint itself takes considerable time. And isn't stopping all threads a little too rash and heavy-handed? To address this, Java 10 introduced an approach that does not require stopping all threads: Thread Local Handshake.

For example, the following scenarios can be handled without stopping all threads: 1, Revoking biased locks. This requires stopping only a single thread to revoke the biased lock, without stopping all threads. 2, Reducing the overall impact of various types of serviceability queries on VM latency; for example, obtaining stack traces for all threads on a VM with a large number of Java threads can be a slow operation. 3, Performing safer stack trace sampling by reducing reliance on signals. 4, Using what is called asymmetric Dekker synchronization to eliminate some memory barriers through handshakes with Java threads. For example, the “conditional card mark code” used in G1 and CMS will no longer need those “memory barriers.” This allows the “write barrier” emitted by G1 to be optimized, and the branches that attempt to avoid “memory barriers” can also be removed.

JDK 15 Disables and Deprecates Biased Locking

Why deprecate biased locking? The performance gains biased locking brought in the past no longer seem as significant today. Applications that benefit from biased locking tend to use early Java collection APIs (JDK 1.1), namely Hashtable and Vector, which synchronize on every access. JDK 1.2 introduced unsynchronized collections for single-threaded scenarios (HashMap and ArrayList), and JDK 1.5 introduced higher-performance concurrent data structures for multithreaded scenarios. This means that applications benefiting from biased locking because of unnecessary synchronization may see substantial performance improvements if their code is updated to use newer classes. In addition, applications built around thread pool queues and worker threads often perform better with biased locking disabled.

The following implementation uses the Hashtable and Vector APIs: java.lang.Classloader uses Vector java.util.Properties extends Hashtable java.security.Provider extends Properties java.net.URL uses Hashtable java.net.URConnection uses Hashtable java.util.ZipOutputStream uses Vector javax.management.timer.TimerMBean has Vector on the interface

The Era of Autonomous Driving

Virtual threads have brought Java into the self-driving era. Many languages have technologies similar to “virtual threads,” including Go, C#, Erlang, and Lua; they call them “coroutines.” This time, Java has added no new keywords or even new concepts. Virtual threads are much easier to understand than goroutines or coroutines: the name alone gives you a rough idea of what they do.

JDK 19 Virtual Threads

In traditional Java, threads map to operating system threads on a 1:1 basis. Creating and switching threads is expensive, and operating system constraints limit how many can be created. Under high concurrency, it is impossible to create a thread for every request. Thread pools can alleviate this problem by reducing thread creation overhead, but they cannot increase the number of threads available. If there are 2000 concurrent requests and the thread pool has only 1000 threads, only 1000 requests can be processed at a time. The remaining 1000 requests cannot be processed immediately; they can either be rejected or made to wait until a thread becomes available.

Before virtual threads, the approach was to use an asynchronous style. Many frameworks already implemented asynchronous concurrent programming (such as Spring5's Reactor), achieving higher availability through thread sharing. The principle is to reduce thread switching through thread sharing, lowering overhead while also avoiding blocking: threads are used only when the program is executing, and are not occupied when the program needs to wait. The asynchronous style does bring considerable improvements, but it also has drawbacks. Most asynchronous frameworks use chained calls, dividing a program into many steps, each of which may execute on a different thread. You can no longer use familiar concurrent programming APIs such as ThreadLocal, or errors may occur. The programming style also changes substantially, becoming much more complicated than traditional programming, with a steep learning curve. You may also need to adapt many existing modules in the project to the asynchronous model.

Virtual threads work much like some asynchronous frameworks: they also share threads, so naturally they do not need pooling. When using them, you can think of virtual threads as available in unlimited abundance: create as many as you want without worrying about problems. Moreover, virtual threads support debugging and work with Java monitoring tools, which is very important. Virtual threads will greatly reduce your program's memory footprint. All I/O-intensive applications, such as Web Servers, can achieve a substantial increase in I/O throughput on the same hardware. Previously, 1 GB of memory could handle 1000 simultaneous requests. With virtual threads, according to the official claims, it can easily handle 100 ten-thousands of concurrent requests. Whether a particular business scenario can sustain that still depends on load testing, but even discounting that claim, 10 ten-thousands should be easily achievable, at no cost to you; you might not even need to change your code. This is because virtual threads let you retain the traditional programming style of one thread per request. You use virtual threads just like threads, and your program needs very few changes. Virtual threads introduce no new syntax either, so the learning and migration costs are extremely low.

Virtual threads also use the Fork/Join framework under the hood.