Java garbage collection: The 10-release evolution from JDK 8 to JDK 18

Introducing garbage collection, metrics, and trade-offs

The component of the HotSpot JVM that manages the application heap of your application is called the garbage collector (GC). A GC governs the whole lifecycle of application heap objects, beginning when the application allocates memory and continuing through reclaiming that memory for eventual reuse later.

At a very high level, the most basic functionality of garbage collection algorithms in the JVM are the following:

◉ Upon an allocation request for memory from the application, the GC provides memory. Providing that memory should be as quick as possible.

◉ The GC detects memory that the application is never going to use again. Again, this mechanism should be efficient and not take an undue amount of time. This unreachable memory is also commonly called garbage.

◉ The GC then provides that memory again to the application, preferably “in time,” that is, quickly.

There are many more requirements for a good garbage collection algorithm, but these three are the most basic ones and sufficient for this discussion.

There are many ways to satisfy all these requirements, but unfortunately there is no silver bullet and no one-size-fits-all algorithm. For this reason, the JDK provides a few garbage collection algorithms to choose from, and each is optimized for different use cases. Their implementation roughly dictates behavior about one or more of the three main performance metrics of throughput, latency, and memory footprint and how they impact Java applications.

◉ Throughput represents the amount of work that can be done in a given time unit. In terms of this discussion, a garbage collection algorithm that performs more collection work per time unit is preferable, allowing higher throughput of the Java application.

◉ Latency gives an indication of how long a single operation of the application takes. A garbage collection algorithm focused on latency tries to minimize impacting latency. In the context of a GC, the key concerns are whether its operation induces pauses, the extent of any pauses, and how long the pauses may be.

◉ Memory footprint in the context of a GC means how much extra memory beyond the application’s Java heap memory usage the GC needs for proper operation. Data used purely for the management of the Java heap takes away from the application; if the amount of memory the GC (or, more generally, the JVM) uses is less, more memory can be provided to the application’s Java heap.

These three metrics are connected: A high throughput collector may significantly impact latency (but minimizes impact on the application) and the other way around. Lower memory consumption may require the use of algorithms that are less optimal in the other metrics. Lower latency collectors may do more work concurrently or in small steps as part of the execution of the application, taking away more processor resources.

This relationship is often graphed in a triangle with one metric in each corner, as shown in Figure 1. Every garbage collection algorithm occupies a part of that triangle based on where it is targeted and what it is best at.

Figure 1. The GC performance metrics triangle

Trying to improve a GC in one or more of the metrics often penalizes the others.

The OpenJDK GCs in JDK 18

OpenJDK provides a diverse set of five GCs that focus on different performance metrics. Table 1 lists their names, their area of focus, and some of the core concepts used to achieve the desired properties.

Table 1. OpenJDK’s five GCs

Garbage collector	Focus area	Concepts
Parallel	Throughput	Multithreaded stop-the-world (STW) compaction and generational collection
Garbage First (G1)	Balanced performance	Multithreaded STW compaction, concurrent liveness, and generational collection
Z Garbage Collector (ZGC) (since JDK 15)	Latency	Everything concurrent to the application
Shenandoah (since JDK 12)	Latency	Everything concurrent to the application
Serial	Footprint and startup time	Single-threaded STW compaction and generational collection

The Parallel GC is the default collector for JDK 8 and earlier. It focuses on throughput by trying to get work done as quickly as possible with minimal regard to latency (pauses).

The Parallel GC frees memory by evacuating (that is, copying) the in-use memory to other locations in the heap in more compact form, leaving large areas of then-free memory within STW pauses. STW pauses occur when an allocation request cannot be satisfied; then the JVM stops the application completely, lets the garbage collection algorithm perform its memory compaction work with as many processor threads as available, allocates the memory requested in the allocation, and finally continues execution of the application.

The Parallel GC also is a generational collector that maximizes garbage collection efficiency. More on the idea of generational collection is discussed later.

The G1 GC has been the default collector since JDK 9. G1 tries to balance throughput and latency concerns. On the one hand, memory reclamation work is still performed during STW pauses using generations to maximize efficiency—as is done with the Parallel GC—but at the same time, it tries to avoid lengthy operations in these pauses.

G1 performs lengthy work concurrent to the application, that is, while the application is running using multiple threads. This decreases maximum pause times significantly, at the cost of some overall throughput.

The ZGC and Shenandoah GCs focus on latency at the cost of throughput. They attempt to do all garbage collection work without noticeable pauses. Currently neither is generational. They were first introduced in JDK 15 and JDK 12, respectively, as nonexperimental versions.

The Serial GC focuses on footprint and startup time. This GC is like a simpler and slower version of the Parallel GC, as it uses only a single thread for all work within STW pauses. The heap is also organized in generations. However, the Serial GC excels at footprint and startup time, making it particularly suitable for small, short-running applications due to its reduced complexity.

OpenJDK provides another GC, Epsilon, which I omitted from Table 1. Why? Because Epsilon only allows memory allocation and never performs any reclamation, it does not meet all the requirements for a GC. However, Epsilon can be useful for some very narrow and special-niche applications.

Short introduction to the G1 GC

The G1 GC was introduced in JDK 6 update 14 as an experimental feature, and it was fully supported beginning with JDK 7 update 4. G1 has been the default collector for the HotSpot JVM since JDK 9 due to its versatility: It is stable, mature, very actively maintained, and it’s being improved all the time. I hope the remainder of this article will prove that to you.

How does G1 achieve this balance between throughput and latency?

One key technique is generational garbage collection. It exploits the observation that the most recently allocated objects are the most likely ones that can be reclaimed almost immediately (they “die” quickly). So G1, and any other generational GC, splits the Java heap into two areas: a so-called young generation into which objects are initially allocated and an old generation where objects that live longer than a few garbage collection cycles for the young generation are placed so they can be reclaimed with less effort.

The young generation is typically much smaller than the old generation. Therefore, the effort for collecting it, plus the fact that a tracing GC such as G1 processes only reachable (live) objects during young-generation collections, means the time spent garbage collecting the young generation generally is short, and a lot of memory is reclaimed at the same time.

At some point, longer-living objects are moved into the old generation.

Therefore, from time to time, there is a need to collect garbage and reclaim memory from the old generation as it fills up. Since the old generation is typically large, and it often contains a significant number of live objects, this can take quite some time. (For example, the Parallel GC’s full collections often take many times longer than its young-generation collections.)

For this reason, G1 splits old-generation garbage collection work into two phases.

◉ G1 first traces through the live objects concurrently to the Java application. This moves a large part of the work needed for reclaiming memory from the old generation out of the garbage collection pauses, thus reducing latency. The actual memory reclamation, if done all at once, would still be very time consuming on large application heaps.

◉ Therefore, G1 incrementally reclaims memory from the old generation. After the tracing of live objects, for every one of the next few regular young-generation collections, G1 compacts a small part of the old generation in addition to the whole young generation, reclaiming memory there as well over time.

Reclaiming the old generation incrementally is a bit more inefficient than doing all this work at once (as the Parallel GC does) due to inaccuracies in tracing through the object graph as well as the time and space overhead for managing support data structures for incremental garbage collections, but it significantly decreases the maximum time spent in pauses. As a rough guide, garbage collection times for incremental garbage collection pauses take around the same time as the ones reclaiming only memory from the young generation.

In addition, you can set the pause time goal for both of these types of garbage collection pauses via the MaxGCPauseMillis command-line option; G1 tries to keep the time spent below this value. The default value for this duration is 200 ms. That might or might not be appropriate for your application, but it is only a guide for the maximum. G1 will keep pause times lower than that value if possible. Therefore, a good first attempt to improve pause times is trying to decrease the value of MaxGCPauseMillis.

Progress from JDK 8 to JDK 18

Now that I’ve introduced OpenJDK’s GCs, I’ll detail improvements that have been made to the three metrics—throughput, latency, and memory footprint—for the GCs during the last 10 JDK releases.

Throughput gains for G1. To demonstrate the throughput and latency improvements, this article uses the SPECjbb2015 benchmark. SPECjbb2015 is a common industry benchmark that measures Java server performance by simulating a mix of operations within a supermarket company. The benchmark provides two metrics.

◉ maxjOPS corresponds to the maximum number of transactions the system can provide. This is a throughput metric.

◉ criticaljOPS measures throughput under several service-level agreements (SLAs), such as response times, from 10 ms to 100 ms.

This article uses maxjOPS as a base for comparing the throughput for JDK releases and the actual pause time improvements for latency. While criticaljOPS values are representative of latency induced by pause time, there are other sources that contribute to that score. Directly comparing pause times avoids this problem.

Figure 2 shows maxjOPS results for G1 in composite mode on a 16 GB Java heap, graphed relative to JDK 8 for JDK 11 and JDK 18. As you can see, the throughput scores increase significantly simply by moving to later JDK releases. JDK 11 improves by around 5% and JDK 18 by around 18%, respectively, compared to JDK 8. Simply put, with later JDKs, more resources are available and used for actual work in the application.

Figure 2. G1 throughput gains measured with SPECjbb2015 maxjOPS

The discussion below attempts to attribute these throughput improvements to particular garbage collection changes. However, garbage collection performance, particularly throughput, is also very amenable to other generic improvements such as code compilation, so the garbage collection changes are not responsible for all the uplift.

One significant improvement early in JDK 9 was how G1 starts the old-generation collection lazily, as late as possible.

In JDK 8 the user had to manually set the time when G1 started concurrent tracing of live objects for old-generation collection. If the time was set too early, the JVM did not use all the application heap assigned to the old generation before starting the reclamation work. One drawback was that this did not give the objects in the old generation as much time to become reclaimable. So G1 would not only take more processor resources to analyze liveness because more data was still live, but also G1 would do more work than necessary freeing memory for the old generation.

Another problem was that if the time to start old-generation collection were set to be too late, the JVM might run out of memory, causing a very slow full collection. Beginning with JDK 9, G1 automatically determines an optimal point at which to start old-generation tracing, and it even adapts to the current application’s behavior.

Another idea that was implemented in JDK 9 is related to trying to reclaim large objects in the old generation that G1 automatically places there at a higher frequency than the rest of the old generation. Similar to the use of generations, this is another way the GC focuses on “easy pickings” work that has potentially very high gain—after all, large objects are called large objects because they take lots of space. In some (admittedly rare) applications, this even yields such large reductions in the number of garbage collections and total pause times that G1 beats the Parallel GC on throughput.

In general, every release includes optimizations that make garbage collection pauses shorter while performing the same work. This leads to a natural improvement in throughput. There are many optimizations that could be listed in this article, and the following section about latency improvements points out some of them.

Similar to the Parallel GC, G1 got dedicated nonuniform memory access (NUMA) awareness for allocation to the Java heap in JDK 14. Since then, on computers with multiple sockets where memory access times are nonuniform—that is, where memory is somewhat dedicated to the sockets of the computer, and therefore access to some memory can be slower—G1 tries to exploit locality.

When NUMA awareness applies, the G1 GC assumes that objects allocated on one memory node (by a single thread or thread group) will be mostly referenced from other objects on the same node. Therefore, while an object stays in the young generation, G1 keeps objects on the same node, and it evenly distributes the longer-living objects across nodes in the old generation to minimize access-time variation. This is similar to what the Parallel GC implements.

One more improvement I would like to point out here applies to uncommon situations, the most notable probably being full collections. Normally, G1 tries to prevent full collections by ergonomically adjusting internal parameters. However, in some extreme conditions this is not possible, and G1 needs to perform a full collection during a pause. Until JDK 10, the implemented algorithm was single-threaded, and so it was extremely slow. The current implementation is on par with the Parallel GC’s full garbage collection process. It’s still slow, and something you want to avoid, but it’s much better.

Throughput gains for the Parallel GC. Speaking of the Parallel GC, Figure 3 shows maxjOPS score improvements from JDK 8 to JDK 18 on the same heap configuration used earlier. Again, only by substituting the JVM, even with the Parallel GC, you can get a modest 2% to around a nice 10% improvement in throughput. The improvements are smaller than with G1 because the Parallel GC started off from a higher absolute value, and there has been less to gain.

Figure 3. Throughput gains for the Parallel GC measured with SPECjbb2015 maxjOPS

Latency improvements on G1. To demonstrate latency improvements for HotSpot JVM GCs, this section uses the SPECjbb2015 benchmark with a fixed load and then measures pause times. The Java heap size is set to 16 GB. Table 2 summarizes average and 99th percentile (P99) pause times and relative total pause times within the same interval for different JDK versions at the default pause time goal of 200 ms.

Table 2. Latency improvements with the default pause time of 200 ms

	JDK 8, 200 ms	JDK 11, 200 ms	JDK 18, 200 ms
Average (ms)	124	111	89
P99 (ms)	176	111	111
Relative collection time (%)	n/a	-15.8	-34.4

JDK 8 pauses take 124 ms on average, and P99 pauses are 176 ms. JDK 11 improves average pause time to 111 ms and P99 pauses to 134 ms—in total spending 15.8% less time in pauses. JDK 18 significantly improves on that once more, resulting in pauses taking 89 ms on average and P99 pause times taking 104 ms—resulting in 34.4% less time in garbage collection pauses.

I extended the experiment to add a JDK 18 run with a pause time goal set to 50 ms, because I arbitrarily decided that the default for -XX:MaxGCPauseMillis of 200 ms was too long. G1, on average, met the pause time goal, with P99 garbage collection pauses taking 56 ms (see Table 3). Overall, total time spent in pauses did not increase much (0.06%) compared to JDK 8.

In other words, by substituting a JDK 8 JVM with a JDK 18 JVM, you either get significantly decreased average pauses at potentially increased throughput for the same pause time goal, or you can have G1 keep a much smaller pause time goal (50 ms) at the same total time spent in pauses, which roughly corresponds to the same throughput.

Table 3. Latency improvements by setting the pause time goal to 50 ms

	JDK 8, 200 ms	JDK 11, 200 ms	JDK 18, 200 ms	JDK 18, 50 ms
Average (ms)	124	111	89	44
P99 (ms)	176	134	104	56
Relative collection time (%)	n/a	-15.8	-34.4	+0.06

The results in Table 3 were made possible by many improvements since JDK 8. Here are the most notable ones.

A fairly large contribution to reduced latency was the reduction of the metadata needed to collect parts of the old generation. The so-called remembered sets have been trimmed significantly by both improvements to the data structures themselves as well as to not storing and updating never-needed information. In today’s computer architectures, a reduction in metadata to be managed means much less memory traffic, which improves performance.

Another aspect related to remembered sets is the fact that the algorithm for finding references that point into currently evacuated areas of the heap has been improved to be more amenable to parallelization. Instead of looking through that data structure in parallel and trying to filter out duplicates in the inner loops, G1 now separately filters out remembered-set duplicates in parallel and then parallelizes the processing of the remainder. This makes both steps more efficient and much easier to parallelize.

Further, the processing of these remembered-set entries has been looked at very thoroughly to trim unnecessary code and optimize for the common paths.

Another focus in JDKs later than JDK 8 has been improving the actual parallelization of tasks within a pause: Changes have attempted to improve parallelization either by making phases parallel or by creating larger parallel phases out of smaller serial ones to avoid unnecessary synchronization points. Significant resources have been spent to improve work balancing within parallel phases so that if a thread is out of work, it should be cleverer when looking for work to steal from other threads.

By the way, later JDKs started looking at more uncommon situations, one of them being evacuation failure. Evacuation failure occurs during garbage collection if there is no more space to copy objects into.

Garbage collection pauses on ZGC. In case your application requires even shorter garbage collection pause times, Table 4 shows a comparison with one of the latency-focused collectors, ZGC, on the same workload used earlier. It shows the pause-time durations presented earlier for G1 plus an additional rightmost column showing ZGC.

Table 4. ZGC latency compared to G1 latency

	JDK 8, 200 ms, G1	JDK 18, 200 ms, G1	JDK 18, 50 ms, G1	JDK 18, ZGC
Average (ms)	124	89	44	0.01
P99 (ms)	176	104	56	0.031

ZGC delivers on its promise of submillisecond pause time goals, moving all reclamation work concurrent to the application. Only some minor work to provide closure of garbage collection phases still needs pauses. As expected, these pauses will be very small: in this case, even far below the suggested millisecond range that ZGC aims to provide.

Footprint improvements for G1. The last metric this article will examine is progress in the memory footprint of the G1 garbage collection algorithm. Here, the footprint of the algorithm is defined as the amount of extra memory outside of the Java heap that it needs to provide its functionality.

In G1, in addition to static data dependent on the Java heap size, which takes up approximately 3.2% of the size of the Java heap, often the other main consumer of additional memory is remembered sets that enable generational garbage collection and, in particular, incremental garbage collection of the old generation.

One class of applications that stresses G1’s remembered sets is object caches: They frequently generate references between areas within the old generation of the heap as they add and remove newly cached entries.

Figure 4 shows G1 native memory usage changes from JDK 8 to JDK 18 on a test application that implements such an object cache: Objects that represent cached information are queried, added, and removed in a least-recently-used fashion from a large heap. This example uses a Java heap of 20 GB, and it uses the JVM’s native memory tracking (NMT) facility to determine memory usage.

Figure 4. The G1 GC’s native memory footprint

With JDK 8, after a short warmup period, G1 native memory usage settles at around 5.8 GB of native memory. JDK 11 improved on that, reducing the native memory footprint to around 4 GB; JDK 17 improved it to around 1.8 GB; and JDK 18 settles at around 1.25 GB of garbage collection native memory usage. This is a reduction of extra memory usage from almost 30% of the Java heap in JDK 8 to around 6% of extra memory usage in JDK 18.

There is no particular cost in throughput or latency associated with these changes, as previous sections showed. Indeed, reducing the metadata the G1 GC maintains generally improved the other metrics so far.

The main principle for these changes from JDK 8 through JDK 18 has been to maintain garbage collection metadata only on a very strict as-needed basis, maintaining only what is expected to be needed when it is needed. For this reason, G1 re-creates and manages this memory concurrently, freeing data as quickly as possible. In JDK 18, enhancements to the representation of this metadata and storing it more densely contributed significantly to the improvement of the memory footprint.

Figure 4 also shows that in later JDK releases G1 increased its aggressiveness, step by step, in giving back memory to the operating system by looking at the difference between peaks and valleys in steady-state operations—in the last release, G1 even does this process concurrently.

The future of garbage collection

Although it is hard to predict what the future holds and what the many projects to improve garbage collection and, in particular, G1, will provide, some of the following developments are more likely to end up in the HotSpot JVM in the future.

One problem that is actively being worked on is removing the need to lock out garbage collection when Java objects are used in native code: Java threads triggering a garbage collection must wait until no other regions are holding references to Java objects in native code. In the worst cases, native code may block garbage collection for minutes. This can lead to software developers choosing to not use native code at all, affecting throughput adversely. With the changes suggested in JEP 423 (Region pinning for G1), this will become a nonissue for the G1 GC.

Another known disadvantage of using G1 compared to the throughput collector, Parallel GC, is its impact on throughput—users report differences in the range of 10% to 20% in extreme cases. The cause of this problem is known, and there have been a few suggestions on how to improve this drawback without compromising other qualities of the G1 GC.

Fairly recently, it’s been determined that pause times and, in particular, work distribution efficiency in the garbage collection pauses are still less than optimal.

One current focus of attention is removing one-half of G1’s largest helper data structure, the mark bitmaps. There are two bitmaps used in the G1 algorithm that help with determining which objects are currently live and can be safely concurrently inspected for references by G1. An open enhancement request indicates that the purpose of one of these bitmaps could be replaced by other means. That would immediately reduce G1 metadata by a fixed 1.5% of the Java heap size.

There is much ongoing activity to change the ZGC and Shenandoah GCs to be generational. In many applications, the current single-generational design of these GCs has too many disadvantages regarding throughput and timeliness of reclamation, often requiring much larger heap sizes to compensate.

Source: oracle.com

Continue reading

JVM Garbage Collection and Optimizations

Overview

When troubleshooting systems for performance-related issues, memory optimizations are a place that needs a deep analysis of what each system stores in the memory, how long those are stored, and access patterns. This post is to keep a note on the background information and valuable points to note in such an effort, specific to Java-based implementations as a deep understanding of the JVM behaviors is very beneficial in the process.

Java language provides much convenience to the developers by taking care of the memory management to a great extent letting the focus be on the rest of the logic. Still having a good understanding of how Java does this underneath, rationalize several best practices we follow in Java implementations and help design the programs better and think seriously on some aspects that can later lead to memory leaks and system stability in the long run. Java Garbage Collector has a big role in this been responsible for freeing up memory by removing memory garbage.

JVM

This information is widely available, yet I am summarizing here for reference in one place. :)

JVM enables Java code to run in hardware and OS independent manner. It operates on memory locations allocated for own process by the OS acting as another abstraction of a physical machine.

JVMs can be implemented based on the open standard as published at [1], widely known implementations been Oracle Hotspot JVM, almost the same open-source version OpenJDK, IBM J9,  JRockit and Dalvik VM used in Android OS with some deviations.

In brief JVM loads and executes compiled Java byte code using the resources allocated to it from the platform, it runs on.

JVM Structure

ClassLoaders

loads the byte code in the JVM memory  (load, link(verify, prepare, resolve –> if failed NoClassDef found exception is issued), initialize) Bootstrap class loaders, Extension class loaders, Application class loaders

Memory and runtime data area

This captures a few important sections below, though it is not comprehensive.

◉ Native method stack – The java native library stack which is platform-dependent, mostly written in C language.

◉ JVM stack (the currently executing method stack trace is kept, per thread. Recursive method calls can cause the stack to be filled and overflow(java.lang.StackOverFlowError) if proper breaks are not set. -Xss JVM option allows configuring the stack size.), PC register (program counter, points to the next instruction to be executed per thread. )

◉ Method area(stores Class data, size governed by XX:MaxPermSize, PermGen space 64MB default, if it is to serve huge server app loading millions of classes, then we can tune by increasing to avoid issues of OOM: PermGen space. From Java 8 onwards this PermGen space is referred as Metaspace with no limit in java8 by default though it is allowed to be fine-tuned and limit), Heap(Xms, Xmx), Run time constant pool

Execution engine

This engine executes the bytecode which is assigned to the runtime data areas through the classloader. It makes use of the Interpreter, Garbage Collector, Hotspot profiler, JIT compiler for optimized execution of the program.

Now we know where the Garbage Collector sits in the JVM architecture. Let’s go deep into the internals.

Garbage Collector

It is the Java automatic memory management process which removes the objects that are not used anymore. Then comes the question, how does it decide if the object is used or not.

It defines two categories of objects as,

live objects – reachable objects that are referenced from another object. Ultimately the reference links will reach the root which is the main thread which creates the whole object graph.

dead objects – unreachable objects that are not referenced by any other that are just lying in the heap.

this categorization and garbage collection is based on two facts as below.

1. Most of the objects soon become unreachable after the creation. Mostly the short-lived objects which live only within a method context.

2. Old objects rarely refer to young objects. For example, a long-lived cache would hardly refer a newer object from it.

Garbage Collection Steps

Newly created object instances reside in the Java heap, which goes to different generations as shown below. Garbage collection is done by a daemon thread called ‘Garbage Collector’ which directs the objects through different spaces within the heap.

Garbage Collection is done in 3 steps.

1. Mark – Starting from the root and traverse through the object graph marking the reachable objects as alive.

2. Sweep – Delete the unmarked objects.

3. Compact – Defragment the memory making the allocations contiguous for the live objects. It’s considered the most time taking process.

The Heap Area is divided as below.

Old(tenured) generation – Objects that survived for a long, stay here until it get marked unreachable and cleaned up in a major garbage collection which runs through the whole heap.

Young generation – this is further divided into 3 as Eden space and 2 Survivor spaces.

Garbage collection at two stages as ‘Minor’ or ‘Major’. Both these garbage collections are stop-the-world operations that stop every other memory access. Minor GC might not be felt by the application though as it only scans through the young generation space will be small in size.

Garbage Collector

The memory life cycle goes as below as shown in the above animation.

1. Newly created objects reside in the Eden space. (Just as humans started from Eden garden :) ) Until Eden space goes full it keeps on getting new objects added there.

2. When the Eden space is full, a minor GC runs, marks the live objects, move those live objects to ‘Survivor from’ space and sweep the Eden space which becomes free.

3. Then it keeps on filling the Eden space with new objects as the program runs. Now when the Eden space is full, we have previously moved objects in the ‘Survivor from’ space too. Minor GC runs marking objects in both these spaces, move the remaining live objects as a whole to the other survivor space. Wonder why not copy the live objects from Eden space to the remaining space of ‘Survivor from’ rather than moving all to the other survivor space? Well, moving all to the other has proven more efficient in compact step over compacting the area with objects in it.

4. This cycle will repeat moving objects between the suvivor spaces until a configured threshold (-XX:MaxTenuringThreshold) is met. (It keep tracks of how many numbers of GC cycles have been survived by each object). When the threshold is met, those objects will be moved to the tenured space.

5. As time passes, if the tenured space also gets filled up, the major GC kicks in and traverse through the whole Heap memory space performing the GC steps. This pause can be felt in human interactions and is not desired.

When there is a memory leak or huge caches that reside for long time, tenured space gets filled up with the time. At such times, those objects might not be even detected as dead. This results in major GCs running frequently as it detects tenured space is full, but it fails to clean up enough memory as nothing can be swept out.

This error ‘java.lang.OutOfMemoryError’ in the logs would hint us clearly when memory is not enough. Also if we see frequent CPU hikes with high memory usage, it can be a symptom of frequent GC run due to some kind of memory handling issue that needs attention.

Performance

When focusing on JVM fine-tuning focusing on memory utilization, the major deciding factor is what is more critical from Responsiveness/latency and Throughput. If the throughput is of utmost importance as in batch processing, we can compromise with having some pauses for major GC to run, if it helps overall throughput. Because the application occasionally going less responsive might not be an issue there.

On the other hand, if responsiveness is of utmost importance as in a UI based application, we should try to avoid major GC. Doing this namely, would not help though. For example, we can delay a major GC by increasing the space for the young generation. But then the minor GC would start to take much time as it needs to traverse and compact a huge space now. Hence have the correct size, the correct ratio between young and old generations needs to be carefully done to achieve this. Sometimes this can even go into the application design details to fine-tune memory usages with the object creation patterns and caching locations. It will be a topic for another post to analyze the heap dumps and flame graphs to decide on the best things to be cached.

Garbage Collectors

As the role of garbage collection is having this much impact on the performance of an application, so much of the effort have been put by the engineers to improve it. The result is, we have a choice on the best garbage collector to use as per the requirements. Below is a non-comprehensive list of options.

1. Serial Collector

Runs in a single thread. Only suitable for basic applications.

2. Concurrent Collector (CMS – Concurrent Mark and Sweep)

A single thread performs garbage collection. It only stops the world in mark and re-mark phase. The rest of the work is done while the application is running and does not wait for the old generation to be full. This is a good choice when the memory space is large, has a high number of CPUs to cater for concurrent execution, and when the application demands the shortest pauses with responsiveness been the critical factor. This has been the most favored in most of the web applications in the past.

3. Parallel Collector

This collector makes use of multiple CPUs. It waits for the old generation to be full or near full, but when it runs it stops the world. Multiple threads do the mark, sweep, and compacting making the garbage collection much faster. When the memory is not very large and the number of CPUs is limited this is a good option to cater to demands on throughput which can withstand pauses.

4. G1(Garbage First) collector (1.7 upwards)

This option improves garbage collection to be more predictable by allowing configurations such as pausing time when GC runs. It is said to have the good of both worlds of parallelism and concurrency. It divides the memory into regions and each region is considered as either an Eden, survivor or a tenured space. If the region is having more unreachable objects then that region is garbage collected first.

Default Garbage Collector in Versions

◉ Java 7 – Parallel GC
◉ Java 8 – Parallel GC
◉ Java 9 – G1 GC
◉ Java 10 – G1 GC
◉ Java 11 – G1 GC (ZGC provided as an experimental feature along with Epsilon)
◉ Java 12 – G1 GC (Shenandoah GC introduced. OpenJDK only.)

Tune-up parameters for the garbage collector

The rule of thumb for tuning up the JVM is not to do so unless there is an issue to be addressed with the default settings or decided after a lot of deliberation with proven effects after long-running production-level load patterns. This is because Java Ergonomics has advanced a lot and would be most of the time able to perform a lot of optimizations if the application is not behaving ugly. A comprehensive list of options can be found at [5] including configuring the sizes of the heap spaces, thresholds, type of garbage collector to use, etc.

Diagnose

Below configurations are useful to diagnose memory issues with the help of GC behavior in addition to the heap dumps.

-XX:-PrintGCDetails – Print details of garbage collection.
-Xloggc:<file-name> – Print GC logging details to a given file.
-XX:-UseGCLogFileRotation – Enable GC log file rotation when the above configuration is done.
-XX:-HeapDumpOnOutOfMemoryError – Dump the heap content for further analysis if a OOM error occurs.
-XX:OnOutOfMemoryError=”<cmd args=””>;<cmd args=””>” – Set of commands to be run, if an OOM error occurs. Allows to execute any custom task when facing the error.

Continue reading

Java Garbage Collection

Garbage Collection is one of the most important features in Java which makes it popular among all the programming languages. The process of garbage collection is implicitly done in Java. Therefore, it is also called Automatic Garbage Collection in Java. The programmer does not need to explicitly write the code to delete the objects.

Let’s start discussing the concept of the Garbage Collection in Java.

Garbage Collection in Java

Garbage collection is the technique used in Java to deallocate or remove unreachable objects and unused memory. From the name itself, we can understand that Garbage Collection deals with tracking and deleting the garbage from the memory area.

However, in reality, Garbage Collection tracks each and every object available in the JVM heap space and removes the unused ones.

We know that all the objects that we dynamically create are allocated in the heap memory of the application. Normally, it is the duty of the programmer to both create and delete the objects in the program, but the programmer usually ignores the deletion of the object. This creates a problem of OutOfMemoryErrors due to insufficient memory because of not deleting the unwanted objects.

In Java, the programmer does not have to worry about the problem of releasing the memory of these unused or unwanted objects, as the garbage collection system always runs in the background, and its main aim is to free the memory heap by deleting unreachable objects.

Essentially, Garbage Collection is the process of tracking down all the objects that are still in use and marking the rest of them as garbage. The Garbage Collection process in Java is considered an automatic memory management schema because programmers do not have to explicitly deallocate the objects. The garbage collection in Java runs on low-priority threads.

The implementation of the Garbage Collection is present in the JVM (Java Virtual Machine). Each JVM can implement garbage collection. But there is only one requirement; that it should meet the JVM specification. Oracle’s HotSpot is one of the most common JVMs that offers a robust and mature set of garbage collection options.

Object Life Cycle in Java

The object life cycle in Java can be divided into 3 stages:

1. Object Creation

To create an object, generally, we use a new keyword. For example:

MyClass obj = new MyClass();

We created the object obj of class MyClass. When we create the object, a specific amount of memory is allocated for storing that object. The amount of memory allocated for objects can vary on the basis of architecture and JVM.

2. Object Usage

In this stage, the Object is in use by the other objects of the application in Java. During its usage, the object resides in memory and may refer to or contain references to other objects.

3. Object Destruction

The garbage collection system monitors objects and keeps a count on the number of references to each object. There is no need for such objects in our programs if there are no references to this object, so it makes perfect sense to deallocate this unused memory.

Unreachable Objects in Java

When an object does not contain any “reachable” reference to it, then we call it an unreachable object. These objects can also be known as unreferenced objects.

Example of unreachable objects: 

Double d = new Double(5.6);
// the new Double object is reachable via the reference in 'd'
d = null;
// the Integer object is no longer reachable. Now d is an unreachable object.

Eligibility for Garbage Collection in Java

An object can be eligible for garbage collection in Java if and only if it is unreachable. In the above program, after declaring d as null; double object 4 in the heap area becomes eligible for garbage collection.

Object Eligibility

Though Java has automatic garbage collection, an object should be made unreachable manually. There are different ways to know whether the object is eligible for Garbage Collection in Java.

There are generally four ways in Java to make an object eligible for garbage collection:

◉ Nullifying the reference variable
◉ Reassigning the reference variable
◉ Island of isolation
◉ Creating objects inside a class

Ways of requesting JVM to run Garbage Collector

Even if we make an object eligible for Garbage Collection in Java, it may or may not be eligible for Java Virtual Machine (JVM) to destroy. So there are some ways to request JVM to destroy this object and perform garbage collection.

There are two ways to request JVM for Garbage collection in Java which are:

◉ Using System.gc() method
◉ Using Runtime.getRuntime().gc() method

Code to understand the above two methods:

package com.oraclejavacertified.garbagecollection;
public class Demo
{
  public static void main(String[] args) throws InterruptedException
  {
    Demo obj1 = new Demo();
    Demo obj2= new Demo();
    // requesting JVM for running Garbage Collector
    System.gc();
    // Nullifying the reference variable
    obj2 = null;
    // requesting JVM for running Garbage Collector
    Runtime.getRuntime().gc();
  }
  @Override
  // finalize method is called on object once before garbage collecting it
  protected void finalize() throws Throwable
  {
    System.out.println("Garbage Collector ");
    System.out.println("Garbage collected object: " + this);
  }
}

Output:

Garbage Collector
Garbage collected object: com.oraclejavacertified.garbagecollection.Demo@17f18626

Before removing an object from memory, the garbage collection thread invokes the finalize() method of that object and gives an opportunity to perform any sort of cleanup required.

A Real-life Example of Garbage Collection

Let’s take a real-life example of a garbage collector.

Suppose you go for an internship at a particular company and you have to write a program that counts the number of employees working in the company, excluding interns. To implement this task, you have to use the concept of a garbage collector.

The actual task given by the company:

Question. Write a program to create a class Employee having the following data members.

1. An ID for storing unique id for every employee.

And, the class will have the following methods:

1. A default constructor to initialize the id of the employee.
2. A method show() to display ID.
3. A method showNextId() for displaying the ID of the next employee.

Any beginner, who doesn’t know the concept of garbage collector will write the following code to count the number of employees:

Code to count the number of employees in the company without using garbage collection:

class Employee
{
  private int ID;
  private static int nextId=1;
  //we make it static because it should be common among all and shared by all the objects
  public Employee()
  {
    this.ID = nextId++;
  }
  public void show()
  {
    System.out.println("Id=" +ID);
  }
  public void showNextId()
  {
    System.out.println("Next employee id will be="+nextId);
  }
}
public class CountEmployees
{
  public static void main(String args[])
  {
    Employee A=new Employee();
    Employee B=new Employee();
    Employee C=new Employee();
    A.show();
    B.show();
    C.show();
    A.showNextId();
    B.showNextId();
    C.showNextId();
    {
      //It is a sub block to keep all those interns.
      Employee X=new Employee();
      Employee Y=new Employee();
      X.show();
      Y.show();
      X.showNextId();
      Y.showNextId();
    }
    //After this brace, X and Y will be removed.
    //Therefore, now it should show nextId as 4.
    A.showNextId();
    //Output of this line should be 4 but the output we will get is 6.
  }
}

Output:

Id=1
Id=2
Id=3
Next employee id will be=4
Next employee id will be=4
Next employee id will be=4
Id=4
Id=5
Next employee id will be=6
Next employee id will be=6
Next employee id will be=6

Now to get the correct output:

If we write the same code using the garbage collection technique, the garbage collector will see that the two objects are free. To decrement the value of the variable nextId, the garbage collector will call the finalize() method only when the programmers override it in their class.

And as we know that we have to request a garbage collector, and to do this, we have to write the following three steps before closing the brace of sub-block.

1. Set references to null (that is, X = Y = null;)
2. Call System.gc();
3. Call System.runFinalization();

Correct Code to count the number of employees (excluding interns) using garbage collection

//Correct code to count the number of employees excluding interns.
class Employee
{
  private int ID;
  private static int nextId=1;
  //we declare it static because it should be common among all and shared by all the objects
  public Employee()
  {
    this.ID = nextId++;
  }
  public void show()
  {
    System.out.println("Id="+ID);
  }
  public void showNextId()
  {
    System.out.println("Next employee id will be="+nextId);
  }
  protected void finalize()
  {
    --nextId;
    //In this case,
    //gc will call finalize()
    //for 2 times for 2 objects.
  }
}
public class CountEmployees
{
  public static void main(String args[])
  {
    Employee A=new Employee();
    Employee B=new Employee();
    Employee C=new Employee();
    A.show();
    B.show();
    C.show();
    A.showNextId();
    B.showNextId();
    C.showNextId();
    {
      //It is a sub-block to keep all those interns.
      Employee X=new Employee();
      Employee Y=new Employee();
      X.show();
      Y.show();
      X.showNextId();
      Y.showNextId();
      X = Y = null;
      System.gc();
      System.runFinalization();
    }
    E.showNextId();
  }
}

Output:

Id=1
Id=2
Id=3
Next employee id will be=4
Next employee id will be=4
Next employee id will be=4
Id=4
Id=5
Next employee id will be=6
Next employee id will be=6
Next employee id will be=4

Garbage Collection Algorithms in Java

Garbage Collection Algorithms in Java helps to remove the unreferenced or unreachable objects. These algorithms always run in the background. There are several different types of Garbage Collection Algorithms in Java that run in the background. And among them, one of the algorithms is a Mark and Sweep algorithm.

Mark and Sweep Algorithm

Mark and Sweep algorithm is a fundamental and initial algorithm for Garbage Collection in Java. This algorithm basically performs two primary functions: mark and sweep. Firstly, it should track and detect unreachable objects and, secondly, it should release these objects from the memory heap area so that the programmer can use it again.

1. Mark phase – Mark live objects

It is the first phase of the algorithm in which there is the detection of all the objects that are still alive. It is a stage where the garbage collector identifies which parts of memory are in use and which are not in use.

In this phase when the condition is made, its check bit is set to 0 or false. We set the marked bit to 1 or true for all the reachable objects.

Here we can consider each object as a node and then we visit all the objects or nodes that are reachable from this object/node, and it repeats until we have visited all the reachable nodes.

◉ The root is a variable that refers to an object and is directly accessible by a local variable. We will assume that we have only one root.

◉ We can use markedBit(obj) to access the mark bit for an object.

Mark Phase Algorithm:

Mark(root)
If markedBit(root) = false then
    markedBit(root) = true
    For each v referenced by a root
         Mark(v)

Note: We can call Mark() for all the root variables if we have more than one root.

2. Sweep phase – Get rid of dead objects

The sweep phase algorithm “clears” all the inaccessible or unreachable objects that is, it releases the stored memory area for all the inaccessible objects. Each of the items whose check value is set to false is removed from the stack memory, for every single other reachable object, we set the value of the stamped bit to false.

Presently the check bit for all the reachable objects is set to false.

Sweep Collection Algorithm:

Sweep()
For each object p in a heap
    If markedBit(p) = true then
       markedBit(p) = false
    else
       heap.release(p)

‘Mark and Sweep’ algorithm is also called a tracing garbage collector because this algorithm is used to trace the objects. For example

◉ Marked bits set to false.
◉ Reachable objects are set to true.
◉ Non-reachable objects get clear from the heap.

Advantages of the Mark and Sweep Algorithm

◉ It is a cyclic process.
◉ There are no additional overheads that occur during the execution of an algorithm.

Disadvantages of the Mark and Sweep Algorithm

◉ While the Java garbage collection algorithm runs, normal program execution stops.
◉ It runs differently several times on a program.

Implementations or Types of Garbage Collection

JVM has four types of Garbage Collector implementations which are –

◉ Serial Garbage Collector
◉ Parallel Garbage Collector
◉ CMS Garbage Collector
◉ G1 Garbage Collector

Now, we will briefly discuss each type of garbage collector.

1. Serial Garbage Collector

It is the simplest Garbage Collector implementation as it basically works with a single thread and all the garbage collection events are conducted serially in one thread. As this collector can work on a single thread, it freezes all application threads when it runs. Hence, it is not preferred to use it in multi-threaded applications like server environments.

We can use the following argument to enable Serial Garbage Collector:

java -XX:+UseSerialGC -jar Application.java

2. Parallel Garbage Collector

It is the default Garbage Collector of the JVM and sometimes called Throughput Collectors. Unlike the Serial Garbage Collector, the Parallel Garbage Collector uses multiple threads to manage heap space. But at the same time, it also suspends other application threads while performing garbage Collection. Using this Garbage Collector, we can specify the maximum garbage collection threads throughput and footprint (heap size) and, pause time.

We can use the following argument to enable Parallel Garbage Collector,

java -XX:+UseParallelGC -jar Application.java

3. CMS (Concurrent Mark Sweep) Garbage Collector

The CMS Garbage Collection implementation uses multiple threads for garbage collection. This Garbage Collector is designed for applications that can afford to share processor resources with the garbage collector while the application is running and that prefer shorter garbage collection pauses. Simply we can say that applications using CMS respond slower on average but do not stop responding to perform garbage collection.

We can use the following flag to enable the CMS Garbage Collector:

java -XX:+UseParNewGC -jar Application.java

4. G1(Garbage First) Garbage Collector

G1 (Garbage First) Garbage Collector is the newest garbage collector which is designed as a replacement for CMS. It performs more efficiently as compared to CMS Garbage Collector. It is similar to CMS and is designed for applications running on multiprocessor machines with large memory space.

To enable the G1 Garbage Collector, we can use the following argument:

java -XX:+UseG1GC -jar Application.java

Advantages of Garbage Collection:

◉ There is no need to manually handle the memory allocation/deallocation because the JVM automatically performs the Garbage Collection for unused space in Java.

◉ There is no overhead of handling the Dangling Pointer.

◉ Garbage Collection takes care of a good portion of Automatic Memory Leak management.

Disadvantages of Garbage Collection:

◉ There is a more requirement of CPU power besides the original application, as JVM has to keep track of object reference creation/deletion. This may affect the performance of requests which require a huge memory.

◉ Programmers do not have any control over the scheduling of CPU time dedicated to freeing the unreachable objects.

◉ Using some Garbage Collection implementations an application may stop unpredictably.

◉ Automatic memory management is not much efficient proper manual memory allocation/deallocation.

Continue reading