Avoiding Apache Tomcat Out Of Memory Errors
Out Of Memory Errors, or OOMEs, are one of the most common problems faced by Apache Tomcat users. Generally, these errors occur during development, but can even occur on production servers that are experiencing an unusually high spike of traffic. Tomcat 7 includes fixes and workarounds to prevent some of the causes of OOMEs, but nothing substitutes a good understanding of why these errors occur.
This guide will help you understand why these errors are so prevalent and seemingly hard to fix, and show you how organizations using Apache Tomcat in enterprise production environments use Tcat to fix and avoid these errors.
What Causes An OOME?
The literal cause of an Out Of Memory Error is simple: a given Tomcat instance uses up all of the heap memory allocated to it, causing an application or server crash.
Thus, the errors themselves are not particularly puzzling. The reason why OOMEs have become such a persistent topic of discussion in the Apache Tomcat community is that they are difficult to trace to their root cause.
A stack trace will not show the cause, as the problem usually lies not with Tomcat, but the web application.
Meanwhile, the 'incorrect' web app code causing Tomcat to run out of memory is usually technically correct. That is to say, in any context other than a container managed environment, the code would be considered standard. It is easy to see why a developer might have trouble determining which 'correct' section of their code was incorrect.
Here are some of the most common root causes of an OOME:
- a simple case of the heap size being too small
- running out of file descriptors
- more open threads than the host OS allows
- code with high amounts of recursion
- code that loads a very large file into memory
- code that retaining references to objects or classloaders
- a large number of web apps and a small PermGen
Fixing Out Of Memory Errors
If you are encountering Out Of Memory Errors, the first step is to determine exactly where the error is coming from.
Sometimes, the fix is as simple as increasing the starting heap size or PermGen. For more information on how to configure these options, visit our guides to Tomcat Performance Tuning and Tomcat JVM Configuration. In other cases, fixing an OOME will require you to refactor some of your code to make it suitable for a container-managed environment.
Below, you'll find some best practices to follow when writing web applications for Tomcat (or any other container-managed environment).
Close All Threads
If your application opens any threads, the same application must terminate them. Failing to do so will quickly eat up memory. In Tomcat 7, an application that does not properly close threads will trigger a warning, but for earlier versions, finding this error in your application code is a matter of due diligence. Similarly, a thread designated as ThreadLocal may remain open if the designation is not removed before the end of the request lifecycle.
Some web applications use custom classloaders to load drivers. Well-behaved web apps will unload these drivers before the end of the undeploy process.
Release Logging Framework Resources
Although technically unnecessary in some environments, when running a logging framework within a container-managed environment such as Apache Tomcat, you should ensure that the framework releases all of its resources when you are done with it.
If your web application uses a Java API capability that retains a ContextClassLoader reference, it will eventually cause a memory leak in a container-managed environment. As of Tomcat 7, workarounds for several common cases of this problem are a part of the codebase, but this does not guarantee that your web application is safe. There is no substitute for testing.
Testing For Out Of Memory Errors
Create a test instance of Tomcat with the web applications you wish to check for leaks deployed for testing. After obtaining a heap dump, you will be looking for instances of WebappClassLoader - a number of instances greater than the number of web applications you have deployed to your test server, you have a memory leak.
By determining which of the loaders is superfluous (started==false), you should be able to locate the code in your application that is causing the leak by holding onto a reference.
Many administrators of enterprise Tomcat infrastructures use Tcat, the enterprise version of Apache Tomcat, to avoid and quickly fix memory leaks.
Tcat allows adminstrators to create custom e-mail or SMS alerts for any attribute of a server or web application, such as unusual memory activity.
Next, using Tcat's centralized management console, it is possible to roll back to a stable version of the application with a single click, diagnose the problem, fine-tune the configuration of the server and JVM, save the enw configuration, and apply the changes to the entire infrastructure with a single click.
Tcat is built on 100% Apache Tomcat, with no modifications to the core binaries, and works with your existing infrastructure - no need to rip and replace.
Don't wait - Once you've tried enterprise Apache Tomcat, you won't go back. Download Tcat today!