FULL PRODUCT VERSION :
java version "1.4.1_01"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.1_01-b01)
Java HotSpot(TM) Client VM (build 1.4.1_01-b01, mixed mode)

FULL OPERATING SYSTEM VERSION :
SunOS 5.9 Generic_112233- customer  sun4u sparc SUNW,Sun-Fire-880

ADDITIONAL OPERATING SYSTEMS :
Linux 2.4.18-17.8.0 #1 Tue Oct 8 13:51:08 EDT 2002 i686 i686
i386 GNU/Linux



A DESCRIPTION OF THE PROBLEM :
The JVM seems to have a memory leak: when run with the -Xmx
and -Xms options, then the JVM seems to start filling up all
the available memory of the system, and then throws an out
of memory exception.

Even without using the -Xm* options the program grows uncontrolably on Solaris.

The problem presents both on a Red-hat Linux system and on a
Solaris system, even using -Xint or -Xfuture options.
Maybe also other systems are affected by this bug, though I
cannot confirm.

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
1. Run the included program
2. the system will swap and eventually you'll have an OutOfMemoryError and 
a core dump.

EXPECTED VERSUS ACTUAL BEHAVIOR :
Obviously, I didn't expect it to fill up all my memory :-)

REPRODUCIBILITY :
This bug can be reproduced always.

---------- BEGIN SOURCE ----------
import java.util.zip.*;

public class Bug {
        public static void main( String args[] ) {
                while ( true ) {
                        /* If ANY of these two lines is not commented, the JVM
                         runs out of memory */
                        final Deflater deflater = new Deflater( 9, true );
                        final Inflater inflater = new Inflater( true );
                }
        }
}
---------- END SOURCE ----------

CUSTOMER WORKAROUND :
One way to avoid the problem is by passing the -Xincgc
option to the JVM
(Review ID: 173339) 
======================================================================
  xxxxx@xxxxx   10/15/04 20:43 GMT

Invoke "end()" on any Inflator or Deflator once it is no longer in use. This will clean up the process space (non Java/GC) memory.

  xxxxx@xxxxx   2003-01-07

Read 5092131 for more work around ideas.
  xxxxx@xxxxx   10/15/04 20:51 GMT

Since the Inflater/Deflaters are not explicitly closed, their memory is only freed by the finalizer which cannot be relied upon to occur in a timely manner. So getting an OutOfMemoryError from this program is not a bug.

However, there may be a problem since the VM throws the error but does not crash in 1.4 and it throws then crashes in 1.4.1 and 1.4.2. Since the Inflater and Deflater java and native code has not changed in years, I don't think this is a libraries problem. I am moving the bug to runtime. 
 
  xxxxx@xxxxx   2003-01-06

This is really two problems:

First, the libraries use a lot of non-Java (i.e., not garbage collectible) memory for these routines, which is preserved in the java memory and only deleted on finalize (from GC).  The allocated space for the java memory item is itself very small. This has the effect of growing the total process space like crazy while the java allocated space remains quite small. This is demostrated by the use of GCALot, which forces GC. In that case, the process space remains small.

Second, the VM has places where it does not handle running out of memory. I suspect this will require significant effort to chase down.

  xxxxx@xxxxx   2003-01-07

The 1.5.0 vm correctly throws an OOM and exits w/o crashing, so I'm
closing this bug as 'not a bug'.

  xxxxx@xxxxx   2003-12-11

The JDC comment by rezaei says:

"How was this closed? The evaluation comment dated 2003-12-11
 says "throws an OOM". That is NOT correct behavior for a
 program that is not holding any references. The garbage
 collector is badly broken if allocation of objects leads to
 when no objects are reachable."

We're throwing an OOM not because the java heap is (anywhere close to) full,
but because the non-java (i.e., C/C++) heap is full: you can get an OOM
when either heap is full.  The reason for the OOM is because unreachable, but
not yet finalized, java objects are holding pointers to C/C++ heap memory, and
that memory is deallocated pnly when the finalizers run.  The JLS doesn't
specify when finalizers get run, so if the VM delays executing them long
enough, the C/C++ heap memory doesn't get freed soon enough to prevent an OOM.

I'm reopening the bug and transferring it to the gc group, because when to run
finalizers is a gc policy decision.

  xxxxx@xxxxx   2004-01-05


"I think we're seeing the usual problem with Java's inability to
collect non-heap-memory resources in a timely fashion.
The finalizable objects which can unmap the memory will be collected,
but the GC is not aware of the urgency, since they appear to be 
simple small Java objects."

In the absence of a general solution to the non-heap resource
exhaustion problem, users of classes with close() methods should
make sure to call them as soon as possible.

The problem is more thoroughly explained in

5092131: using 1 MB pagesize throws "not enough space" error with 32bit JVM
  xxxxx@xxxxx   10/15/04 20:51 GMT

Any attempt to fix this problem would require some surgery
of GC policy -- and to make the GC subsystem aware of
non-Java-heap resource exhaustion and to react to such
exhaustion with GC (presumably to get the finalizable
objects on to the reference handler queues) and tnen to
wait sufficiently long for the finalizers to run
before attempting to allocate said limited resource.
Such a policy architecture would take some effort to
implement properly, and will not be attempted for Mustang
because of, ahem, resource limitations. We are deferring
consideration of this bug to the Dolphin time-frame.
  xxxxx@xxxxx   2005-04-22 02:31:53 GMT
This particular problem with Infator can likely be resolved by using java2d disposer or similar approach.
Posted Date : 2008-04-16 18:49:53.0

Seems to happen in Linux 1.4.0 as well.

Think it happens in NT 4.0 sp6 as well

On Windows 2000 sp3 too

How was this closed? The evaluation comment dated 2003-12-11
says "throws an OOM". That is NOT correct behavior for a
program that is not holding any references. The garbage
collector is badly broken if allocation of objects leads to
OOM when no objects are reachable.

I firt wrote this for bug 4484370 (which is closed), but it
might well fit just as well here.

I've been battling with the same kind of problem the last
week.  "uname -srvmpio":
Linux 2.4.20-9smp #5 SMP Fri May 9 17:37:37 PDT 2003 i686
i686 i386 GNU/Linux

"java -version":
java version "1.4.1_01"
Java(TM) 2 Runtime Environment, Standard Edition (build
1.4.1_01-b01)
Java HotSpot(TM) Client VM (build 1.4.1_01-b01, mixed mode)

Running a web server Cauch Resin 2.1.10.  What we see is
that memory footprint suddenly starts increasing 10 MByte or
more per 5 seconds, for no apparent reason.  The max heap
size is set to "-Xmx512m" and there is 1 GByte of physical
RAM in the machine. The normal heap usage is 100-200 Mbyte.
 When the footprint exceeds 1 GByte swapping kills the
performance and it is only a matter of time before the
process dies. We have seen the process reach 2.5 GByte
before disappearing.  Attaching "strace -p" to the process
produces nothing but the normal gettimeofday() and poll()
calls from a background timer thread every 5 seconds. 
Dozens of "kill -QUIT" triggered stack traces on these
processes reveal no apparent pattern in the Java code being
executed.  I don't have a core file handy, but if there is
interest I can probably provide one.  The workaround for us
was to switch to JDK 1.5 beta, which doesn't exhibit the
problem, at least not now.  We have been using the 1.4.1_01
JDK and the same kernel version for 6 months+ without seeing
the problem in the past, and the code changes that appear to
have touched a nerve seem very innocent  to us.  Jakarta
Struts with their "tiles" implementation, and more hevy use
of the TimeZone class (which has a known performance issue
in the old JVM (bug 4936890)).

This still isn't fixed in 1.5.0 beta 1!  Hello???????

We've been chasing a mysterious memory leak problem for
months and months, and my guess is that this is it.  How can
Java be considered a robust language when something this
basic is so tragically broken???????

SUN, FIX THIS, PLEASE!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

Holy Weakness, Batman!

Yo, SUN!  This is a killer!

This bug is an example of the following true statement: if your JVM
implementation happens to associate native memory (i.e., C heap
memory, not Java heap memory) with objects that doesn't get
freed until the object finalizer is run, then for your JVM to
function correctly, you must ensure that the finalizer thread
performs a complete run before throwing any OOM errors.

This is just a side effect of an implementation decision. To
turn it around and say "finalization is not guaranteed therefore
your OOM is not a bug" is putting the cart before the horse.

If Sun is not willing to implement the policy that all finalizers
must run before throwing an OOM, then to fix this bug it must
make another choice, e.g. reimplement the ZIP stuff in pure java.

There are surely other examples of this kind of bug. They should
be enumerated and a similar decision made.

java.lang.OutOfMemoryError
        at java.util.zip.Deflater.init(Native Method)
        at java.util.zip.Deflater.<init>(Deflater.java:117)
        at java.util.zip.Deflater.<init>(Deflater.java:126)
        at org.ajax4jsf.framework.resource.ResourceBuilderImpl.encrypt(ResourceBuilderImpl.java:555)
        at org.ajax4jsf.framework.resource.ResourceBuilderImpl.getUri(ResourceBuilderImpl.java:284)
        at org.ajax4jsf.framework.resource.InternetResourceBase.getUri(InternetResourceBase.java:211)
        at org.ajax4jsf.framework.renderer.HeaderResourcesRendererBase.getUrisSet(HeaderResourcesRendererBase.java:98)
        at org.ajax4jsf.framework.renderer.HeaderResourcesRendererBase.getHeaderStyles(HeaderResourcesRendererBase.java:71)
        at org.ajax4jsf.framework.ajax.AjaxContext.processHeadResources(AjaxContext.java:465)
        at org.ajax4jsf.framework.ajax.AjaxContext.processHeadResources(AjaxContext.java:478)
        at org.ajax4jsf.framework.ajax.AjaxContext.processHeadResources(AjaxContext.java:387)
        at org.ajax4jsf.framework.ajax.AjaxViewHandler.renderView(AjaxViewHandler.java:261)
        at org.apache.myfaces.lifecycle.LifecycleImpl.render(LifecycleImpl.java:384)
        at javax.faces.webapp.FacesServlet.service(FacesServlet.java:138)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:252)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
        at org.jboss.seam.web.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:63)
        at org.jboss.seam.web.RedirectFilter.doFilter(RedirectFilter.java:45)
        at org.jboss.seam.web.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:49)
        at org.jboss.seam.web.ExceptionFilter.doFilter(ExceptionFilter.java:57)
        at org.jboss.seam.web.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:49)
        at org.jboss.seam.web.MultipartFilter.doFilter(MultipartFilter.java:79)
        at org.jboss.seam.web.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:49)
        at org.jboss.seam.web.SeamFilter.doFilter(SeamFilter.java:84)

This happens on JDK 1.5.11

This is a ridiculous bug, and it's been there for 5 years. WTF??

If you can't make the garbage collector behave, re-implement these classes in native Java. This just brought our servers to a halt (using all 8GB native memory).

I was able to reproduce this bug in a clean environment, running my test with the following settings, this bug eats about 1000 MB every 5 seconds in my testing code. That's a lot of memory to eat. No wonder it kills our 8GB servers in a very short time period.

Regression:
It doesn't happen in Java6. 
It also seems to be related to the way the GC is set up, so I am copying the entire command FYI. I would be happy to provide full testing code as well (I have to rewrite to remove dependencies on our proprietary code base). Please contact me - otherwise I will assume that no one reads this and won't bother. 

Here are the settings I used:

"C:\Program Files\java\jre1.5.0_10\bin\java.exe" -Xmn100M -Xms200M -Xmx300M -Xss128K -Xrs -Xloggc:log/gc.log -Dhttp.maxConnections=200 -XX:+UseConcMarkSweepGC -XX:+PrintGCTimeStamps -XX:+PrintGCDetails -Dcom.sun.management.jmxremote.port=9337 -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false -cp weblib-all.jar;prophet\BuildFilesGlobal\ant\lib\junit.jar DeflaterTestClass

PS: Applying the work around works just fine. We now call end() on these when not used anymore and the problem disappeared.

I realize that changing the VM's garbage collection behavior is a big change to make, but I think there is a much easier way to fix this bug: Change the zip implementation so that instead of allocating native memory (malloc() or whatever), it uses JNI to allocate memory on the Java heap (and, of course, store a reference to that memory in a Java member variable, so that it won't get gc'd too quickly).

According to the docs (e.g. http://java.sun.com/j2se/1.5.0/docs/api/java/util/zip/Deflater.html ), Sun's ZIP code is built on the ZLIB compression library.  And according to the documentation for ZLIB, that library allows the caller to plug in any allocation code it wants (see http://www.zlib.net/manual.html ).

This would probably be a pretty easy change to make, and it would eliminate the use of native memory without having to make a risky change such as changing the GC behavior.