Java Solaris Communities Sun Store Join SDN My Profile Why Join?
 
Bug Database
Bug Detail
Quick Lists
Top 25 Bugs
Top 25 RFE's
Recently Closed Bugs
Printable Page Printable Page


Bug Database
Bug ID: 4513817
Votes 39
Synopsis File.deleteOnExit consumes memory
Category java:classes_io
Reported Against merlin-beta2
Release Fixed
State 11-Closed, duplicate of 6291034, bug
Priority: 3-Medium
Related Bugs 6291034 , 4872014 , 6188718 , 4809375
Submit Date 11-OCT-2001
Description


I'm using the imageio.jar form 1.4beta, but with:
Java(TM) 2 Runtime Environment, Standard Edition (build 1.3.1-b24)
Java HotSpot(TM) Client VM (build 1.3.1-b24, mixed mode)


Every time I use "ImageIO.write(img, "png", outStream)" a temporary file is
created, and the "deleteOnExit()" function is called on that temporary file.
Every "deleteOnExit()" allocates a "struct dlEntry" in the shared "io_util.c"
native source file. This "struct dlEntry" is about 1K in size, and will never be
freed until the system exits, so every time I do an ImageIO.write() I take up
another 1K of memory! This is in a JSP engine that is creating dynamic images,
so I want it to be able to run a long time, but at 1 image/minute it will
consume 1.5MB per day that the JSP engine is up. This is absurd. It seems that
no package should use "deleteOnExit()" dynamically - it guarantees a memory
leak. In a long-running system like a JSP engine, this is completely unacceptable.

It would be better if the temporary file were deleted when the image data was
garbage collected, or some such, so there was no need to allocate a struct
dlEntry in the native code.
(Review ID: 133570) 
======================================================================

Its possible to solve the ImageIo problem if there is a version of
deleteOnExit that returns a unique identifier. Clients such as ImageIO
who remove the file before exit can then call an API to remove the
file referenced by that unique identifier from the internal list of
files to delete on exit.

long deleteOnExit2() 
public static void removeFromDeleteList(long)

  xxxxx@xxxxx   2003-04- customer 
============================
Posted Date : 2005-08-23 05:37:10.0
Work Around


Stop and re-start the application (Tomcat/Jakarta in this case) on a regular basis.
======================================================================

One workaround is to disallow the use of cache files by passing false to the
static ImageIO.setUseCache() method.  This avoids the deleteOnExit() problems
and should have little adverse effect on performance, especially in the case
of server-side applications working with relatively small images.
  xxxxx@xxxxx   2001-11-16
Evaluation
See comments.  This memory leak is a major reliability issue for developers
using Image I/O in their server side applications.
  xxxxx@xxxxx   2002-11-11

We should investigate ways to improve the implementation of File.deleteOnExit
for long-running systems.

--   xxxxx@xxxxx   2002/11/21

It would be a fairly simple matter to remove a file from the delete on
exit list when it gets deleted, but that would be a violation of spec.
File.deleteOnExit says "Requests that the file or directory denoted by this
abstract pathname be deleted when the virtual machine terminates." So
just because the file is deleted does not mean that it will not get re-opened later. The File object might even get garbage collected but another of the same
name might be created later. We are obligated by the spec to remember the path
and delete a file there when the VM terminates. There is no way around the
memory leak without relaxing this spec.

We may be able to offer something in the new FileSystem API in Tiger that
solves this problem in a compatible way.

  xxxxx@xxxxx   2002-11-22



  xxxxx@xxxxx   2004-05-13
The current spec is abiguous. There may be several files, over the
life of the JVM, which meet the criteria of "denoted by this abstract
pathname".  In this case, to which one of these files does the method
apply?  

I interpret the specification as refering to "the file", a tangible
entity, which exists at the time of the call.  If the specification were
intended to refer to a file which does not currently exist (and may
never exist), it would have been more appropriate to say "a file".

If this interpretation is accepted, then the current implementation is
faulty, and delete() should remove the entry from the list.

This would be a minor binary incompatibility, which would be revealed 
if an application created more than one file with the specified
pathname, and relied on the last such file being deleted.  I view this as
very unlikely, while the memory leak is a real reliablilty problem.  Changing
the implementation to improve reliability is a reasonable trade-off.
Posted Date : 2005-08-23 05:37:10.0

--


4809375 re-implements deleteOnExit mechanism so that the list of files to be deleted is stored as a list in the java heap. The implementation records the file name as most once so if deleteOnExit is invoked multiple times on the same File then it will not consume any additional memory. This doesn't address the case of temporary files with unique file names but it should improve the diagnosability of such situations as the list will be visible to a java memory profiler.
Posted Date : 2005-11-25 09:33:53.0

For diagnosing deleteOnExit issues you can see the following blog:
 http://blogs.sun.com/roller/page/chegar#diagnosing_deleteonexit_issues
Posted Date : 2006-04-12 08:31:46.0

This bug, as described in the description section and relating the Image IO code that uses deleteOnExit to ensure that temporary files are cleaned up, has been fixed by 6291034. It now avoids deleteOnExit completely and uses its own shutdown hook to remove these files. So I am closing this bug as a duplicate of it.

For anyone who is concerned with the general problem of not being able to remove files marked deleteOnExit please refer to RFE 4872014.
Posted Date : 2006-07-17 14:26:25.0
Comments
  
  Include a link with my name & email   

Submitted On 15-NOV-2001
mindyleejones
Here's a good test program to illustrate the bug:

class test2
{


 public static void main(String[] args){
  try{
   while (true){
        File test = File.createTempFile("test",".tst");
        test.deleteOnExit();
        test.delete();
   } 
  } catch (Exception e){e.printStackTrace();}
  
 }
}
on a regular basis you can check the process heap size 
growth (on solaris) by doing a :
/usr/proc/bin/pmap <pid> | grep heap

Once the file is deleted manually (and succesfully) with 
the delete() function, there is no reason for the vm to 
remember to deleteOnExit that file anymore.  For server 
applications that run for a long time, this can be a 
serious leak.

Granted, this function may be intended for deleting 
whatever may have the abstract filename path at VM exit, 
regardless of how many files have been created and deleted 
with that name.  In that case I'd request, as a bug fix 
(feature request), the ability to cancel the deleteOnExit 
and free up the memory.  At the very least the memory 
implications of using this function should be documented in 
the api docs.


Submitted On 05-SEP-2002
Lenbok
And here is another perturbation of the test program that
also illustrates the naivety of the current implementation:

class test3 {
  public static void main(String[] args) {
    try {
      File test = File.createTempFile("test",".tst");
      while (true) {
        test.deleteOnExit();
      } 
    } catch (Exception e){e.printStackTrace();} 
  }
}

The deleteOnExit() implementation doesn't even check to see
if it has already been called *for the same object*!

This is a terrible implementation. If the File is explicitly
delete()'d, then the deleteOnExit() should be cancelled. The
ability to explicitly cancel a deleteOnExit() should also be
provided.

(The category of this bug should be moved to reflect that it
is really File.deleteOnExit() that has the problems.)



Submitted On 30-MAY-2003
alangreenspan
There absolutely must be a way to cancel the pending 
delete.  We have observed this bug crashing servers that use 
this API in temporary file manipulation.  The bug is very 
difficult to diagnose since the memory leak is in JVM private 
memory, not the Java heap.  Suggest adding 
File.cancelDeleteOnExit in 1.4.2.


Submitted On 26-NOV-2003
varnam
As others mentioned , there should be a way to undo the
effect of DeleteOnExit call  or JVM get rid of the 
datastructure when actual file is deleted.  Had to spend
three  days to fugure out memory growth in jvm process 
heap  was becaue of this call to get rid of temporary files.



Submitted On 03-MAR-2004
forrestg
Unfortunately, Struts has now tripped over this bug.  Web applications 
using Struts 1.1 will now cause the container in which they execute to 
core.  Using previously posted test code I was able to trivially 
reproduce cores with four distinct, current JVMs (Solaris, Darwin, AIX, and 
Linux).  The severity of this issue regardless of the number of votes 
couldn't possibly be higher especially in light of the fact that it was 
reported more than two years ago.


Submitted On 09-JUL-2004
matthias.ernst
We have been bitten by this one, too. It was very difficult to find and cost us a lot of work, nerves and customer reputation.


Submitted On 28-AUG-2005
matthias.ernst
The worst part: the data structure is kept in native memory. If it only was in Java heap, you might be able to diagnose the problem much quicker.


Submitted On 15-NOV-2005
philcam
Apache's commons-fileupload 1.0 package suffers from this bug. A work-around will be available in the 1.1 release, which is not yet available.

This bug should be resolved, as it negatively affects reliability in practically all production-grade jvm's.



Submitted On 27-JAN-2006
spainhou
This bug still exists and is causing tomcat to run out of heap space when generating a png or jpg image.


Submitted On 01-FEB-2006
javaweather
This bug took significant time to identify and track down as it inspection of native non-heap memory.  It would be much appreciated if there was a way to default property of
ImageIO.setUseCache(false)
so having the memory leak would not be the default behaviour.
It would be much appreciated.
Thanks



PLEASE NOTE: JDK6 is formerly known as Project Mustang