FULL PRODUCT VERSION :
n/a - applicable to all JRE's

FULL OPERATING SYSTEM VERSION :
n/a - applicable to all JRE's

ADDITIONAL OPERATING SYSTEMS :

n/a - applicable to all JRE's


A DESCRIPTION OF THE PROBLEM :
It would be very useful if the JRE had the ability to
return unused memory to the O.S.
For example, I have a large 24/7 Java program that requires
> 800MB of heap, and runs for approx 30 mins each day.

I do not want to burden the O.S. with such a memory
requirement (or even to burden the O.S. with a permanent
swap space requirement).

As a workaround, I have a java program that performs some
24/7 maintenance, then every day System.exec()'s the large
java program. This process exits after approx 30 mins and
memory is returned to the pool. My 'harness' then sleeps
until the next day.

Ideally, I should have an option to tell the JRE that
rather than return memory to the JRE heap, it should free
unused memory so that the O.S. can make use of it.

I imagine this would be easy to implement using some O.S.
free() system call.


STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
Design issue, rather than a bug.

EXPECTED VERSUS ACTUAL BEHAVIOR :
I would like a new System API call to tell the JRE to free
memory to the O.S.

This bug can be reproduced always.

CUSTOMER WORKAROUND :
As a workaround, I have a java program that performs some
24/7 maintenance, then every day System.exec()'s the large
java program. This process exits after approx 30 mins and
memory is returned to the pool. My 'harness' then sleeps
until the next day.

This is not very elegant, I would like to simply call a
method that does the work, and call something
System.freeVMMemory() after.
(Review ID: 147258) 
======================================================================
Posted Date : 2005-08-08 16:57:09.0

Allocate more swap space and be confident that the operating system's virtual memory management system is doing its job. That is, relax and consider the cost of disk space for swap memory in relation to modifications and fixes to application logic to make them more aware of memory usage.

  xxxxx@xxxxx   2002-10-23

There may be room for enhancement of memory usage such that the operating environment can take back portions of memory previously allocated by the Java implementation vis a vis swap space. For various reasons this may not manifest as a gross change in process size as reported by simple tools like the "ps" command, but the underlying virtual memory usage may be reducible to reflect less demand on the heap. This RFE should be kept open while the feasibility of these improvements is explored. 

There is nothing specific to Java about this situation. Reclamation of memory
used by a process varies from moderately involved to extremely difficult or 
impossible depending on the circumstances.

It would be nice if Sun's Java virtual machine could simply do one lib C call to "free(pointer_to_big_wad_o_stuff)" and expect the operating system to shrink the process back down. However the memory allocations of Java implementations don't track application demands on memory in a straight forward manner, leading to awkward bits of memory pinned in regions that may otherwise be unused at various times.

Although it's easy to imagine the Java object store being arranged to support compaction in such a way that some portion of memory in use can be declared free to the operating system, in practice this is a big challenge. For instance on Unixen systems one might require forbidding any memory allocations above the process sbrk following creation of the Java object store, so as to allow the sbrk to be moved back down or up further and track an application's cycles of memory usage. This is a drastic design constraint as it implies that all other memory usage by the Java implementation must be known and fixed before assigning the location of the heap or must be movable. Some forms of memory allocation that take place outside the Java VM or core libraries are simply not under the kind of control required to cooperate with such a hypothetical "dynamic heap" scenario. The presence of JNI makes such a restriction a major challenge. Simply being aware of and tracking allocations "above" the Java heap for the purpose of migrating them and thus allowing the sbrk to be moved back down would be expensive assuming it is possible (and it does not seem possible for various reasons).

The cost and ramifications of arranging for Java heaps to shrink so as to give back swap space (on systems that support this) is hard to justify. In the example given for this RFE 800 megabytes seems like a lot of swap space, but in terms of hardware cost it is relatively trivial. This is not to belittle the importance of efficient use of resources but to try to put this situation into proper perspective.

It should also be pointed out that the memory usage of a process is sometims not obvious. The actual allocation of swap space by Sun Java is lazy and so in some cases even if the process is 800mb that does not imply the 800mb of swap space is pinned down for the lifetime of the process. 

There may be room for enhancement of Java's use of local regions of memory (e.g. relinquishing claims on individual VM pages or sets of pages) but it is unlikely that these improvements would manifest as a change in process size as shown by common tools like the "ps" command on Solaris and the relationship to anything other than swap space usage is unclear (and any effects on swap utilization are not a given now anyay). But because of the potential for improvements this RFE should stay open for further investigation when engineering resources are available.

  xxxxx@xxxxx   2002-10-23

We have a similar problem. We have a large server application 
that frequently leaves more than a gigabyte of memory 
reported by freeMemory(); We would like to return this 
memory to the operating system. 

U S E     L O C K E D    M E M O R Y

ALLOW FOR NON-SWAPABLE MEMORY jvm switches, like min 
locked, max locked, lock eden, etc...

Trying running a large program JVM that does not swap, (lock
the memory with tools like memory+): it is very fast.

It does not swap all in from disk to perform a stupid GC of
30 seconds!! I swear, I constantly see 10, 20 or even 30
seconds FullGC on my eclipse IDE.

The jvm and the OS, in this way of working together are
wasting CPU and dsisk I/O.

THIS IS URGENT: no decent server can be written in java
right now, it swaps all the time, even with plenty or
physical ram.

I have exactly the same problem.  I have a client app that is 
only using 8MB of heap, but the heap size stay at 40MB. The 
heap size soars during an expensive load operation and never 
returns to it's original state.  I can't believe this was only 
reported during.  Will this be fixed in 1.4.2_02 or 1.5??

I have the same problem. Even though I have used 
-XX:MaxHeapFreeRatio=20 -XX:MinHeapFreeRatio=10, 
they do not seem to affect the free memory ratio. JVM 
still keeps around 40% free. 
Since my problem occurs in an applet, unlike the first 
comment, I can not stop and restart the applet.

1. The ratio should be forced strictly. Why the -XX 
option would exist if it is not forced?

2. I certainly believe the 70% free ratio for hotspot or 
client JVM is too high. 20% is more reasonable. When 
the server JVM is selected, the ratio can be defaulted to 
a higher value.

3. When the activity of the JVM is low for a long period 
of time, e.g. for 3-4 mins, and the free mem ratio is 
70%, still the heap size should be dropped and the 
free mem should be lowered to 20-30% level.

4. Why the default free min heap ratio is 40%? This 
might be reasonable when your memory allocation is 
1-2MBs but it does not make sense when the memory 
allocation is 100s or 10s of MBs.

In addition to using -XX:MaxHeapFreeRatio=20 -XX:MinHeapFreeRatio=10 I found that it
seems to be necessary to call Runtime.gc()
explicitely to convince it to give back memory
to the OS. And even more than once. Then
it at least goes in the requested direction. Here
is my run() implementation for a Runnable I call
ConvinceGC. The variables count and out
are initialized in the constructor, rt is the Runtime.

  public void run() {
    while( count-->0 ) {
      rt.gc();
      if( out!=null ) {
	long free = rt.freeMemory();
	long total = rt.totalMemory();
	String pcent 
	  = new Double((double)free/(double)total*100.0).toString();
	pcent = pcent.substring(0,pcent.indexOf(".")+2);
	out.println("ConvinceGC: allocated="+total+
		    ", used="+(total-free)+
		    ", free="+free+" ("+pcent+")");
      }
      if( Thread.interrupted() ) break;
      try {
	Thread.sleep(1000);
      } catch( InterruptedException e ) {
	// Assume this asks us to quit working
	break;
      }
    }
    if( out!=null ) out.println("ConvinceGC exiting");
  }

It would be great to have an API to GC to tell it
"Really, dear GC, now is definitively the time to
give back memory to the OS. Please, please...".

-XX:MinHeapFreeRatio and -XX:MaxHeapFreeRatio  defaults should really be lower. The Default of 40 respectively 70% may make sense for small Applications but larger ones operate much faster on most systems with lower Values, because the system needs less I/O (swapping).
Maybe consider allowing absolute values (e.g. 20m).

As long as this RFE is still open, I'll add my 2 cents:

(1) I spent a lot of time trying to figure out why Tomcat wasn't passing the HeapFreeRatio parameters to Java, before finally realizing that Java was probably getting them but just not able to use them.  It would be good to put the appropriate caveats into http://java.sun.com/docs/hotspot/gc5.0/gc_tuning_5.html, saying they can be set but we shouldn't expect them to be obeyed all of the time.

(2) The workaround given in this RFE seems to imply that the cost of virtual memory is just disk space (?).  The more serious issue is disk I/O.  I won't pretend to understand the details of Java's full GC, but full GCs by their nature tend to page in much of the address space, and we have seen serious swapping problems when virtual memory is only 20% larger than physical memory.  

In sum this is a serious practical issue for those of us who use Java in a server environment.  If it isn't possible to fix (and the difficulties pointed out in this RFE's Evaluation are well-taken) then it should be carefully documented in the "Tuning Garbage Collection" guide.

This issue is partially responsible for Java's reputation as a memory hog.

I am the CTO of a company whose main product is a Java desktop application with
tens of thousands of installations running daily. One of the major problems our
users report in feedback and in satisfaction surveys is memory consumption. They
open the task manager (most of them use Windows) and see our application using
200MB of RAM even though most of the time it actually uses only around 40MB (the
200MB are only needed temporarily for operations like resizing an image). Fixing
this bug could resolve this issue and improve consumer perception of Java a lot!
(Even if it doesn't make a big difference technically.) For our purposes, making
the windows task manager showing the 40MB we actually use within the JVM instead
of the 200MB the JVM keeps allocated because we used that much temporarily would
already fix the problem.

Does this bug really apply to all operating systems equally?

It turns out I was wrong and Java does sometimes return memory to the OS, even on Windows. Example:
http://pastebin.org/44909

So I guess it is really a question of optimizing the Java program itself, accompanied by a tuning of the MaxHeapFreeRatio parameter.