The testcase below can demonstrate that Full GC kicks in when old generation is only 54% of its capacity. The heap size should be set to 3GB
public class CoreGenerator implements Runnable
static Random generator = new Random();
private String genString( int length )
final byte space = 32;
final int range = 128 - 32;
char newMessage = new char[ length ];
for ( int j = 0; j < length; j++ )
newMessage[ j ] = (char)( space + (byte) generator.nextInt( range ) );
return new String( newMessage );
public void run()
HashMap hm = new HashMap( 1024 );
final int maxLength = 4 * 1024 * 1024;
for ( int i = 1; i < 2 * 1024 * 1024; i++ )
hm.put( new Integer( i ), genString( generator.nextInt( maxLength ) ) );
if (0 == ( i % 20 ) )
hm.put( new Integer( generator.nextInt( i ) ), genString( generator.nextInt( maxLength ) ) );
public static void main( String args )
for ( int i = 0; i < 20; i++ )
new Thread( new CoreGenerator() ).start();
Run this test program with foillowing options:
java -DORB.OrbName=CoreTest -Xms3g -Xmx3g -XX:NewSize=300m -XX:MaxNewSize=300m -XX:SurvivorRatio=4 -XX:+PrintCompilation -XX:+UseParallelGC -XX:ParallelGCThreads=24 -XX:MaxPermSize=256MB -XX:PermSize=256m -XX:-UseAdaptiveSizePolicy -XX:+DisableExplicitGC -XX:+PrintTenuringDistribution -XX:+PrintHeapAtGC -XX:+PrintGCTimeStamps -Xnoclassgc -verbose:gc CoreGenerator
Note that ParalleGC is used for young gen.
Peter & Ramki also looked at this a bit. On the hashmap resizing, when a
hashmap (or hashtable) fills up, it expands by doubling in size.
> ... It looks like it's just going to run out of
> memory, ...
Ah, it does run out of memory. And since it doesn't
have catch blocks, the OutOfMemoryError kills off
one of the 20 threads that they start, which frees
up a pile and a half of space, which allows the other
threads to continue.
I noticed their use of a HashMap in the test and wonder
if this may be related to HashMap resizing that causes
a _huge_ allocation request at some point sufficiently
far into the run... Might be useful to have the
HashMap print out its size periodically, so one
can track its resizing (or have verbose gc print out
the allocation request that triggered a collection).
Email from Rajesh; CBOE is not using CMS:
Because of some confusion earlier, I said customer was using Concurrent
collector. I just confirmed with them and they are NOT. They are just
using UseParallelGC only.
I has asked CBOE to send us the GC logs and will forward you as soon as
I get them
This particular problem occurs because they are running with AdaptiveSizePolicy
disabled and a large heap with -Xms == -Xmx.
AdaptiveSizePolicy::_promo_size is set in the AdaptiveSizePolicy constructor to
the initial size of the old gen. They are running with -Xms==-Xmx==3g and
-XX:-UseAdaptiveSizePolicy, so _promo_size is initialized to a large value and
then never updated.
AdaptiveSizePolicy::should_full_GC() is used even when UseAdaptiveSizePolicy is
false, and it contains this calculation which overflows:
size_t upper_bound = _live_at_last_full_gc + _promo_size;
The result is a bogus small value of upper_bound, which causes should_full_GC()
to return true, so a full GC is done instead of just a scavenge.