FULL PRODUCT VERSION :
1.6.14_b01

ADDITIONAL OS VERSION INFORMATION :
 customer  Windows XP [Version 5.1.2600]

A DESCRIPTION OF THE PROBLEM :
We are using java.nio for communication and have no caching of direct byte buffers. When testing build 1.6.0_14-ea-b01 with G1 we experienced very soon OutOfMemoryErrors:

Switching off (or: not on) G1garbage collector fixes the problem.

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
NIO communication with HeapByteBuffer's

EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
No OutOfMemoryError
ACTUAL -
java.lang.OutOfMemoryError: Direct buffer memory

ERROR MESSAGES/STACK TRACES THAT OCCUR :
java.lang.OutOfMemoryError: Direct buffer memory
at java.nio.Bits.reserveMemory(Bits.java:633)
at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:95)
at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:288)
at sun.nio.ch.IOUtil.write(IOUtil.java:134)
at sun.nio.ch.SocketChannelImpl.write0(SocketChannelImpl.java:365)
at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:388)
at de.espirit.firstspirit.server.io.GatheringWriteBuffer.write(GatheringWriteBuffer.java:34)
...

java.lang.OutOfMemoryError: Direct buffer memory
at java.nio.Bits.reserveMemory(Bits.java:633)
at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:95)
at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:288)
at sun.nio.ch.Util.getTemporaryDirectBuffer(Util.java:57)
at sun.nio.ch.IOUtil.write(IOUtil.java:69)
at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:334)
at de.espirit.firstspirit.server.io.DirectChannelIO.write(DirectChannelIO.java:31)
...

REPRODUCIBILITY :
This bug can be reproduced always.

CUSTOMER SUBMITTED WORKAROUND :
Switch of (or: not on) G1 garbage collector
Posted Date : 2009-03-03 05:36:34.0

N/A

The stack trace indicates that the application is doing gathering writes with an array containing heap buffers. Prior to jdk7, the scatter/gather operations weren't integrated with the buffer caching and so there is a direct buffer allocated to shadow each non-direct buffer in the array. This problem has been resolved in jdk7 as part of the socket-channel completion work (see 6781363 and 6465310). Note that this behavior is independent of the collector (not specific to G1). Nelson is going to contact the submitter and ask them to test with jdk7 b50 or newer.
Posted Date : 2009-03-04 11:29:34.0

6465310 "(fc) scatter/gather operations on non-direct buffers doesn't use buffer cache" is marked as DUPLICATE of 6781363 but the changes concerning the issue described there should be backported.