United StatesChange Country, Oracle Worldwide Web Sites Communities I am a... I want to...
Bug ID: 6401245 Small JDWP packets with the socket transport causes slow debugging on linux 2.6.15 kernel and newer
6401245 : Small JDWP packets with the socket transport causes slow debugging on linux 2.6.15 kernel and newer

Details
Type:
Bug
Submit Date:
2006-03-21
Status:
Resolved
Updated Date:
2010-12-08
Project Name:
JDK
Resolved Date:
2006-04-01
Component:
core-svc
OS:
linux
Sub-Component:
debugger
CPU:
x86
Priority:
P2
Resolution:
Fixed
Affected Versions:
5.0,6
Fixed Versions:
6

Related Reports
Backport:
JDK-2136537 - Small JDWP packets with the socket transport causes slow debugging on linux 2.6.15 kernel and newer

Sub Tasks

Description
FULL PRODUCT VERSION :
java version "1.5.0_06"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_06-b05)
Java HotSpot(TM) Client VM (build 1.5.0_06-b05, mixed mode, sharing)


ADDITIONAL OS VERSION INFORMATION :
Linux kernel 2.6.15 and higher

EXTRA RELEVANT SYSTEM CONFIGURATION :
See Linux Kernel bug and developers response that its actually a JDK but with inappropriate use of Nagles Algorithm. http://bugzilla.kernel.org/show_bug.cgi?id=6177

A DESCRIPTION OF THE PROBLEM :
Debugging under Linux 2.6.15 and higher is horribly slow because of remote debuggings use of many small packets and TCP_NODELAY being set on. (While we can control the socket on the debugger client the socket in the JDK probably *shouldn't* set TCP_NODELAY for debugger sockets or at least their should be an option to not enable TCP_NODELAY) The slow down increases with with size of the debug frame. (The more variables the larger the slow down.)

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
Fire up netbeans, eclipse, Intellij or any debugger and attach to a JDK on Linux while running kernel 2.6.15 or higher. Set some break points and start stepping through code. If the frame has a collection with say 100 items in it step through a for loop of each item. It will take an hour or so. In order to at least partially disable Nagles Algorithm run "sudo sysctl -w net.ipv4.tcp_abc=0" as root and then rerun the test. It will perform as expected.

EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
Should be fast
ACTUAL -
Very slow (hours)

REPRODUCIBILITY :
This bug can be reproduced always.

---------- BEGIN SOURCE ----------
Any code with lots of member variables. A good test is to remote debug the java petstore and set some break points while iterating over code.
---------- END SOURCE ----------

CUSTOMER SUBMITTED WORKAROUND :
run "sudo sysctl -w net.ipv4.tcp_abc=0" as root to disable Nagles prior to debugging. However this disables it system wide and isn't appropriate!!!!

Comments
EVALUATION

--

This will be fixed in Mustang build 79, and in J2SE 5.0 update 8.
Mustang build 79 is scheduled to be available at http://mustang.dev.java.net
by April 10.  The schedule for 5.0 Update 8 has not been finalized.  It will
probably be released in the fall (2006).
2006-03-31 
EVALUATION

--

The Linux kernel version 2.6.15 contains an implementation of RFC
3465.  This is intended to speed up some forms of network usage and
increase security, but it slows down others, JDWP being one of
them. We can avoid this slowdown by changing the back-end agent to not
write lots of small packets.  It shouldn't be doing this in the first
place since each packet has overhead which soaks up bandwidth.

At this time, this problem does not happen on any version of Windows , 
Solaris, or Linux kernel 2.6.14 or earlier.
2006-03-27 
EVALUATION

--

We have a preliminary fix to the socket transport that it writes the JDWP header in a single send. We are currently testing this change.
2006-03-22 
WORK AROUND

CUSTOMER SUBMITTED WORKAROUNDs (unverified):

1) Use a linux kernel before 2.6.15 (this is a change in 2.6.15)

or

2) run "sudo sysctl -w net.ipv4.tcp_abc=0" as root to disable Nagles prior to debugging. However this disables it system wide may not be appropriate.
2006-03-21 
WORK AROUND

--

Another workaround is of course to you another transport library. Since 5.0 there has been a transport interface to allow other transport libraries be used.
2006-03-21 
EVALUATION

Yes, this is a bug in the socket transport library. It appears to have existed since the original JPDA implementation (1998 or 1999) but wasn't reported until now. The socket transport disables nagle but then writes the JDWP packet headers as a sequence of 4 or 5 writes (a write for each field and there are 4 or 5 fields depending on if the packet is a command or reply packet). This needs to be fixed in Mustang and 5.0u8 as it will make debugging impossible for developers that upgrade their Linux kernel.
2006-03-21


Hardware and Software, Engineered to Work Together