When the named pipe transport was implemented for Windows under
6651507, some deadlocks were discovered due to poor behavor of the
Windows APIs. Workarounds were developed at the time involving the
addition of more threads, as well as an initial write buffer to hold
the results of the first few messages from the browser to the client
attached JVM, which was found to be necessary at one point in the
development to avoid deadlocks.
Investigation of the customer's report for this bug indicated that
this initial write buffer was getting corrupted. Initially we believed
that the data being sent over the named pipe was being corrupted, but
detailed investigation revealed that this was not the case. The
corruption occurred at the point when the initial write buffer filled
up and we were forced to wait for a connection from the client in
order to flush it to the named pipe.
Thanks to ###@###.### for the idea to completely remove the
initial write buffer, which is possible at this time because the
background connection thread already present handles the deadlock case
the initial write buffer was originally intended to fix. My original
fix involved expanding the initial write buffer to a list of buffers,
which also solved the problem but added significant complexity, now
understood to be unnecessary.
Some other minor cleanups were made in the WindowsNamedPipe
implementation in the spirit of robustness although they were not the
root cause of this bug.
While investigating this bug, two other issues were uncovered. First,
the filtering of disabled JREs in the JVMManager was occasionally
removing the wrong JRE in the case that two different copies of the
same JRE were present in deployment.properties, one disabled, one
enabled. Second, there was a lurking synchronization bug in the
SerializingTransport base class where subclasses could inadvertently
break the critical synchronization around serialization of messages.
This is very likely the reason the earlier SocketTransport was failing
on the same test case.
Also deleted a few files that were intended to have been deleted in
This fix was tested with the regression tests for related bugs 6659079
and 6661386 as well as a new targeted test case.