United StatesChange Country, Oracle Worldwide Web Sites Communities I am a... I want to...
Bug ID: 6631048 Problem when writing on output stream of HttpURLConnection
6631048 : Problem when writing on output stream of HttpURLConnection

Details
Type:
Bug
Submit Date:
2007-11-16
Status:
Closed
Updated Date:
2011-03-08
Project Name:
JDK
Resolved Date:
2011-03-08
Component:
core-libs
OS:
windows_xp
Sub-Component:
java.net
CPU:
x86
Priority:
P4
Resolution:
Fixed
Affected Versions:
6
Fixed Versions:
7

Related Reports
Backport:
Relates:

Sub Tasks

Description
FULL PRODUCT VERSION :
java version "1.6.0_05-ea"
Java(TM) SE Runtime Environment (build 1.6.0_05-ea-b06)
Java HotSpot(TM) Client VM (build 1.6.0_05-ea-b06, mixed mode, sharing)

ADDITIONAL OS VERSION INFORMATION :
Microsoft Windows XP [Version 5.1.2600]
(C) Copyright 1985-2001 Microsoft Corp.

A DESCRIPTION OF THE PROBLEM :
when using the HttpURLConnection's output stream to write data we are facing problem. The chunking size is set to 48K.  This piece of code works well in jdk 5. but the same code fails for jdk 6. Looking into the source code, there seems a 10K  threshold limit is introduced in jdk 6. This piece of code is creating problem.

I have attached a test program to reproduce the problem. The program works fine for JDK 5 without any problem. The exception stack trace is:

java.lang.ArrayIndexOutOfBoundsException
	at java.lang.System.arraycopy(Native Method)
	at sun.net.www.http.ChunkedOutputStream.write(ChunkedOutputStream.java:161)
	at sun.net.www.protocol.http.HttpURLConnection$StreamingOutputStream.write(HttpURLConnection.java:2492)
	at com.documentum.ucf.common.transport.impl.TestHttpURLConnection.writeData(TestHttpURLConnection.java:63)
	at com.documentum.ucf.common.transport.impl.TestHttpURLConnection.testHttpUrlConnectionWrite(TestHttpURLConnection.java:45)
	at com.documentum.ucf.common.transport.impl.TestHttpURLConnection.main(TestHttpURLConnection.java:26)

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
1. compile the source
2. Run it with jdk 1.5 - you wont see any problem
3. Run it with jdk 6 - you will find the problem.

You can connect to a Test servlet, in which case also you can see the problem.


ERROR MESSAGES/STACK TRACES THAT OCCUR :
C:\Trials\Test\src>java TestHttpURLConnection
Writing buf len, len: 32120,32120
java.lang.ArrayIndexOutOfBoundsException
        at java.lang.System.arraycopy(Native Method)
        at sun.net.www.http.ChunkedOutputStream.write(Unknown Source)
        at sun.net.www.protocol.http.HttpURLConnection$StreamingOutputStream.write(Unknown Source)
        at TestHttpURLConnection.writeData(TestHttpURLConnection.java:60)
        at TestHttpURLConnection.testHttpUrlConnectionWrite(TestHttpURLConnection.java:42)
        at TestHttpURLConnection.main(TestHttpURLConnection.java:24)

REPRODUCIBILITY :
This bug can be reproduced always.

---------- BEGIN SOURCE ----------
import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.net.HttpURLConnection;
import java.net.URL;

public class TestHttpURLConnection
{

    public static void main (String[] args)
    {
        TestHttpURLConnection testHttpURLConnection = new TestHttpURLConnection();
        try
        {
            testHttpURLConnection.testHttpUrlConnectionWrite();
        }
        catch (Exception e)
        {
            e.printStackTrace();
        }
    }

    public void testHttpUrlConnectionWrite () throws Exception
    {
        //Test - If you want to , change it to the url of a test servlet in any app server, the problem will be there even otherwise.
        URL url = new URL("http://some_host:8080");

        HttpURLConnection urlConnection = open(url);

        setChunkingMode(urlConnection, CHUNK_SIZE);

        OutputStream out = urlConnection.getOutputStream();

        writeData(out);

        out.close();
    }

    private void writeData (OutputStream out) throws IOException
    {
        //you can replace this with any file or simply a byte[] data
        String file = "C:\\tmp\\test_1m.pdf";

        InputStream is = new FileInputStream(file);

        byte[] buf = new byte[BUFFER_SIZE];
        int len = 0;

        while ((len = is.read(buf)) >= 0)
        {
            System.out.println("Writing buf len, len: " + buf.length + "," + len);
            out.write(buf, 0, len);
        }

        is.close();
    }

    private HttpURLConnection open (URL url) throws IOException
    {
        HttpURLConnection urlConnection = (HttpURLConnection) url.openConnection();
        urlConnection.setDoInput(true);
        urlConnection.setDoOutput(true);
        return urlConnection;
    }

    private void setChunkingMode (HttpURLConnection urlConnection, int chunkSize) throws IOException
    {
        urlConnection.setChunkedStreamingMode(chunkSize);
    }

    /**
       * Test comments - I played with the BUFFER_SIZE and CHUNK_SIZE values, but that didn't fix the problem.
       */
    private static final int BUFFER_SIZE = 32120;

    private static final int CHUNK_SIZE = 48 * 1024;

}


---------- END SOURCE ----------

CUSTOMER SUBMITTED WORKAROUND :
I coudn't find anything. By changing the buffer size and chunk size, you may reduce the problem, but happens though.

                                    

Comments
EVALUATION

I ran the testcase on jdk5.0u13 and jdk1.6.0_03 and here is the output I see:

jdk1.5.0_13 output: 
-------------------

Writing buf len, len: 32120,32120
Writing buf len, len: 32120,32120
Writing buf len, len: 32120,32120
java.io.IOException: Error writing request body to server
        at sun.net.www.protocol.http.HttpURLConnection$StreamingOutputStream.che
ckError(HttpURLConnection.java:2285)
        at sun.net.www.protocol.http.HttpURLConnection$StreamingOutputStream.wri
te(HttpURLConnection.java:2268)
        at TestHttpURLConnection.writeData(TestHttpURLConnection.java:54)
        at TestHttpURLConnection.testHttpUrlConnectionWrite(TestHttpURLConnectio
n.java:36)
        at TestHttpURLConnection.main(TestHttpURLConnection.java:17)


jdk1.6.0_03 output: 
-------------------

Writing buf len, len: 32120,32120
java.lang.ArrayIndexOutOfBoundsException
        at java.lang.System.arraycopy(Native Method)
        at sun.net.www.http.ChunkedOutputStream.write(ChunkedOutputStream.java:1
61)
        at sun.net.www.protocol.http.HttpURLConnection$StreamingOutputStream.wri
te(HttpURLConnection.java:2513)
        at TestHttpURLConnection.writeData(TestHttpURLConnection.java:54)
        at TestHttpURLConnection.testHttpUrlConnectionWrite(TestHttpURLConnectio
n.java:36)
        at TestHttpURLConnection.main(TestHttpURLConnection.java:17)


The submitter has claimed that the testcase runs fine on 5.0 which is not what I observe here. Could you please clarify and let me know what version of 5.0 did you try out with?
                                     
2007-12-17
WORK AROUND

The workaround is to write to the output stream in chunks less than 10K. For example:
  os.write (buf, 0, (10 * 1024));
  os.write (buf, (10 * 1024), (10 * 1024));
  os.write (buf, (20 * 1024), (10 * 1024));
                                     
2007-12-17
EVALUATION

I don't think that the IOException being thrown from 1.5 is a problem. This just indicates that the server is not accepting the data or has closed the connection. This could be the case if the server is not setup correctly. I have tested against a valid server and the test program completes successfully with 1.5 and throws ArrayIndexOutOfBoundsException with 6.0.

This appears to be a regression introduced from the fix for 6526165. The fix for 6526165 causes byte array writes greater than 10K ( 10 * 1024 bytes ) to potentially by pass the internal buffering of ChunkedOutputStream and write directly to the underlying stream. The reason this fails is that in the case described in the description section, the chunk size is greater than the byte array being written, and both are greater than 10K. The internal logic of ChunkedOutputStream.write does not expect this. 

The fix is simply to determine if the amount of data being written is greater than both 10K and what is remaining in the current chunk. Otherwise simply write it to the internal buffer.
                                     
2007-12-17



Hardware and Software, Engineered to Work Together