FULL PRODUCT VERSION :
java version "1.5.0_06"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_06-b05)
Java HotSpot(TM) Client VM (build 1.5.0_06-b05, mixed mode, sharing)


ADDITIONAL OS VERSION INFORMATION :
 customer  Windows XP
Professional Version 2002
Service Pack 2


A DESCRIPTION OF THE PROBLEM :
The XSL Transformation code in the standard JDK has a severe memory leak, which leads to an OutOfMemory exception when it is executed multiple times in a program.  To replicate, run the test program.

When the start button is pressed in the test program, it creates and runs a new thread which performs an XSL transformation.  This transformation reads an input XML file, and then generates an output HTML file, based on the rules XSLT file which is also read in.  Looking at the test program, you will notice that there are no dangling references to any of the transformation objects.  Start the test program under a profiler, and then monitor the memory usage.  For each successive click on the start button, the memory profile of the program exhibits a class memory leak i.e. memory usage grows until the JVM runs out of memory, whereby an OutOfMemory Exception is thrown.

I have included a sample input XML file and rules XSLT file, however any valid XML and appropriate XSLT document will do.  You may wish to use a larger XML file, and run the test program with a small JVM memory size, to more easily show the memory leak.

I have looked at the JVM heap under a profiler, and I believe that the bug ultimately lies in com.sun.org. customer .xml.internal.utils.XMLReaderManager.  This class is a factory, which creates instances of XMLReader when getXMLReader() is called.  The class also caches created XMLReaders, on a per-thread basis.  When a thread is finished with an XMLReader, it calls releaseXMLReader() - XMLReaderManager marks its cached instance of XMLReader as being free, but hangs onto the instance.  This is the problem, as the XMLReader has a reference to the complete source XML document that it has just transformed, as well as some other data structures.

The problem gets worse when the thread that performed the XSL transformation dies.  The cached instance of XMLReader in XMLReaderManager is automatically freed, as per the semantics of the ThreadLocal var that is fine.  But the  Hashtable in XMLReaderManager still hangs onto a ref to the XMLReader instance, because the instance is a key to an entry in that table.  So, if you start up a background thread to perform the transformation, and you do this multiple times as in the test program, you have multiple instances of XMLReader and the complete XML document being hung onto, and never freed.  Hence the OutOfMemory exception.

As a comparison, you can comment out the creation of the background thread in the test program, and just do the transformation in the Swing thread, when the start button is pressed.  You will notice in the profiler that only one instance of XMLReader is being held onto by XMLReaderManager, and successive clicks on the start button does not lead to an OutOfMemory exception; the XMLReader though still holds onto the last XML document it transformed, when really that memory should be released post the transformation.

I could be wrong about the ultimate source of the bug being XMLReaderManager i.e. the bug instead could be that reset() in not being called appropriately on the cached XMLReaders, which is meant to empty its state.  I am not wrong though about a memory leak in the XSL Transformation.


STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
See above notes.


EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
See above notes.
ACTUAL -
See above notes.

REPRODUCIBILITY :
This bug can be reproduced always.

---------- BEGIN SOURCE ----------
The test program is:

import java.awt.*;
import java.awt.event.*;
import java.io.*;
import javax.swing.*;
import javax.xml.transform.Transformer;
import javax.xml.transform.TransformerFactory;
import javax.xml.transform.stream.StreamResult;
import javax.xml.transform.stream.StreamSource;

public class TestXSLTransform extends JFrame {

  public TestXSLTransform() {
    init();
    validate();
    setVisible(true);
  }

  public static void main(String[] args) {
    SwingUtilities.invokeLater(new Runnable() {
      public void run() {
        new TestXSLTransform();
      }
    });
  }

  private void init() {
    addWindowListener(new WindowAdapter() {
      public void windowClosing(WindowEvent evt) {
        System.exit(0);
      }
    });

    getContentPane().setLayout(new BorderLayout());

    JButton btn = new JButton("Start");
    btn.addActionListener(new ActionListener() {
      public void actionPerformed(ActionEvent e) {
        Thread t = new Thread(new Runnable() {
          public void run() {
            transform();
          }
        });

        t.start();
      }
    });


    getContentPane().add(btn, BorderLayout.CENTER);
    setSize(new Dimension(200, 100));
  }

  private void transform() {
    System.out.println("start transform");

    try {
      InputStream in = new BufferedInputStream(new FileInputStream("input.xml"));
      OutputStream out = new BufferedOutputStream(new FileOutputStream("output.html"));
      InputStream rulesIn = new BufferedInputStream(new FileInputStream("rules.xsl"));

      TransformerFactory tFactory = TransformerFactory.newInstance();
      Transformer transformer = tFactory.newTransformer(new StreamSource(rulesIn));

      transformer.transform(new StreamSource(in), new StreamResult(out));

      in.close();
      out.close();
      rulesIn.close();

    } catch (Exception e) {
      System.out.println("exception = " + e);
    }

    System.out.println("end transform");
  }

}


-----------------------------------

The input.xml file is:

<?xml version='1.0' encoding='utf-8'?>
<report>
<body>
<table>

<table_row>
<table_cell><cell_value><![CDATA[In  customer ]]></cell_value><cell_type>string</cell_type></table_cell>
<table_cell><cell_value><![CDATA[Unit Pricing]]></cell_value><cell_type>string</cell_type></table_cell>
<table_cell><cell_value><![CDATA[TEST_PAUL]]></cell_value><cell_type>string</cell_type></table_cell>
<table_cell><cell_value><![CDATA[30-Dec-2005]]></cell_value><cell_type>string</cell_type></table_cell>
<table_cell><cell_value><![CDATA[1]]></cell_value><cell_type>string</cell_type></table_cell>
<table_cell><cell_value><![CDATA[System]]></cell_value><cell_type>string</cell_type></table_cell>
<table_cell><cell_value><![CDATA[366162]]></cell_value><cell_type>string</cell_type></table_cell>
<table_cell><cell_value><![CDATA[05-May-2006 10:52:39]]></cell_value><cell_type>string</cell_type></table_cell>
<table_cell><cell_value><![CDATA[0]]></cell_value><cell_type>string</cell_type></table_cell>
<table_cell><cell_value><![CDATA[19305345]]></cell_value><cell_type>string</cell_type></table_cell>
<table_cell><cell_value><![CDATA[0]]></cell_value><cell_type>string</cell_type></table_cell>
<table_cell><cell_value><![CDATA[0]]></cell_value><cell_type>string</cell_type></table_cell>
<table_cell><cell_value><![CDATA[server]]></cell_value><cell_type>string</cell_type></table_cell>
</table_row>

</table>
</body>
</report>

-----------------------------------

The rules.xsl file is:

<?xml version="1.0"?>
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0">
  <xsl:output method="html" indent="yes"/>

  <xsl:template match="report">
    <html>
      <xsl:apply-templates/>
    </html>
  </xsl:template>

  <xsl:template match="body">
    <body>
      <xsl:apply-templates/>
    </body>
  </xsl:template>

  <xsl:template match="table">
    <table border="1">
      <xsl:apply-templates/>
    </table>
  </xsl:template>

  <xsl:template match="table_cell">
    <xsl:choose>
      <xsl:when test="cell_value != ''">
        <td>
          <xsl:value-of select="cell_value"/>
        </td>
      </xsl:when>
      <xsl:otherwise>
        <td>
          -
        </td>
      </xsl:otherwise>
    </xsl:choose>
  </xsl:template>

  <xsl:template match="table_row">
    <tr>
      <xsl:apply-templates/>
    </tr>
  </xsl:template>

</xsl:stylesheet>

---------- END SOURCE ----------

CUSTOMER SUBMITTED WORKAROUND :
I do not have a work-around.
Posted Date : 2006-06-06 20:56:31.0

N/A

I'm able to reproduce this using JDK 5_06, jconsole, a max heap size of 5 M. However, this does not appear to be reproducible using JDK 6 (Mustang) beta (I started over 200 threads using the GUI button). If Xerces is the problem, we should look for any patches applied to Mustang that may be backported to JDK 5.
Posted Date : 2006-06-08 15:33:04.0

To respond to ashokM's comments:

This patch had been integrated into JDK5 since update 12. It's possible that you could have hit a different issue. Could you contact support with a testcase that shows the issue you're having?

Thanks
Posted Date : 2008-06-18 07:25:38.0

This is a serious problem for us.  We have a J2EE application that uses less than 250M when running with Java 1.4.2_10.  When we try to run with Java 1.5.0_06, this exact same application runs out of memory.  Profiling shows that the XMLReaderManager is the culprit.  This is stopping us from upgrading our application with plans on using new functionality based on Java 5 features.

After painstaking JProbing, we think that the reason why we run out of memory on our server application, is the xsl transformations.  We do xsl fo printing on the server, and everytime a user prints, some memory is lost.  This really sucks, because our clients are coming back to us complaining the system stopped working.

We really do need an immediate fix and a jdk 1.5 update.  Is there any possibility that someone spends some time on this?  This *is* a pretty serious bug, because right now:

"Java can not be used to do ongoing xsl processing".  OUCH !

I think I figured out a workaround.  Take xalan.jar and serializer.jar from the latest apache xalan project (2.7.0), and run the JVM with the following options.  This will effectively replace the default transformer factory implementation, without having to override javax.xml classes (not legal, only apache gets away with this ? ;-)...

java -cp .:xalan.jar:serializer.jar -Djavax.xml.transform.TransformerFactory=org.apache.xalan.xsltc.trax.TransformerFactoryImpl TestXSLTransform

Problem also confirmed with 1.5.0_09

Workaround :

Ok, this is not a perfect Workaround, but it helps to minimize
the Leak. The trick is to 'feed' an empty XML-Stream to
the transformer before you discard the transformer.

Try :

StringReader sr = new StringReader("<?xml version=\"1.0\" encoding=\"ISO-8859-1\"?><DUMMY/>");
transformer.transform(new StreamSource(sr), new StreamResult(new StringWriter()));

at the end of your procedure/thread. For me this was sufficient as my JProfiler shows. Only a few unrecoverable bytes were left on the heap and not the megabytes from my huge XML-document.

okidoky's 9/20/06 workaround fixes the problem.   magnus42's 10/23/06 workaround does not minimize the problem as much as is claimed; it leaks more than just "a few unrecoverable bytes" and leads to a rapid crash in my application.  also, as reported, jdk6 fixes the problem.

Do we have this fix in latter JDK1.5 version ( > JDK1.5.0_11)?? Or is there any plan to fix it in JDK1.5.x!!.

Is this problem same across all platforms, even we are seeing the same issue in Solaris!!

We are also trying with the comments posted by okidoky (by replacing the xalan jar).

-ashokM

We have tried with jdk1.5 u15, but its not working. Where as the workaround by replacing the xalan.jar is working. Its in solaris platform.

-ashokM

This is a huge issue for us right now. We have a big web application which uses jdk1.5 and we are seeing a out of memory exception. After heap analysis and profiling we found the issue is with the xalan version. Also I tried to copy the xalan.jar from the xalan site, but it did not work either. Instead it gave me the following problem.

exception is java.lang.NoClassDefFoundError: com.sun.org.apache.xpath.internal.XPathAPI
        at org.apache.commons.digester.Digester.createSAXException(Digester.java:2919)
        at org.apache.commons.digester.Digester.createSAXException(Digester.java:2945)
        at org.apache.commons.digester.Digester.endElement(Digester.java:1133)
        at org.apache.xerces.parsers.AbstractSAXParser.endElement(Unknown Source)
        at org.apache.xerces.impl.dtd.XMLDTDValidator.endNamespaceScope(Unknown Source)
        at org.apache.xerces.impl.dtd.XMLDTDValidator.handleEndElement(Unknown Source)
        at org.apache.xerces.impl.dtd.XMLDTDValidator.endElement(Unknown Source)
        at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanEndElement(Unknown Source)
        at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatcher.dispatch(Unknown Source)
        at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source)
        at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
        at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
        at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
        at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source)
        at org.apache.commons.digester.Digester.parse(Digester.java:1685)
        at org.mule.config.builders.AbstractDigesterConfiguration.process(AbstractDigesterConfiguration.java:101)


How should I resolve this issue? It seems that the package sturucture has been changed for the XMLPathAPI as a result my Mule does not start.