One licensee reported this issue on their customer's application.

JDK: 1.4.1_03, 1.4.2_xx
OS:  Windows Server 2003

Below is collected stack trace.
----->
java.lang.StackOverflowError
	at com.sun.tools.javac.v8.comp.Resolve.resolveIdent(Resolve.java:798)
	at com.sun.tools.javac.v8.comp.Attr.visitIdent(Attr.java:1033)
	at com.sun.tools.javac.v8.tree.Tree$Ident.accept(Tree.java:1111)
	at com.sun.tools.javac.v8.comp.Attr.attribTree(Attr.java:256)
	at com.sun.tools.javac.v8.comp.Attr.attribArgs(Attr.java:320)
	at com.sun.tools.javac.v8.comp.Attr.visitApply(Attr.java:756)
	at com.sun.tools.javac.v8.tree.Tree$Apply.accept(Tree.java:877)
         ....
<-----

The javac compiling environment is as follows:

the total number of Java source program(put in the dir at each package): 411
the number of the referred jar(set at -classpath): 17

set cp=<jar files>
javac -classpath %cp% -d d:\biscomp\bin -sourcepath d:\biscomp\src d:\biscomp\src\....\*.java

Put the above command in each package.

The licensee tried the option "-J-Xss10m" but it did not work. After
their investigation they found the following conclusion.

1.The cause of StackOverFlowError

When compiling a source(javac ... *.java), the classes depending on
the compiled source become translation objects if not existing on
classpath. There is a max limit of source/class number set to be 502
when analyzing a source code. Analytical information that javac holds
internally grows to translate a huge amount of source program at a
time and that results in a stack overflow.

2.Why the option "-J-Xss10m" did not work

The thread where stack overflow occurred was generated by OS when
starting javac.exe. Its size was fixed to be 256KB on Windows so
"-J-Xss10m" did not work. This option is valid only for the threads
generated by Java VM.

To solve the problem, one way is not to make a recursive call in the
processing of javac. 

This problem was not seen in jdk 5.0.
Posted Date : 2007-10-31 03:57:57.0

Test case and instruction for reproduction has been added.
Please see Comment section.
Posted Date : 2007-11-27 10:13:04.0

N/A

Fixing by starting the javac "work" in a new thread, to avoid any possible limitation of the primordial thread stack size by the OS.  This applies to 5.0 and earlier, later JDKs have the launcher start the application in a new thread.
Posted Date : 2008-08-25 19:28:25.0