FULL PRODUCT VERSION :
$java -version

java version "1.4.1_01"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.1_01-b01)
Java HotSpot(TM) Client VM (build 1.4.1_01-b01, mixed mode)

FULL OPERATING SYSTEM VERSION :
Slackware 8.0
$uname -a
Linux ranch 2.4.18 #3 Fri Apr 26 23:24:18 EDT 2002 i686 unknown
$ls /lib/libc-*
/lib/libc-2.2.3.so

EXTRA RELEVANT SYSTEM CONFIGURATION :
RAM = 385376K
Swap = 128484K

A DESCRIPTION OF THE PROBLEM :
VM crashes when run with the following command:

java -verbose:gc -Xmx1895m GCTest

VM error message:

Unexpected Signal : 11 occurred at PC=0x403E4EB5
Function=start_thread__2osP6Thread+0x2D
Library=/usr/local/j2sdk1.4.1_01/jre/lib/i386/client/libjvm.so

Current Java thread:

Dynamic libraries:
08048000-0804e000 r-xp 00000000 03:06 17800
/usr/local/j2sdk1.4.1_01/bin/java
0804e000-0804f000 rw-p 00005000 03:06 17800
/usr/local/j2sdk1.4.1_01/bin/java
40000000-40015000 r-xp 00000000 03:09 111638
/lib/ld-2.2.3.so
40015000-40016000 rw-p 00014000 03:09 111638
/lib/ld-2.2.3.so
4001f000-4002d000 r-xp 00000000 03:09 111658
/lib/libpthread-0.9.so
4002d000-40035000 rw-p 0000d000 03:09 111658
/lib/libpthread-0.9.so
40035000-40038000 r-xp 00000000 03:09 111646
/lib/libdl-2.2.3.so
40038000-40039000 rw-p 00002000 03:09 111646
/lib/libdl-2.2.3.so
40039000-40140000 r-xp 00000000 03:09 111641
/lib/libc-2.2.3.so
40140000-40146000 rw-p 00106000 03:09 111641
/lib/libc-2.2.3.so
4014a000-40462000 r-xp 00000000 03:06 17287
/usr/local/j2sdk1.4.1_01/jre/lib/i386/client/libjvm.so
40462000-40616000 rw-p 00317000 03:06 17287
/usr/local/j2sdk1.4.1_01/jre/lib/i386/client/libjvm.so
40626000-40637000 r-xp 00000000 03:09 111649
/lib/libnsl-2.2.3.so
40637000-40639000 rw-p 00010000 03:09 111649
/lib/libnsl-2.2.3.so
4063b000-4065c000 r-xp 00000000 03:09 111647
/lib/libm-2.2.3.so
4065c000-4065d000 rw-p 00020000 03:09 111647
/lib/libm-2.2.3.so
4065d000-40666000 r-xp 00000000 03:06 17281
/usr/local/j2sdk1.4.1_01/jre/lib/i386/native_threads/libhpi.so
40666000-40667000 rw-p 00008000 03:06 17281
/usr/local/j2sdk1.4.1_01/jre/lib/i386/native_threads/libhpi.so
40668000-40678000 r-xp 00000000 03:06 17291
/usr/local/j2sdk1.4.1_01/jre/lib/i386/libverify.so
40678000-4067a000 rw-p 0000f000 03:06 17291
/usr/local/j2sdk1.4.1_01/jre/lib/i386/libverify.so
4067a000-4069b000 r-xp 00000000 03:06 17292
/usr/local/j2sdk1.4.1_01/jre/lib/i386/libjava.so
4069b000-4069d000 rw-p 00020000 03:06 17292
/usr/local/j2sdk1.4.1_01/jre/lib/i386/libjava.so
4069d000-406b2000 r-xp 00000000 03:06 17294
/usr/local/j2sdk1.4.1_01/jre/lib/i386/libzip.so
406b2000-406b4000 rw-p 00014000 03:06 17294
/usr/local/j2sdk1.4.1_01/jre/lib/i386/libzip.so
406b4000-41d82000 r--s 00000000 03:06 225858
/usr/local/j2sdk1.4.1_01/jre/lib/rt.jar
41dc5000-41ddc000 r--s 00000000 03:06 225789
/usr/local/j2sdk1.4.1_01/jre/lib/sunrsasign.jar
41ddc000-41e4d000 r--s 00000000 03:06 225791
/usr/local/j2sdk1.4.1_01/jre/lib/jsse.jar
41e4d000-41e60000 r--s 00000000 03:06 225790
/usr/local/j2sdk1.4.1_01/jre/lib/jce.jar
41e60000-4211c000 r--s 00000000 03:06 225856
/usr/local/j2sdk1.4.1_01/jre/lib/charsets.jar
441c4000-441ce000 r-xp 00000000 03:09 111650
/lib/libnss_compat-2.2.3.so
441ce000-441d0000 rw-p 00009000 03:09 111650
/lib/libnss_compat-2.2.3.so

Local Time = Thu Nov 14 11:00:08 2002
Elapsed Time = 0
#
# HotSpot Virtual Machine Error : 11
# Error ID : 4F530E43505002E6
# Please report this error at
# http://java.sun.com/cgi-bin/bugreport.cgi
#
# Java VM: Java HotSpot(TM) Client VM (1.4.1_01-b01 mixed mode)
#
# An error report file has been saved as hs_err_pid1638.log.
# Please refer to the file for further information.
#
Aborted

GCTest.java:

class GCTest {
  public static void main(String[] args) {
    Obj test = new Obj(1000000000);
   }
  private static class Obj {
    private byte[] data;
    public Obj(int size) {
      data = new byte[size];
    }
  }
}

Reducing Xmx by 1M eliminates the crash:
$java -verbose:gc -Xmx1894m GCTest
[GC 280K->138K(1984K), 0.0117300 secs]
[Full GC 138K->138K(1984K), 0.0475560 secs]
[Full GC 138K->138K(1984K), 0.0450460 secs]
Exception in thread "main" java.lang.OutOfMemoryError

Crashes occur for Xmx of 1895 or 1896. With Xmx of 1897 or
more, the following message is displayed:
Fatal: Unable to create signal thread (may be due to
resource limit)
Aborted

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
1.javac GCTest.java
2.java -verbose:gc -Xmx1895m GCTest
3.

EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED:
Fatal: Unable to create signal thread (may be due to
resource limit)
Aborted

ACTUAL:
crash

ERROR MESSAGES/STACK TRACES THAT OCCUR :
Unexpected Signal : 11 occurred at PC=0x403E4EB5
Function=start_thread__2osP6Thread+0x2D
Library=/usr/local/j2sdk1.4.1_01/jre/lib/i386/client/libjvm.so

Current Java thread:

Dynamic libraries:
08048000-0804e000 r-xp 00000000 03:06 17800      /usr/local/j2sdk1.4.1_01/bin/java
0804e000-0804f000 rw-p 00005000 03:06 17800      /usr/local/j2sdk1.4.1_01/bin/java
40000000-40015000 r-xp 00000000 03:09 111638     /lib/ld-2.2.3.so
40015000-40016000 rw-p 00014000 03:09 111638     /lib/ld-2.2.3.so
4001f000-4002d000 r-xp 00000000 03:09 111658     /lib/libpthread-0.9.so
4002d000-40035000 rw-p 0000d000 03:09 111658     /lib/libpthread-0.9.so
40035000-40038000 r-xp 00000000 03:09 111646     /lib/libdl-2.2.3.so
40038000-40039000 rw-p 00002000 03:09 111646     /lib/libdl-2.2.3.so
40039000-40140000 r-xp 00000000 03:09 111641     /lib/libc-2.2.3.so
40140000-40146000 rw-p 00106000 03:09 111641     /lib/libc-2.2.3.so
4014a000-40462000 r-xp 00000000 03:06 17287
/usr/local/j2sdk1.4.1_01/jre/lib/i386/client/libjvm.so
40462000-40616000 rw-p 00317000 03:06 17287
/usr/local/j2sdk1.4.1_01/jre/lib/i386/client/libjvm.so
40626000-40637000 r-xp 00000000 03:09 111649     /lib/libnsl-2.2.3.so
40637000-40639000 rw-p 00010000 03:09 111649     /lib/libnsl-2.2.3.so
4063b000-4065c000 r-xp 00000000 03:09 111647     /lib/libm-2.2.3.so
4065c000-4065d000 rw-p 00020000 03:09 111647     /lib/libm-2.2.3.so
4065d000-40666000 r-xp 00000000 03:06 17281
/usr/local/j2sdk1.4.1_01/jre/lib/i386/native_threads/libhpi.so
40666000-40667000 rw-p 00008000 03:06 17281
/usr/local/j2sdk1.4.1_01/jre/lib/i386/native_threads/libhpi.so
40668000-40678000 r-xp 00000000 03:06 17291
/usr/local/j2sdk1.4.1_01/jre/lib/i386/libverify.so
40678000-4067a000 rw-p 0000f000 03:06 17291
/usr/local/j2sdk1.4.1_01/jre/lib/i386/libverify.so
4067a000-4069b000 r-xp 00000000 03:06 17292
/usr/local/j2sdk1.4.1_01/jre/lib/i386/libjava.so
4069b000-4069d000 rw-p 00020000 03:06 17292
/usr/local/j2sdk1.4.1_01/jre/lib/i386/libjava.so
4069d000-406b2000 r-xp 00000000 03:06 17294
/usr/local/j2sdk1.4.1_01/jre/lib/i386/libzip.so
406b2000-406b4000 rw-p 00014000 03:06 17294
/usr/local/j2sdk1.4.1_01/jre/lib/i386/libzip.so
406b4000-41d82000 r--s 00000000 03:06 225858
/usr/local/j2sdk1.4.1_01/jre/lib/rt.jar
41dc5000-41ddc000 r--s 00000000 03:06 225789
/usr/local/j2sdk1.4.1_01/jre/lib/sunrsasign.jar
41ddc000-41e4d000 r--s 00000000 03:06 225791
/usr/local/j2sdk1.4.1_01/jre/lib/jsse.jar
41e4d000-41e60000 r--s 00000000 03:06 225790
/usr/local/j2sdk1.4.1_01/jre/lib/jce.jar
41e60000-4211c000 r--s 00000000 03:06 225856
/usr/local/j2sdk1.4.1_01/jre/lib/charsets.jar
441c4000-441ce000 r-xp 00000000 03:09 111650     /lib/libnss_compat-2.2.3.so
441ce000-441d0000 rw-p 00009000 03:09 111650     /lib/libnss_compat-2.2.3.so

Local Time = Thu Nov 14 11:11:25 2002
Elapsed Time = 0
#
# HotSpot Virtual Machine Error : 11
# Error ID : 4F530E43505002E6
# Please report this error at
# http://java.sun.com/cgi-bin/bugreport.cgi
#
# Java VM: Java HotSpot(TM) Client VM (1.4.1_01-b01 mixed mode)
#
# An error report file has been saved as hs_err_pid1814.log.
# Please refer to the file for further information.
#
Aborted

REPRODUCIBILITY :
This bug can be reproduced always.

---------- BEGIN SOURCE ----------
class GCTest {
  public static void main(String[] args) {
    Obj test = new Obj(1000000000);
   }
  private static class Obj {
    private byte[] data;
    public Obj(int size) {
      data = new byte[size];
    }
  }
}
---------- END SOURCE ----------

CUSTOMER WORKAROUND :
change Xmx
(Review ID: 167052) 
======================================================================

N/A

I can't reproduce the crash on my Redhat 7.3 box. 

Note a similar bug (4669640) was fixed for 1.4.1. However, it's possible that
there might be other cases that we don't check the result of memory allocation
and (incorrectly) assume allocation is always successful during initialization.
That could cause this crash.

Given that the crash only happens with -Xmx1895 or -Xmx1896 on Slackware, and
the "fix" is to let VM die more gracefully rather than a crash, I don't think 
it is a Mantis showstopper and I'm commiting it to Tiger - we need
to look through the code and make sure VM can die gracefully under low memory
conditions.

  xxxxx@xxxxx   2002-12-04

-----------------------------

This is the same problem as 4669640. When creating an internal thread, JVM
incorrectly assumes it must succeed. Apparently the problematic code has
been copied to several places in JVM:

compileBroker.cpp  (fixed by patc)
jvmtiEnv.cpp
jvmpi.cpp
os.cpp             (fixed by huanghui)
lowMemoryDetector.cpp

We still need to fix jvmtiEnv.cpp, jvmpi.cpp and lowMemoryDetector.cpp

  xxxxx@xxxxx   2003-11-18

------------------------------

Fix is to gracefully shut down JVM if we can't proceed without the thread
(e.g. compiler thread, signal thread, etc.); otherwise return an error
code to the caller (jvmti, jvmpi).

  xxxxx@xxxxx   2003-11-20

It may occur in Windows Only,User Ap can only use <2GB 
memory.