United StatesChange Country, Oracle Worldwide Web Sites Communities I am a... I want to...
Bug ID: 7041378 (se) SEGV when running with ulimit -n unlimited (sol)
7041378 : (se) SEGV when running with ulimit -n unlimited (sol)

Details
Type:
Bug
Submit Date:
2011-05-03
Status:
Closed
Updated Date:
2013-04-20
Project Name:
JDK
Resolved Date:
2011-06-22
Component:
core-libs
OS:
solaris_10
Sub-Component:
java.nio
CPU:
generic
Priority:
P2
Resolution:
Fixed
Affected Versions:
6u23
Fixed Versions:
6u27

Related Reports
Relates:
JDK-6772303 - (se) IOException: Invalid argument" thrown on a call to Selector.select(value) with -d64

Sub Tasks

Description
The following bug has been submitted to Oracle's BugDB system:

https://bug.oraclecorp.com/pls/bug/webbug_print.show?c_rptno=12417227

The bug demonstrates a crash on solaris-sparcv9 that arises when running with "ulimit -n unlimited". The problem is that the back-port of 6772303 to 6uX introduces a bug that causes RLIM64_INFINITY to be handled incorrectly. The result is a heap corruption and a subsequent SEGV.

Comments
SUGGESTED FIX

diff --git a/src/solaris/native/sun/nio/ch/DevPollArrayWrapper.c b/src/solaris/native/sun/nio/ch/DevPollArrayWrapper.c
--- a/src/solaris/native/sun/nio/ch/DevPollArrayWrapper.c
+++ b/src/solaris/native/sun/nio/ch/DevPollArrayWrapper.c
@@ -175,7 +175,6 @@ Java_sun_nio_ch_DevPollArrayWrapper_fdLi
         JNU_ThrowIOExceptionWithLastError(env, 
                                           "getrlimit failed");
     }
-    return (jint)rlp.rlim_max;
     if (rlp.rlim_max < 0 || rlp.rlim_max > java_lang_Integer_MAX_VALUE) {
         return java_lang_Integer_MAX_VALUE;
     } else {
2011-05-03 
EVALUATION

This bug does not duplicate with jdk7. Problem is with the back-port of 6772303 to jdk6 (6u18). Easily fixed.
2011-05-03


Hardware and Software, Engineered to Work Together