Alexey Gibadullin,   xxxxx@xxxxx  

While working on the bug

    4637504 (process) REGRESSION: java.lang.Process does not clean up file 
resources
    
I have extended Test.java from the bugreport to Test2.java

public class Test2 {
    public static void main(String argv[]) {
        int threads = Integer.parseInt(argv[0]);
        int processes = Integer.parseInt(argv[1]);        
        
        for (int i = 0; i < threads; i++)
            (new MyThread(i, processes)).start();
    }
}

class MyThread extends Thread {
    private int number;
    private int processes;    
    
    MyThread(int n, int p) {
        number = n;
        processes = p;
    }

    public void run() {
        for (int i = 0; i < processes; i++) {
            try {
                Runtime runtime = Runtime.getRuntime();
                Process process = runtime.exec("echo " + i);
                process.waitFor();
            } catch (Exception e) {
                System.out.println("Thread " + number + " got " + e);
                e.printStackTrace(System.out);
                return;
            }
            System.out.println("Thread " + number + ". Process " + i);
        }
    }
}

Test2 reveals that the bug #4637504 still present in Mantis-b07, if processes
are executed from a thread (4637504 was integrated into hopper).

% uname -a
SunOS novo152 5.8 Generic_108528-16 sun4u sparc SUNW,Ultra-5_10

% ../jdk1.4.2-b07/solaris-sparc/bin/java Test2 1 400
Thread 0. Process 0
Thread 0. Process 1
Thread 0. Process 2
...
Thread 0. Process 334
Thread 0. Process 335
Thread 0. Process 336
Thread 0. Process 337
Thread 0 got java.io.IOException: Too many open files
java.io.IOException: Too many open files
        at java.lang.UNIXProcess.forkAndExec(Native Method)
        at java.lang.UNIXProcess.<init>(UNIXProcess.java:54)
        at java.lang.Runtime.execInternal(Native Method)
        at java.lang.Runtime.exec(Runtime.java:566)
        at java.lang.Runtime.exec(Runtime.java:428)
        at java.lang.Runtime.exec(Runtime.java:364)
        at java.lang.Runtime.exec(Runtime.java:326)
        at MyThread.run(Test2.java:24)
        
You may also increase number of threads. java.io.IOException is also thrown, 
but it needs less number of processes:

% ../jdk1.4.2-b07/solaris-sparcv9/bin/java -d64 -server Test2 3 200
Thread 0. Process 0
Thread 1. Process 0
Thread 2. Process 0
Thread 0. Process 1
Thread 1. Process 1
Thread 2. Process 1
Thread 2. Process 2
Thread 0. Process 2
...
Thread 0. Process 108
Thread 1. Process 107
Thread 2. Process 110
Thread 0. Process 109
Thread 1. Process 108
Thread 2. Process 111
Thread 2. Process 112
Thread 0. Process 110
Thread 1. Process 109
Thread 2. Process 113
Thread 0. Process 111
Thread 1. Process 110
Thread 0 got java.io.IOException: Too many open files
java.io.IOException: Too many open files
        at java.lang.UNIXProcess.forkAndExec(Native Method)
        at java.lang.UNIXProcess.<init>(UNIXProcess.java:54)
        at java.lang.Runtime.execInternal(Native Method)
        at java.lang.Runtime.exec(Runtime.java:566)
        at java.lang.Runtime.exec(Runtime.java:428)
        at java.lang.Runtime.exec(Runtime.java:364)
        at java.lang.Runtime.exec(Runtime.java:326)
        at MyThread.run(Test2.java:24)
Thread 1 got java.io.IOException: Too many open files
java.io.IOException: Too many open files
        at java.lang.UNIXProcess.forkAndExec(Native Method)
        at java.lang.UNIXProcess.<init>(UNIXProcess.java:54)
        at java.lang.Runtime.execInternal(Native Method)
        at java.lang.Runtime.exec(Runtime.java:566)
        at java.lang.Runtime.exec(Runtime.java:428)
        at java.lang.Runtime.exec(Runtime.java:364)
        at java.lang.Runtime.exec(Runtime.java:326)
        at MyThread.run(Test2.java:24)
Thread 2. Process 114
Thread 2 got java.io.IOException: Too many open files
java.io.IOException: Too many open files
        at java.lang.UNIXProcess.forkAndExec(Native Method)
        at java.lang.UNIXProcess.<init>(UNIXProcess.java:54)
        at java.lang.Runtime.execInternal(Native Method)
        at java.lang.Runtime.exec(Runtime.java:566)
        at java.lang.Runtime.exec(Runtime.java:428)
        at java.lang.Runtime.exec(Runtime.java:364)
        at java.lang.Runtime.exec(Runtime.java:326)
        at MyThread.run(Test2.java:24)

The test fails on Solsparc and Solsparcv9 (5.8) and passes 
on Windows2000, Linux Red Hat 7.2, Solx86 (5.8), Solsparc and Solsparcv9 (5.9).

======================================================================





Alexey Gibadullin,   xxxxx@xxxxx  

If I add

    process.destroy();                
    
after

    process.waitFor();
    
the test passes fine even on Solsparc 5.8. However, the spec for
java.lang.Process says:

    public abstract int waitFor() throws InterruptedException
    
    causes the current thread to wait, if necessary, until the process
    represented by this Process  customer  has terminated. This method returns
    immediately if the subprocess has already terminated. If the subprocess 
    has not yet terminated, the calling thread will be blocked until the
    subprocess exits. 
    
    
    public abstract void destroy()
    
    Kills the subprocess. The subprocess represented by this Process  customer  
    is forcibly terminated. 
    
Therefore, if the subprocess exits and waitFor() returns the exit value, 
there is no need to call destroy(), since destroy() does nothing but killing
the subprocess.



======================================================================




JCK tests execution fails due to this bug. If JDK1.4.1 (b-21) or
JDK1.4.2 (b-10) is used to run JavaTest harness then a lot of JCK tests
fail in multiJVM mode with "java.io.IOException: Too many open files".

This bug can be observed in case of one thread. 
To do this execute following test:
------------- test.java ----------------------
public class test {
    public static void main(String argv[]) {
        int i = 0;
        try {
            for(i = 0; i < 100000; i++) {
                Process p =  Runtime.getRuntime().exec("ls");
            }
        } catch (java.io.IOException e) {
            System.out.println("Iteration: " + i + "  Exception: " + e);
        }
    }
}
------------- log ----------------------------
$javac test.java
$
$java -version
java version "1.4.2-beta"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.2-beta-b10)
Java HotSpot(TM) Client VM (build 1.4.2-beta-b10, mixed mode)
$
$uname -a
SunOS novo49 5.8 Generic_108528-15 sun4u sparc SUNW,Ultra-2
$
$java -Xfuture test
Iteration: 338  Exception: java.io.IOException: Too many open files
----------------------------------------------




======================================================================

N/A

Will not fix.  The current behavior dates back to the rewrite of the Process
code back in 1.2/1.3 which removed the dangerous buffering of all subprocess
output.  In order to release all resources, user code must either invoke
Process.destroy or manually close the three subprocess streams.

I've submitted 4801027 to request clarification of the Process spec on this
point.

--   xxxxx@xxxxx   2003/1/9

The related bug fix says that all the streams are closed on
destroy(). The
same needs to happen when you call waitFor() or exitValue()
because
the process is no longer running after a successful call.
This bug definitely does not exist in JDK 1.3.1.

FWIW, I don't agree with this resolution. When waitFor() or
exitValue() returns successfully, the process has already
terminated - there is no buffering issue. As mentioned in my
earliser comment, JDK 1.3.1 does
not have this bug.

Even though you can be sure that the process has terminated with a call to waitFor and exitValue, this does not imply that the JVM has closed the fileDescriptors is has open to received the output generated by the process.  This is the reason that we get Too Many Open Files exceptions.

destroy() ensures those file descriptors get closed.

I got this error from using ColdFusion MX on Windows2003 version.  Wonder if you have tested with this Windows2003 o/s?  

Just ran into this myself, in 1.4.2.  Why not make it clear in the Javadoc for Process that destroy must be called even after waitFor returns?  That didn't seem obvious to me.