FULL PRODUCT VERSION :
  Bug reproduced in 1.4, 1.3.1_01,_03, and _04.

java version "1.3.1_04"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.3.1_04-b02)
Java HotSpot(TM) Client VM (build 1.3.1_04-b02, mixed mode)


FULL OPERATING SYSTEM VERSION :

Windows NT 4.00.1381

A DESCRIPTION OF THE PROBLEM :

My basic requirement is to run two process sequentially in
the same environment, for instance to run a setenv.bat
script and followed by a java app.  The reason for this is
to set up the system environment variables in a script
(using Runtime.exec() passing environment array is not an
option).

I have tried the following

Runtime.exec("setenv.bat && java ...");
Runtime.exec("cmd.exe /K setenv.bat && java ...");
Runtime.exec("run.bat");  // Where run.bat is "setenv.bat &&
java ..."

I have a thread waiting on the return process with
Process.waitFor.  In my main thread, I call
Process.destroy().  Process.waitFor() returns with exit code
1, but the java process continues.

If I kill the java process externally, Process.waitFor()
will also return.

I can not find any workaround for this, and it seriously
limits my ability to launch external applications.


STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
Prepare by creating the batch files specified in the source
section and modifying the path to the batch files both
within the batch files and in the java source file.

1. java ExecTest
The system.out should stop after 5 seconds.

2. java ExecTest and
The system.out will continue after the destroy.

3. java ExecTest script
The system.out will continue after the destroy.


EXPECTED VERSUS ACTUAL BEHAVIOR :
The child processes should be killed, just as if you had hit
control-c.  Instead, it continues (as evidenced by the
printing to the command line).  However, Process.waitFor
DOES return.  I can find no way to successfully kill the
child processes.


REPRODUCIBILITY :
This bug can be reproduced always.

---------- BEGIN SOURCE ----------
// FILE: foo.bat
rem This just makes sure the batch file returns 0 exit code
// END OF FILE: foo.bat

// FILE: test.bat - contains a few alternatives, none of which work
rem Application is not properly killed in any of these cases:

rem Option 1
c:\foo.bat && java com.bluemartini.dev.ExecTestWait

rem Option 2
rem java com.bluemartini.dev.ExecTestWait

rem Option 3
rem call c:\foo.bat
rem java com.bluemartini.dev.ExecTestWait
// END OF FILE: test.bat

// FILE: ExecTest.java
import java.util.*;
import java.io.*;

public class ExecTest {
    private static final int RUN_JAVA = 0;
    private static final int RUN_SCRIPT = 1;
    private static final int RUN_AND = 2;

    private static int state = RUN_JAVA;

    public static void main(String[] args) throws Exception {
        if (args.length > 0) {
            String sState = args[0];
            if (sState.equals("and")) {
                state = RUN_AND;
            } else if (sState.equals("script")) {
                state = RUN_SCRIPT;
            } else if (sState.equals("java")) {
                state = RUN_JAVA;
            }
        }
        new ExecTest();
    }

    public ExecTest() throws Exception {
        // This behaves properly
        Process p;
        if (state == RUN_SCRIPT) {
            // ExecTestWait continues to run after destroy called
            p = Runtime.getRuntime().exec("c:\\test.bat");
        } else if (state == RUN_AND) {
            // ExecTestWait continues to run after destroy called
            p = Runtime.getRuntime().exec("c:\\foo.bat && java
com.bluemartini.dev.ExecTestWait");
        } else {
            // This behaves properly
            p = Runtime.getRuntime().exec("java com.bluemartini.dev.ExecTestWait");
        }

        ProcessWatcher pw = new ProcessWatcher(p) {
            public void outPrint(String s) {
                System.out.println("[" + Thread.currentThread().getName() +
":out] " + s);
            }
            public void errPrint(String s) {
                System.out.println("[" + Thread.currentThread().getName() +
":err] " + s);
            }

            public void processTerminated(int code) {
                System.out.println("[" + Thread.currentThread().getName() + "]
Process ended");
            }
        };

        Thread.sleep(5000);
        System.out.println("Killing process");
        p.destroy();
    }

    public interface TailPipeListener {
        void tailPipeOutput(String s);
    }

    public class ProcessWatcher extends Thread {

        Process proc_;
        Thread outputThread_;
        Thread errorThread_;
        boolean bInterrupt_ = false;

        TailPipe tailPipeOut_;
        TailPipe tailPipeErr_;

        public ProcessWatcher(Process p) {
            proc_ = p;

            tailPipeOut_ = new TailPipe(proc_, false);
            tailPipeOut_.addListener(new TailPipeListener() {
                public void tailPipeOutput(String s) {
                    outPrint(s);
                }
            });

            tailPipeErr_ = new TailPipe(proc_, true);
            tailPipeErr_.addListener(new TailPipeListener() {
                public void tailPipeOutput(String s) {
                    errPrint(s);
                }
            });

            setName("ProcessWatcher");
            start();
        }

        public void run() {
            try {
                tailPipeOut_.follow(200);
                tailPipeErr_.follow(200);
                int exitValue = proc_.waitFor();
                processTerminated(exitValue);
            } catch (InterruptedException e) {
                e.printStackTrace();
            } catch (IOException e) {
                e.printStackTrace();
            } finally {
//                bInterrupt_ = true;
//                tailPipeOut_.stopFollow();
//                tailPipeErr_.stopFollow();
            }
        }

        protected void processTerminated(int code) {
        }

        protected void outPrint(String s) {
        }

        protected void errPrint(String s) {
        }
    }

    public class TailPipe {
        private Vector vListeners_ = new Vector();
        private boolean bFollowing_ = false;
        private PipeThread followThread_;

        File file_;
        Process process_;
        boolean bErrorStream_;

        public TailPipe(File file) {
            file_ = file;
        }

        public TailPipe(Process process, boolean errorStream) {
            process_ = process;
            bErrorStream_ = errorStream;
        }

    //    public void tail(int lines) {
    //    }

    //    public void follow(int lines, int latency) {
    //        bFollowing_ = true;
    //        PipeThread thread = new PipeThread(latency);
    //        thread.start();
    //    }

        public void follow(int latency) throws IOException {
            try {
                followThread_ = new PipeThread(latency);
            } catch (Exception e) {
                if (e instanceof IOException) {
                    throw (IOException)e;
                } else {
                    e.printStackTrace();
                    throw new IOException(e.getMessage());
                }
            }
            bFollowing_ = true;
            followThread_.start();
        }

        public void stopFollow() {
            bFollowing_ = false;
            followThread_.interrupt(); // ### Do I really have to do this?
        }

        private class PipeThread extends Thread {
            int latency_ = -1;
            BufferedReader pipeIn_;
            char[] buf_ = new char[32768];

            public PipeThread(int latency) throws Exception {
                latency_ = latency;
                if (file_ != null) {
                    pipeIn_ = new BufferedReader(new FileReader(file_));
                } else if (process_ != null) {
                    if (bErrorStream_) {
                        pipeIn_ = new BufferedReader(new
InputStreamReader(process_.getErrorStream()));
                    } else {
                        pipeIn_ = new BufferedReader(new
InputStreamReader(process_.getInputStream()));
                    }
                }
            }
            public void run() {
                int charsRead;
                while (bFollowing_) {
                    try {
                        charsRead = pipeIn_.read(buf_);
                        fireTailPipeOutput(new String(buf_, 0, charsRead));
                    } catch(Exception e) {
                        try {
                            pipeIn_.close();
                        } catch (IOException ioe) {
                            ioe.printStackTrace();
                        }
                        bFollowing_ = false;
                    }
                    try {
                        Thread.sleep(latency_);
                    } catch (InterruptedException e) {
                    }
                }
                try {
                    pipeIn_.close();
                } catch (IOException ioe) {
                }
            }
        }

        private void fireTailPipeOutput(String s) {
            Enumeration enum = vListeners_.elements();
            while (enum.hasMoreElements()) {
                TailPipeListener ln = (TailPipeListener)enum.nextElement();
                ln.tailPipeOutput(s);
            }
        }

        public void addListener(TailPipeListener lnr) {
            vListeners_.addElement(lnr);
        }
    }
}
// END OF FILE: ExecTest.java

// FILE: ExecTestWait.java
public class ExecTestWait {
    public static void main(String[] args) throws Exception {
        System.out.println("Sleep process started ...");
        while(true) {
            System.out.println("sleeping");
            Thread.sleep(1000);
            Thread.yield();
        }
    }
}
// END OF FILE: ExecTestWait.java

---------- END SOURCE ----------

CUSTOMER WORKAROUND :
None found.
(Review ID: 165865) 
======================================================================
Posted Date : 2005-09-27 20:00:48.0

For the particular problem addressed by the submitter,
setting environment variables before running an external process,
the new in 1.5.0 ProcessBuilder API is a perfect match.

The following excellent SDN comment explains what's going on:

---------------------------------------------------------------------
The fundamental problem here is that, unlike Unix, Windows does that
maintain parent-child relationships between processes. A process can
kill its own immediate children, but unless you make other
arrangements to obtain the information, can't kill any
'grand-children' because it has no way of finding them. Ctrl-C types
at a command prompt is just a character that the command processor
interprets and not a signal sent from outside. When you 'destroy' a
child command script, that process does not get the opportunity to
terminate any child processes it may know about.

Recent versions of WIndows (2000 or later) do provide a "Job" concept
which acts as a container for processes. Killing a Job does terminate
all processes associated with that job. However Jobs do not contain
other jobs, so fully emulating the Unix behaviour is probably
impossible.
---------------------------------------------------------------------
Note that Unix emulation environments on Windows, like Cygwin,
suffer from the same problem.  Any fix would be difficult.

Even if we could figure out how to fix this, we might choose not to do so
for the usual reason -- compatibility.
Posted Date : 2005-09-27 20:00:48.0

I'm experiencing the same problem, I have a Java program 
running for weeks on Linux (Sun JDK 1.4) and a "ps -u 
myuser"  command shows processes active still performing the 
exec("command") while the proc.waitFor() has already ended.
The only clumsy workaround is to kill these processes by hand.

Harry Metske
h.metske@rf.rabobank.nl

Workaround is to always launch the process with "java xxx"
and pass the environment into Runtime.exec.  To get the
environment requires doing a Runtime.exec("cmd /C set");

I'm having a similar Problem on Windows XP, all that
Process.destroy() seems to do is to Pause the process while
the JVM is still running.  As soon as the JVM exits the
Process resumes until it has terminated, normally or otherwise.

[Also don't understand what aneth means in the reply for a
workaround.]

should also have indicated that I'm using j2sdk1.4.1_02 or
j2re1.4.1_02 on Windows 98 thru XP

I work on a product in Cisco, and it is not causing all the child 
processes to exit.

I am experiencing the same problem with process.destroy(). 
The child sub-processes must be killed, instead, it is still 
running.  Is there a differnet way to successfully kill the
child processes?

Should that really be called bug? Practically you launch two
processes in the poor method which can only return one
process. Also, apparently these two process, shell+batch
file and JVM, do not have parent/child relationship. Could
you claim that it is the correct canonical use of the
Runtime.exec() method? I can't think so, though I think
current behavior of the method is erroneously too generous!

I'll also post my suggested work-around... which is really
nasty and specific to the version of ps installed on our
Linux machine, but it could be adapted:

The StreamGobbler class is another small Thread that gathers
up un-needed output from a process...
class ProcessKiller extends Thread {
    // this provides info on all running processes, with all
a user's
    // processes grouped together in the output. Need to
filter the
    // output to only 'slave' owned accounts that are not
java processes and
    // have been running for more than 60 seconds.
    // The columns displayed are process id, running time,
owner and the
    // command used to start the process. Therefore, the
owner is the third
    // word on a line, while the command is everything after
that, if words
    // are counted as characters with spaces between them.
    private static String[] ps = {"ps", "x", "-eo", "\"%p %t
%U %a\"",
        "--sort", "user"};

    // forms the start of the kill command, process ids are
added on the end
    private static String kill = "kill";

    private Runtime rt;
    private boolean shouldRun = false;

    /**
    * Initialises the killer
    */
    public ProcessKiller(){
        // set Priority to somewhere between the background
and normal
        // in the javadoc MIN_PRIORITY is 1 while
NORM_PRIORITY is 5
        setPriority(Thread.MIN_PRIORITY+2);
        setDaemon(true);
        rt = Runtime.getRuntime();

        String osName = System.getProperty("os.name");
        if(!(osName == null || osName.equals("")) &&
osName.equals("Linux"))
            shouldRun = true;
    }

    /**
    * Only starts execution if being called on a Linux OS,
sleeps for 30
    * seconds between checks on running processes.
    */
    public void run(){
        if(!shouldRun)
            return;
        while(true){
            checkRunningProcesses();
            // sleep for 30 seconds between checks
            try{
                sleep(30000);
            }catch(InterruptedException e){}
        }
    }

    /**
    * Checks non-java slave-owned processes to see there
running time.
    * Because of the way the server is configured, there not
should be any
    * of these processes left running 30 seconds after they
start
    */
    private void checkRunningProcesses(){
        try{
            // run ps, consuming the outputs
            Process psProc = rt.exec(ps);
            StreamGobbler errorGobbler = new
                StreamGobbler(psProc.getErrorStream(), "ERROR");
            errorGobbler.start();
            InputStream in = psProc.getInputStream();
            BufferedReader br = new BufferedReader(
                new InputStreamReader(in));

            StringWriter psOutput = new StringWriter();
            readFromBuffer(br, psOutput);
            // wait for process to finish
            try{
                psProc.waitFor();
            } catch( InterruptedException ie ){}


            br.close();

            // tokeniser converts string into tokens
consisting of lines
            StringTokenizer tok = new
StringTokenizer(psOutput.toString(),
                "\n\r\f");

            int currentPos = 0;
            StringBuffer temp = new StringBuffer();
            while(tok.hasMoreElements()){
                String newTok = tok.nextToken();
            // only keep lines with no occurence of java and
an occurence of slave
            if(newTok.indexOf("java") == -1 &&
                newTok.indexOf("slave") != -1)
                    temp.append(newTok+"\n");
            }

The example here is perhaps a poor one, what about the case
where someone calls a shell script that has been submitted
for some test. This script could be anything, say a student
submission for an assignment which will then have
lecturer-defined tests run on it. In such a case it is
sometimes necessary to forcibly terminate a process if it
has gone on so long that it has failed the test. There needs
to be some way to access information about processes spawned
by the original process and have them destroyed as well,
even if this is not included as part of the functionality of
destroy().

Perhaps a method that returns a Process[] of spawned
processes? Or a destoryThisAndItsFilthySpawn() method?

            String fromGrep = temp.toString();
            // the lines left contain the execution times of
processes,
            // remove lines with less than 30 seconds of
execution

            tok = new StringTokenizer(fromGrep);
            Vector pids = new Vector();
            String prevTok = "", newTok = null;
            while(tok.hasMoreElements()){
                if(newTok != null)
                    prevTok = newTok;
                newTok = tok.nextToken(" \t\n\r\f");

                // first occurence of ':' on a line means
this token is the time
                // as process ids don't contain ':'.
                if(newTok.indexOf(':') != -1){
                    int numberOfSecs = parseSecs(newTok);

                    // previous token is the process id
                    if(numberOfSecs > 30 &&
tok.nextToken().indexOf("slave") != -1)
                        pids.add(prevTok);

                    // move tokeniser to end of line
                    tok.nextToken("\n\r\f");
                }
            }

            // pids vector contains the process id's of all
processes owned by
            // slave but not java and running for more than
30 seconds
            if(pids.size() == 0)
                return;

            String[] killCmd = new String[pids.size()+1];
            killCmd[0] = kill;
            for(int i = 0; i<pids.size(); i++){
                killCmd[i+1] = (String) pids.get(i);
            }

            // run the kill command
            Process killProc = rt.exec(killCmd);
            StreamGobbler err = new
StreamGobbler(killProc.getErrorStream(),
                "ERROR");
            StreamGobbler outGob = new
StreamGobbler(killProc.getInputStream(),
                "OUTPUT");
            err.start();
            outGob.start();

        } catch(IOException ioe){
            System.err.println(ioe.getMessage());
        }
    }

    private void readFromBuffer(final BufferedReader br,
            final StringWriter retBuff){
        Runnable r = new Runnable(){
            public void run(){
                String line = null;
                try{
                    while((line = br.readLine()) != null){
                        retBuff.write(line.replaceAll("\"",
"")+"\n");
                    }
                }catch(IOException ioe){
                    System.err.println(ioe.getMessage());
                }
            }
        };
        Thread reader = new Thread(r);
        reader.start();
    }

    private int parseSecs(String time){
        // input is in form dd-hh:mm:ss, where days and
hours are optional
        StringTokenizer tok = new StringTokenizer(time, ":-");
        int numToks = 0;
        try{
            numToks = tok.countTokens();
            int days = 0, hours = 0, minutes = 0, seconds = 0;
            if(numToks == 4)
                days = Integer.parseInt(tok.nextToken());
            if(numToks > 2)
                hours = Integer.parseInt(tok.nextToken());
            minutes = Integer.parseInt(tok.nextToken());
            seconds = Integer.parseInt(tok.nextToken());

            return seconds + (minutes*60) + (hours*60*60) +
(days*24*60*60);
        }catch(NoSuchElementException nsee){
            return 0;
        }
    }

I am eagerly waiting for the resolution of this bug, as it 
is really stopping me from using external processes 
correctly.

The key is not so much that sub-child processes are killed explicitly by java but that the termination handler of the exec()ed process is being by-passed.

Java should provide the means for being able to cause an exec()ed processes termination handler to be invoked. 

I would really enjoy a destroyWithChildren method on the Runtime class.
It would also be very, very useful if the Runtime class had a getPid method?  Why doesn't it?

As it is defined today, the destroy() method is not related to waitFor() and to inputStream.read() methods. The semantics must be coherent: call on destroy() from one thread must unlock any other thread waiting on waitFor() and/or waiting on inputStream.read() methods.

Also, it would be nice to add methods that waitFor() / destroy() a process tree. Currently, it seems that destroy() working on the process itself, while waitFor() waits for the process tree.

Also, add methods to get java.nio.Channel, not input stream, so it will be possible to have non-blocking implementation as well. Again, everything must be coherent.

Also, add support to run external process and set its input/output to null device. This is needed for tasks that I do not want to read their output, but I do not want them to hang as well.

Also, make ProcessBuilder a serializable class. It is just a container for parameters.

Thanks,
  Doron Rajwan.

The fundamental problem here is that, unlike Unix, Windows does that maintain parent-child relationships between processes. A process can kill its own immediate children, but unless you make other arrangements to obtain the information, can't kill any 'grand-children' because it has no way of finding them. Ctrl-C types at a command prompt is just a character that the command processor interprets and not a signal sent from outside. When you 'destroy' a child command script, that process does not get the opportunity to terminate any child processes it may know about.
Recent versions of WIndows (2000 or later) do provide a "Job" concept which acts as a container for processes. Killing a Job does terminate all processes associated with that job. However Jobs do not contain other jobs, so fully emulating the Unix behaviour is probably impossible.

The fundamental problem here is that, unlike Unix, Windows does that maintain parent-child relationships between processes. A process can kill its own immediate children, but unless you make other arrangements to obtain the information, can't kill any 'grand-children' because it has no way of finding them. Ctrl-C types at a command prompt is just a character that the command processor interprets and not a signal sent from outside. When you 'destroy' a child command script, that process does not get the opportunity to terminate any child processes it may know about.
Recent versions of WIndows (2000 or later) do provide a "Job" concept which acts as a container for processes. Killing a Job does terminate all processes associated with that job. However Jobs do not contain other jobs, so fully emulating the Unix behaviour is probably impossible.

What are the arguments against providing reliable killing of processes on Windows by using the Job API behind the scenes on platforms where this is possible?

Something like:
if(OS == Win2000 OR higher)
   createJobObject()

It would seem that Windows does have a way of obtaining all the children and descendants of a process.  It involves forming a tree of all running processes and looking at the parent pid of each.

Of note is that some processes lose their parent and thus point to an invalid id.  However since Windows may reuse this parent id it seems that a child process could point to something that really isn't its parent.

This pitfall can be worked around by looking at the creation timestamp of the process and the supposed parent process.  If the parent was created later than the child then it isn't the real parent.  The process has no parent at that point.  It is an orphan.

One example tool I found written in C++ is at http://www.scheibli.com/projects/getpids/index.html that demonstrates how to look for all processes and the parents.

I've only tested on Windows XP and Vista so I'm not sure earlier versions of Windows may work.  It's better than nothing though.

My above post would not require using the Jobs api.