[dougr 28-Aug-97] I am running:

	 java -fullversion
	java full version "jvm111_23n:97.06.09"

Currently there is no poll() equivalent in the
Java-net (or/and IO) classes.

I am developing the Java version of the rpc.cmsd. One of the goals
is 10,000 connections. Estimate FCS in Mar-98.

However, without a poll like feature in the Java classes, I have
to create one thread for each connection. Then in each thread
block in a read system call. This limis to to less than 2,000
connections.

Or I have to snag some of the Socket source code, grab
the socket() fd, and do the poll with a native method.

There are performance problems with one thread per connection
and blocking reads:

  Measured results on a dual-50mhz SS10 64MB:

  1) Java native threads, no poll:

    Almost 7000-8000 Sockets can be open if they do *nothing* at all.
   OR
    About  2000 Sockets can be in a blocking read() call.

    At which point the OS is swaping so much it takes an L1-A to get out.

  2) With a 'C' program and poll() (select is limited to 1024 fd's):

    10,000 Sockets can be open.
   AND
    10,000 Sockets can be in poll() waiting for data.

    and the system is still usable.

  3) With a 'C' program and NOT using poll()

    10,000 Sockets can be open
   AND
    Almost 2,000 Sockets can be in a blocking read() call, then
    the program exits with a error indication that there was no
    more resources (I forgot the exact errno).


Also,
With HTTP 1.1(?) and connections that stay alive, I would think that
the Java-Web-Server will be running into the same problem.


 1998-01-21   xxxxx@xxxxx  

Request from Ed Randall   xxxxx@xxxxx   is to add the KEEPALIVE
timeout notifications to this event/listenr mechanism.





J:\borsotti\jtest>java -version
java version "1.2"
Classic VM (build JDK-1.2-V, native threads)
Consider the following program:

import java.io.*;
public class Th {
    static Thread killer;
    static Thread waster;

    class Killer extends Thread {
        public void run() {
            System.out.println("killer running");
            try {
                sleep(3000);
                System.out.println("killer interrupts waster" +
                    waster.isInterrupted());
                waster.interrupt();
                System.out.println("killer has interrupted waster" +
                    waster.isInterrupted());
            } catch (InterruptedException e) {
                System.out.println("killer interrupted! " + isInterrupted());
            }
            System.out.println("killer DONE!");
        }
    }
    class Waster extends Thread {
        public void run() {
            System.out.println("waster running");
            try {
                FileOutputStream f = new FileOutputStream("tmp.tmp");
                byte[] ba = new byte[10000000];
                f.write(ba);
            } catch (IOException e) {
                System.out.println("waster ioexception!" + isInterrupted());
            }
            System.out.println("waster DONE!");
        }
    }

    public static void main (String[] args) {
        Th t = new Th();
        killer = t.new Killer();
        waster =  t.new Waster();
        waster.start();
        killer.start();
    }
}


when executed it prints:

J:\borsotti\jtest>java Th
waster running
killer running
killer interrupts wasterfalse
killer has interrupted wastertrue
killer DONE!
^C

J:\borsotti\jtest>dir tmp.tmp
 Volume in drive J is \\tlvhgr\dknmpm3
 Volume Serial Number is 0000-0000

 Directory of J:\borsotti\jtest

12/07/99  04:16p            10,000,000 tmp.tmp
               1 File(s)     10,000,000 bytes

in this run, the program has been activated, the
"waster" thread started to write a long file, the
"killer" thread tried to kill it without success,
and the control-C took effect only at the end of
the execution (as shown by the file size).

Apparently, the write() methods is not aborted as
a result of interrupting the executing thread.
A similar test, in which write() was replaced by
read() shows the same behaviour.

This makes impossible to implement applications in
which some control (e.g. a "stop" button) is provided
to allow the user to abort a long i/o operation which
was wrongly started
========================


Subject: Re: (Review ID: 98744) Incomplete handling of i/o interruption.

Hello Mark,

in my example program, the killer in indeed making
a waster.interrupt(). I.e. it is interrupting a
thread, which is making an io operation.
The purpose, and the meaning of "interruption" has
always been in most system that of making a thread
(process or whatever independently running activity)
which was blocked in some long operation to be
awakened to complete it.
The method sleep() implements it correctly: it stop
sleeping.
Note that ideally, there would be no need for the
concept of interruption if all threads never get blocked
for long time: instead of it they could a sequence of
smaller operation and test in between a flag which could
be set by other threads to interrupt them.
Even a sleep() can be broken in smaller sleep()'s.
This, however, besides being inefficient, it makes also
programs more complex, and is by no means a safe solution
since nobody prevents a thread to make a long operation.
Without interruptions, you have no way to stop a program
which is flooding your terminal with output you no longer
want to see, and that by chance has been written by a 
programmer who did not know that interruptions did not
behave as commonly expected.

I thus should reinforce my request to have interruptions behave
as they should be.

Best Regards

Angelo Borsotti.
(Review ID: 98744)
======================================================================

N/A

This requires significant API changes.  It's too late to do this in JDK 1.2.
--   xxxxx@xxxxx   1/5/1998

This is being addressed as part of the "New I/O" effort under JSR-51.
The new API will appear in Merlin (= J2SE 1.4).  Please see 4313882.

--   xxxxx@xxxxx   2001/5/5

THIS IS A MAJOR PROBLEM.  I HAVE BEEN REQUESTING A FIX FOR THIS SINCE
JDK1.0.X!!!  Your that it's too late won't get sympathy from me.  Please change
the API so that Server developers can get some relief.  I don't know how you
expect full on development without this ability?

Non-blocking IO is also needed to combine AWT
with listening to a socket.
A good API design should enable the programmer to
register FileEventListeners (either on Java 
sockets etc. or concerning native fileDescriptors)
Then the JVM should do an optimized polling, and
create a FileEvent invoking the registered 
FileEventListeners, whenever data is available.
This should not interfere with any other activity
namely waiting for and processing AWT-Events.
Controlling the poll in Java is unacceptable 
because of performance.
Note that FileEventListeners should be registered
relative to a FileDescriptor because this class
is suitable for generalizing over Java sockets,
native sockets etc.
Finally, Socket needs to make its SocketImpl 
accessible (for access to the FileDescriptor)
and it should be possible to create a 
FileDescriptor as a wrapper for a socket, opened
by native code.

As a solution (I have not tried it) you 
might use is.available() instead of is.read(), 
or set a very short timeout for is.read().
Additionally, if Sun plans to do something about
this in the future, they might consider a 
"multi-block" (I forget the academic name) where 
a single thread is blocked on multiple read()'s!
For this you really need OS / Hardware events.

I have just hit the same problem. How is Java supposed to work as a server side
solution if there is a limit of 2000 clients per server ?
Very not impressed that this problem has not been mentioned anywhere else.
the .isAvailable works but causes the processor to go mad polling the IO
streams for nothing 99% of the time 

Start a Java extension package for the solution to this problem... I'm talking
about the javax.* packages... I'm thinking javax.highspeed.* which will be the
foundation for new functionality that uses native libraries to harness the
power of the OS and processor architecture, and in our case the network...
As I'm currently coding a server program, this feature would be extremely
helpful, to the point of dramatically reducing the number of machines we deploy
in our server farm.

Anyone interested in this bug will want to read
http://www.javaworld.com/jw-03-1999/jw-03-volanomark.html
Anyone who still doesn't believe that select()-style
servers are the way to go for high throughput
should probably go have a look at 
http://www.acme.com/software/thttpd/benchmarks.html
which shows Java web server performance is
almost two orders of magnitude slower than
C web servers for some tasks.
The client-per-thread model that Sun likes
is certainly good and should be optimized, but 
Sun really needs to add a multiple-client-per-thread
ability to the Java platform.
Sun's goal here should be, as Apache says,
to give Java web servers enough performance
that benchmark scores are not a reason to pick
a C web server over a Java web server.

In case anyone has code available for using
select() from java (JDK 1.2, WinNT preferably),
pretty please, drop me a mail!

I agree that this is a very important issue for
server scalability.  I don't think that the 
poll()/select() interface is very nice though.
Something like NT CompletionPorts seems better.
Check out the interfaces provided by ACE, 
http://www.cs.wustl.edu/~schmidt/ for more 
information.

I'm collecting notes about this and
related issues at http://www.kegel.com/c10k.html
I challenge Sun to demonstrate a Java web
server that can handle a fraction of the 
punishment that ftp.cdrom.com handles daily.

PLEASE include a fix for this bug pending
for over 26 months in the next MINOR bug fix
release, as we cannot wait until a Java 1.3
appears. I know that this is an API change, but
it's only an extension, it doesn't change
or deprecate old behaviours.

In some cases, the lack of a poll() method means
that each client connection needs TWO threads
(not just one).   One for reading and one
for writing.
Consider the case where a ill-mannered
client might request a high volume of data and
then  stop servicing its input socket.   The data
flow could back up to the point where any
attempts for the  server to write data to the
client blocks.  If a single thread were attempting
to write to service multiple clients, one rouge
client (or een well-implemented clients suffering
from network problems) could stop all
data from flowing.
In the current implementation, 
the only way to prevent this calamity is to have 
a separate thread for each client doing the
writing.   Now if we also need a separate
thread to do the reading, we would require
two threads per client.   
So the cost of not having a poll() may escalate
even faster than earlier posters claimed.

Like adam@organic.com, I would like to see the notifications tied to completion
of specific I/O requests instead of to a general file "ready" state. 
This allows two outstanding requests of the same type on the same file and
simplifies a lot of threading headaches.

I've successfully been using the available()function to find the number of
bytes that can be read without blocking and then reading that number
afterwards. For most purposes I can think of, it is sufficient to poll, say,
once every 1/10th second, so this workaround is usuable in most situations.
Sending data is another matter.
In both cases, Java needs to provide an officially supported path; this is a
very important issue.

I lead an effort to tune an application server which was written entirely in
Java. It was losing a significant amounts of performance to context switching
and stability problems occured as the load was increased. The benefits of using
a Client/Server design with a few threads acting as agents on behalf of many
users was well understood and the architecture was organized  accordingly. The
socket layer was a problem even though some of the work-arounds described here
had been tried.
We reimplemented the socket layer using native
methods to interface with Win/NT's I/O completion
port APIs. Maximum throughput doubled and performance degraded gracefully. The
system was stable even with four times the load which had previously been
untenable.
My personal opinion is that this type of interface should be provided from Java
and the APIs should use an agent based mail-box scheme similar to I/O
completion ports. I believe it is a more efficient design than select; however,
I think we can do better in terms of the bookkeeping that is required of the
user. Note, this same technology works very well for handling the large numbers
of files often associated with databases.
Developers shouldn't have to use native methods to get good results.
Bill Smith
Santa Clara Applications Solution Center
Intel Corporation

There are two issues brought up in this bug.
1. an RFE
2. an implicit BUG REPORT
The fix for the bug report is suggested in the
email. 
Have the JVM internally use poll(), instead of
read() directly.
This requires ZERO API changes.
 
Note the "C code using poll()" vs 
"C code not using poll()" results given.

.isAvailable combined with sleep should work 
fine.  for writing data, pop a thread
from a thread pool and use it for writing.  
this is not really a problem, though I agree
a sugary API would be nice.

If a non blocking API is to be developed I would like to
see a general mechanism that can be used by many different
API's to notify the completion of an operation.
In C++ I have used a class AssyncNotifier as a base
class serving as a way of notifing the caller when
the operation has been completed. In Java the java.io
API could be extended so that each operation has a
version of say read() that takes an object of
AssyncNotifier and calls this objects notify() method
whenever the requested operation has been completed.
The read() call would itself return imediately.
Also there must be a mechanism to have one or several
threads to await the notification of several AssyncNotifiers.
This is like the select()/poll() call in UNIX or even more
like the more general WaitForMulipleObjects()
in Win32)
Since the AssyncNotifier would be a general mechanism 
each application program could benefit from it and use
it for there own use just like waiting for events like
the reception of character from serial device, 
key pressed, disk io completed, socket connected
and so on - all without the need to use one or several threads
for each of these operations.

using isAvailable() with sleep() is nowhere
near the efficiency of having the
JVM use poll() automatically, instead of
blocking read().

This is a major bug. How we can develope serious and real server side
applications
with this limit ?
On Digital Alpha JDK the limit is 500 blocking socket and preclude any further
development.

There is a workaround for reading from a socket but no such workaround for
writing to a socket.
For reading, one can simply setSoTimeout() and the read() on the socket's input
stream will timeout in
the given amount of time.  This is not elegant, but it will work.
For writing, there is no workaround.  OutputStreams don't have any available()
or ready() methods to check if
they're ready for writing.  It is not feasible to use another thread if you
have several thousand connections.  Even if
I have a dedicated timer thread that interrupts the writing thread after a
given interval, I still don't know how many
bytes were written before timeout.  write() should return the number of bytes
written.

xpetos, You are describing the 'observer pattern' used in several places in the
JDK api.
How about this for a work around:  Farm out VM proccesses that will service
client requests.
If 1 VM instance can support 500, 2 can support 1000, etc etc up to the number
of 
VM processes that the server can support.
-Chris

So what about JDK 1.3??  This is a huge problem for high connection volume
server applications.  This has gone on too long (your "too late for"
is at the beginning of 98, almost two years ago).  Please attend to this in JDK
1.3.  Thanks. 

Anybody listening there?  Time to wake up.  It is
rather silly to leave as a last comment that it is
too late for 1.2 and then not even reevaluate it
at 1.3.
This combined with the ongoing lack of things like
generics and const references required for safety 
and the recent Sun revelations that Sun was never 
committed to standards efforts leave me looking 
for a more-open successor to Java.

Yoohoo Sun ? This is the 21st century now... JDK 1.3 is just around the bend...
I suppose someone's going to tell us it's "too late to do this in JDK
1.3" ?

This doesn't require API changes, it requires a new class implementation.
Some native code to hide the details is also, obviously needed.
Let's see, I could do this in about 2 days and debug it in another 2 and be
done with it inside of a week.   What's the deal SUN?

I wonder if it is possible to arrange for something like select()
to work on FileDescriptor objects...?

Poll by looping around InputStream.available()
and read only that many bytes.   With select() you don't know which socket had
the 
event and you have to parse the FD_SET anyways.
Hope this helps.

With all the frenzy about poll()/select() style of IO, please do not forget
that Java also really really needs NON BLOCKING WRITE() calls.

There's a new JSR which proposes to define
a new set of I/O libraries for the Java platform.

See 
http://java.sun.com/aboutJava/communityprocess/jsr/jsr_051_ioapis.html

-Sean

How about setting up a project to implement all of this as native methods using
JNI, with a uniform API across platforms? Has anyone tried this?

I wonder if the spec will have one callback thread (as in JavaSpaces) or 
a thread pool (as in RMI).  I'm hoping they will create a thread pool and
give the user some control over it.  Hopefully via a factory.

Gee, and I was told "will be fixed in 1.3" -- but it's not.  Sigh.  You'd really think that Sun would be 
interested in having "100% Pure Java" _Enterprise Capable_ solutions.  I happen to know that this has 
affected a number of Sun-internal developers (and projects); gotta wonder why the JVM people aren't 
bothering to fix such stuff.   I hope it's not that old "we can make threads faster" religion.  While it may be 
true that the cost of a thread can be reduced, it's NOT true that it can be cut to zero like a select/poll style 
API can.

Both Sun and HP now support using poll() to some extent to
get around this one-thread-per-client requirement.

For HP, see
http://www.devresource.hp.com/JavaATC/JavaPerfTune/pollapi.html

For Sun, the Solaris J2SE 1.2.2 production release includes
in its example sources a demo of how to use JNI to call
poll.

I'm looking forward to JSR-051 getting started...

And again. Though available() plus read(upTpAvail) should work for reading (however MS VJM, InputStream 
from URL blocks(!) on available) there is nothing about writing. Such API should be implemented, also 
InterrupteIOException has enough overhead.

Dear tswain:
When you are using select() you most certainly can work out
which channel had the event, since the FD sets that you get
passed in indicate which descriptors had the events on.  OK,
so this does make the select() syscall one of the most
complex to use correctly, but the info is there, and many
systems use it happily to implement async I/O in
single-threaded processes.

It is really impossible to develop good TCP/IP server 
without select() or poll(). I completely agree!
~Alex Cyn

Mark (the speaker!) didn't show up for this JavaOne BOF (I
left after half an hour).  Is this project still on track?

Some OSes implement a /dev/poll device where file descriptors which are ready
(data write completed, data available, error) stream out when reading. This might
be the most convenient an scaleable solution. Now add a method "setEventHandler()" to
every (Socket|File|*)(In|Out)put(Stream|Reader) which is called when the corresponding
file descriptor is ready. THIS would be scaleable.

No, you use a JavaBeans/JDK-like event observation mechanism
(perhaps a FileEvent class and a add/removeFileListener()
method in Reader, Writer, InputStream, and OutputStream?) 
We have the pattern already in use in the language, so
there's no use in introducing a new one...

This becomes an even bigger problem when dealing with Unix 
interprocess pipes.  The write operations will block if 
there is no one listening on the other side of the pipe.  
If you are writing a server that uses interprocess pipe to 
comunicate to non-java process (one way to integrate to 
legacy systems) the server will essentally hang if the 
other process dies and is unable to recieve a response.

I have used two extremely poor workaround for this 
problem.  On is to place the write operation into a 
seperate thread that is then interupted by the main thread 
after a timeout period has expired.  This of course leads 
to a whole chorus of exceptions that must be caught in both 
the writer thread and the main thread.

The other is two use 2 pipe for every comunication.  One 
pipe is a signal pipe that must be opened to unblock the 
the io open (object contruction) of the writer.  The non-
java application must open the signal pipe after it opens 
the responce pipe to insure that it is listening.  This 
insures that a java write operation will have a process 
listening on the other side of the pipe 95% of time.  It is 
possble for the non-java process to open the signal pipe 
then die.  This leads us back to were we started.

It is extremely important the non-blocking io introduced.  
Especial after the Java Connector API JSR16 give us an 
intetgration path to legacy system that my include 
interprocess pipes.

This thing is nearly 3 years old... what's the deal?

This is pathetic, over 3 years old!

And Sun wonder why we're clamouring for Java to go open 
source! To get it out of the arthritic grip of senile
Sun.

3 years?!?!!?! Imagine going to some other vendor,
and getting a 3 year response time on a bug fix, which 
is what this is. The bug was that Sun imagined that 
Java=pissy little applets. The current IO model simply
doesn't scale to an multi enterprise environment.

well.. the promise was to include aysnc IO with 1.3.  and 
now thats 1.3 is out and has no Async I/O what version do 
we wait for?  this sux big time

Guys,
while Sun is working at this 
(http://java.sun.com/aboutJava/communityprocess/jsr/jsr_051_
ioapis.html), a partial solution could be 
http://www.jrockit.com, a JVM that uses a MxN java-to-
native thread mapping and that promises not to block 
threads on I/O.
I found their whitepaper quite interesting.

Regards

Please vote to fix this bug.  This has been langushing for 
too long.  Sun needs to submit Java to a standards 
organization so things like this get fixed faster.

Shame on you, Sun. This bug is over 3 years old!!!
And I thought that Microsoft are not responsive...

It is obviously the worst nightmire that I ever have since 
I developed my first server-side java software two years 
ago. Sun, it is alreay 3 years old. Shouldn't you fix it on 
next version?! Are you listening?!

Again... what's up with this?

Many years ago I started a project and had to switch to C++
because of this problem.  I have specifically NOT been using
Java for the last 3 years because of this limitation and
Sun's lack of support.  I've recommended against using Java
on every project I've worked on since, and they listened. 
This includes multi-billion dollar government contracts
(some of my original projects).

This request is nearly at the top of the list, as it has
been ever since I can remember.  What gives?

I'm not 100% sure I'm at the right place, but this has gotten my 3 votes regardless.  See I'm from the 
FileNotFoundException <- getInputStream() community, where you have trouble getting just one 
connection.
What helped me was using HTTPClient in place of the java.net.HttpURLConnection class.  Get it at,
http://www.innovation.ch/java/HTTPClient/
Maybe it'll help in some of the other cases here?
For my *much* longer post see "Applets+Plug-in+Proxy=HTTP connect fails" in the Java Plug-in forum at the 
JDC

This MUST be fixed, who the heck has this been assigned to?

Please do NOT implement a select style non blocking io. NT 
style IOCompletion ports are a much better solution. Please 
fix this.

Please re-evaluate this issue. It does not involve any API changes. 
It only begs for implementing interruptable I/O semantics with existing read/write calls.

IMPORTANT NOTE: Sun's own API included in JVM 1.2+ explicitly relies on I/O in java being interruptable 
(see javax.swing.ProgressMonitorInputStream for example).

Interruptable I/O semantics would effectively solve all I/O problems mentioned in comments.

For example, one "watchdog" thread in application would be able to monitor 
any number of I/O operations running in separate threads if they select to register themselves 
(anonymously, only as Threads)
in the watchdog with simple timeouts or more sophisticated schedules. 

Its also not quite true that "This requires significant API
changes.  It's too late to do this in JDK 1.2" ...
API mods have been suggested since at least 1995
on the networking mailing lists and java user groups.

This is a glaring flaw in the class library,
stemming from Sun's continued obstinacy and
insistence that "threads are almost free" - unaddressed
since the first Beta API of Java, which makes Java Servers
with thousands of concurrent connections, almost impossible
to write.

I've been reading around the subject a bit more (looking at
how other languages handle this in particular) and it seems
that the problem with API changes stems at least in part
from how different the various platforms are on this matter
(Windows is a complete nightmare, particularly when you want
to support everything from NT3.5 and 95 up to 2K, plus
whatever bizarre crud comes out of Redmond in the future,
and this is all due to their not having a properly unified
channel communications API.)  Still, the best way I see for
handling this is to have a way of registering callbacks with
readers and writers so code can receive notifications of
when data can be read or written without blocking (there
probably ought to be an error-condition callback as well,
that's a useful one, but you can in practise often overload
that on top of the other two.)  This technique is used
extensively in scripting languages (notably Tcl) where it
leads to very small high-capacity servers.

YAAAA HOOO!!  Thank God!   I hope it really is fixed!!!

-G

Hmmm... it seems partially fixed.  I don't see any
asynchronous I/O in the JDK 1.4 beta1, but there is
polling.

NOBLOCKING MECHANISM IS ABSOLUTE NEED!!

has no comment

PING

hey