JSObject.eval() do not return when JDialog is opened.
Java version: jdk 6u12 build02 or later with new generation plugin enabled.
browser: Firefox 3.0.6, IE6, IE7 (not reproducible on Firefox 2.0)

To reproduce:
1. open attached TestApplet.html file in a web browser(it will run attached TestApplet.java).
2. click "run javascript button" - it will open JDialog and start a thread which opens
javascript alert window.

Expected results:
clicking OK button on javascript alert window should close
the JDialog windows and javascript alert window itself.

Actual results:
on jdk 6u12b01 or earlier (with new plugin enabled) (i.e. before a fix for 6764266)
javascript alert window freezes.

on jdk 6u12b02 or later (with new plugin enabled) (i.e. after a fix for 6764266)
javascript alert window closes but JSObject.eval() never returns and as a result
JDialog.setVisible(false) doesn't get called and JDialog window doesn't close.

Test case passes if an old generation plugin is enabled.

Attached is also original Oracle test case - OracleTestApplet.
After clicking OK on javascript aler window the whole browser
gets frozen. This is obviously because invisible (undecorated) modal JDialog
is still active.
Below is an explanation from Oracle why they need Javascript alert window
running in a separate thread:
"We had introduced this mechanism of setting a invisible JDialog to true
in our Forms code to fix another complex issue - bug 6802701. The problem
there was that when we execute javascript alerts in the Event thread, we
block the Forms client applet. If the applet is blocked beyond a certain
time, then the Forms server thinks that the client is already dead and thus
exits. To resolve this issue, we introduced the mechanism of running the
javascript in a separate thread and keep the forms client alive by running an
invisible JDialog. The advantage that we have here is that the state of Forms
would remain at the same point where the Javascript alert is brought up, but
at the same time keep the Forms client alive."
Posted Date : 2009-03-04 19:47:32.0

.

modality support block the javascript call because they all run on the browser main thread. If you click away the alert js dialog, js call can not return because there is modality calls on the stack.

You can change the code to make the modality dialog show up first. Then the js call will be on the top of the stack. Js call will return after the alert dialog is clicked. Then the modal dialog can be hidden.

In the attached TestApplet code, you can wait until the modal dialog is visible in the run() method before JSObject.getWindow() to call javascript. Then the js call will be pushed on top of the modality support on the main thread.
Posted Date : 2009-02-27 17:52:48.0

More deatils from Hao:
"In the new plugin, when a modal dialog is showing, awt notify plugin by calling registered modality listener.  Plugin send a message from attached jvm to the browser side vm to block the browser main thread. While blocking the main thread, we still pump windows messages so a javascript invocation is possible.

When java invokes javascript, we do similar thing. A message sent to the browser main thread to invoke javascript. When the alert box is poped up, windows is pumping messages so a modality block can still happen on the browser main thread.

The testcase does modal dialog and javascript calls from two threads. They race with each other. On the browser main thread, the two can happen in either order. If javascript invocatio is the first, then the modality code will be on top it. Javascript call can not return unless the modality is popped (i.e. modal dialog is clicked).

I guess the customer want the javascript call can return first. To do this, they must ensure the javascript call happens after the modality push. There is no APIs to tell if the modality has pushed on the browser main thread. The closest thing is to check if a dialog is showing or not. When Dialog.setVisible() is called, awt calls plugin to push the modality. It takes time for plugin to send a message to browser side. If you start javascript immediately after the dialog is showing, they are still racing with each other. So let the javascript thead sleep 200 ms is a good heuristic."
Posted Date : 2009-03-04 19:09:20.0