Tests for deadlocks are unstable. OK not to have one.
                                     

Difficult to verify
                                     

Approved for 7u10b11
                                     

SQE is OK with this fix.
                                     

Webrev: http://cr.openjdk.java.net/~anthony/7u10-1-getMaxTextureSizeDeadlock-7200762.0/
Review: http://mail.openjdk.java.net/pipermail/awt-dev/2012-September/003601.html
Approved by: Artem Ananiev and Andrew Brygin
                                     

SQE comments please
                                     

EVALUATION

A solution for 7u10 is to cache the value of getMaxTextureSize() sometime when the LWCToolkit is initialized, and use this value all the time. Note that the top-level window size is further constrained with the current display size which is less than the MaxTextureSize in 99% of cases. Meaning, the 7160609 will regress in those 1% of cases. We find this risk acceptable.

For JDK 8 a solution is to try and eliminate acquiring the TreeLock on the AppKit thread. It is a complex solution that requires more testing and isn't suitable for 7u10 at the moment.
                                     

SUGGESTED FIX

http://cr.openjdk.java.net/~anthony/7u10-1-getMaxTextureSizeDeadlock-7200762.0/
                                     

EVALUATION

Looks like AWT to me. Nothing in 2D code is locking the tree.

The one thing to check is why getMaxTextureSize() we hang although
I find this was just added in 7u10 b06 as
7160609 : [macosx] JDK crash in libjvm.dylib ( C  [GeForceGLDriver+0x675a]  gldAttachDrawable+0x941)

------



"AWT-EventQueue-0" prio=5 tid=0x00007fdcc12be800 nid=0xa603 runnable [0x000000015a88d000]
   java.lang.Thread.State: RUNNABLE
	at sun.java2d.opengl.CGLGraphicsConfig.getMaxTextureSize(Native Method)
	at sun.java2d.opengl.CGLGraphicsConfig.getMaxTextureHeight(CGLGraphicsConfig.java:532)
	at sun.lwawt.LWWindowPeer.setBounds(LWWindowPeer.java:360)
	at java.awt.Component.reshapeNativePeer(Component.java:2330)
	at java.awt.Component.reshape(Component.java:2278)
	- locked <0x000000011f8006c8> (a java.awt.Component$AWTTreeLock)
	at java.awt.Window.reshape(Window.java:952)
	at java.awt.Component.setBounds(Component.java:2243)
	at java.awt.Window.setBounds(Window.java:3472)
	- locked <0x000000011f8006c8> (a java.awt.Component$AWTTreeLock)
	at java.awt.Component.move(Component.java:2091)
	- locked <0x000000011f8006c8> (a java.awt.Component$AWTTreeLock)
	at java.awt.Component.setLocation(Component.java:2080)
	at java.awt.Window.setLocation(Window.java:921)


--------------



"AppKit Thread" daemon prio=5 tid=0x00007fdcc1192800 nid=0x707 waiting for monitor entry [0x00007fff5c3aa000]
   java.lang.Thread.State: BLOCKED (on object monitor)
	at java.awt.Component.getLocationOnScreen(Component.java:2018)
	- waiting to lock <0x000000011f8006c8> (a java.awt.Component$AWTTreeLock)
	at javax.swing.SwingUtilities.convertPointFromScreen(SwingUtilities.java:422)
	at sun.lwawt.macosx.CDragSourceContextPeer.dragMouseMoved(CDragSourceContextPeer.java:439)
                                     

EVALUATION

A deadlock between the AppKit Thread and EDT.