EVALUATION

http://hg.openjdk.java.net/jdk7/hotspot-comp/hotspot/rev/7d9caaedafce
                                     

SUGGESTED FIX

Don't use has_method_handle_invokes but check if CodeOffsets::DeoptMH is set:

-    if (has_method_handle_invokes()) {
+    if (offsets->value(CodeOffsets::DeoptMH) != -1) {
                                     

EVALUATION

The bug is that the nmethod constructor uses nmethod::has_method_handle_invokes but the _has_method_handle_invokes flag is not initialized until nmethod::copy_scopes_pcs is called and thus always returns false.  This results in _deoptimize_mh_offset to be -1 and so deopt_mh_handler_begin points 1 byte before the nmethod pointer resulting in the assert.
                                     

EVALUATION

I can reproduce the assert doing:

$ gamma -XX:+ShowMessageBoxOnError -Xcomp -XX:+DeoptimizeALot -XX:+UnlockExperimentalVMOptions -XX:+EnableMethodHandles -XX:+EnableInvokeDynamic -XX:-UseCompressedOops -cp /java/devtools/share/junit/latest/junit.jar:. org.junit.runner.JUnitCore test.java.dyn.MethodHandlesTest
VM option '+ShowMessageBoxOnError'
VM option '+DeoptimizeALot'
VM option '+UnlockExperimentalVMOptions'
VM option '+EnableMethodHandles'
VM option '+EnableInvokeDynamic'
VM option '-UseCompressedOops'
JUnit version 4.4
.IIIIII.# To suppress the following error report, specify this argument
# after -XX: or in .hotspotrc:  SuppressErrorAt=/frame_x86.cpp:364
==============================================================================
Unexpected Error
------------------------------------------------------------------------------
Internal Error at frame_x86.cpp:364, pid=8661, tid=15
assert(sender_cb) failed: sanity

Do you want to debug the problem?

To debug, run 'dbx - 8661'; then switch to thread 15
Enter 'yes' to launch dbx automatically (PATH must include dbx)
Otherwise, press RETURN to abort...
==============================================================================

(dbx) where
current thread: t@15
  [1] __waitid(0x0, 0x21d6, 0xfffffd7ffac86640, 0x3, 0xfffffd7fff2fdfc4, 0xfffffd7ffefc6a00), at 0xfffffd7fff26461a 
  [2] waitid(0x0, 0x0, 0x0, 0x0, 0x0, 0x0), at 0xfffffd7fff24d35b 
  [3] waitpid(0x0, 0x0, 0x0, 0x0, 0x0, 0x0), at 0xfffffd7fff20ef94 
=>[4] os::fork_and_exec(cmd = 0xfffffd7fa3d03c74 "dbx - 8661"), line 5978 in "os_solaris.cpp"
  [5] VMError::show_message_box(this = 0xfffffd7ffac869d0, buf = 0xfffffd7fa3d03c74 "dbx - 8661", buflen = 2000), line 53 in "vmError_solaris.cpp"
  [6] VMError::report_and_die(this = 0xfffffd7ffac869d0), line 725 in "vmError.cpp"
  [7] report_vm_error(file = 0xfffffd7fa37e901d "/home/ct232829/hotspot-comp/hotspot/src/cpu/x86/vm/frame_x86.cpp", line = 364, error_msg = 0xfffffd7fa37e905e "assert(sender_cb) failed", detail_msg = 0xfffffd7fa37e9077 "sanity"), line 176 in "debug.cpp"
  [8] frame::sender_for_interpreter_frame(this = 0xfffffd7ffac86c88, map = 0xfffffd7ffac86cb8), line 364 in "frame_x86.cpp"
  [9] frame::sender(this = 0xfffffd7ffac86c88, map = 0xfffffd7ffac86cb8), line 487 in "frame_x86.cpp"
  [10] frame::deoptimize(this = 0xfffffd7ffac87350, thread = 0x447800), line 293 in "frame.cpp"
  [11] Deoptimization::deoptimize_single_frame(thread = 0x447800, fr = CLASS), line 1048 in "deoptimization.cpp"
  [12] Deoptimization::deoptimize(thread = 0x447800, fr = CLASS, map = 0xfffffd7ffac874c0), line 1063 in "deoptimization.cpp"
  [13] JavaThread::deoptimize(this = 0x447800), line 2319 in "thread.cpp"
  [14] VM_DeoptimizeAll::doit(this = 0xfffffd7fffdfcf78), line 115 in "vm_operations.cpp"
  [15] VM_Operation::evaluate(this = 0xfffffd7fffdfcf78), line 46 in "vm_operations.cpp"
  [16] VMThread::evaluate_operation(this = 0x13b8000, op = 0xfffffd7fffdfcf78), line 338 in "vmThread.cpp"
  [17] VMThread::loop(this = 0x13b8000), line 444 in "vmThread.cpp"
  [18] VMThread::run(this = 0x13b8000), line 251 in "vmThread.cpp"
  [19] java_start(thread_addr = 0x13b8000), line 1010 in "os_solaris.cpp"
  [20] _thrp_setup(0x0, 0x0, 0x0, 0x0, 0x0, 0x0), at 0xfffffd7fff25acf5 
  [21] _lwp_start(0x0, 0x0, 0x0, 0x0, 0x0, 0x0), at 0xfffffd7fff25afb0 
(dbx) fr 8
Current function is frame::sender_for_interpreter_frame
  364     assert(sender_cb, "sanity");
(dbx) p *this
*this = {
    _sp            = 0xfffffd7fffdfd360
    _pc            = 0xfffffd7ffb005dd4 "H\x8be??H??E??"
    _cb            = 0xfffffd7ffb004dd0
    _deopt_state   = not_deoptimized
    _fp            = 0xfffffd7fffdfd3d0
    _unextended_sp = 0xfffffd7fffdfd360
    _check_value   = class frame::CheckValueClosure /* STATIC CLASS */
    _check_oop     = class frame::CheckOopClosure /* STATIC CLASS */
    _zap_dead      = class frame::ZapDeadClosure /* STATIC CLASS */
}
(dbx) p findpc((intptr_t) sender_pc)

"Executing findpc"
0xfffffd7ffb370acf is pointing to unknown location
findpc((intptr_t ) sender_pc) = (void)
                                     

EVALUATION

Disabling DeoptimizeALot results in a different crash.  Not sure yet if these are related.

$ gamma -Xcomp -XX:+UnlockExperimentalVMOptions -XX:+EnableMethodHandles -XX:+EnableInvokeDynamic -XX:-UseCompressedOops -cp /java/devtools/share/junit/latest/junit.jar:. org.junit.runner.JUnitCore test.java.dyn.MethodHandlesTest
VM option '+UnlockExperimentalVMOptions'
VM option '+EnableMethodHandles'
VM option '+EnableInvokeDynamic'
VM option '-UseCompressedOops'
JUnit version 4.4
.IIIIII...........................#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0xfffffd7ffb11142c, pid=8672, tid=1
#
# JRE version: 7.0
# Java VM: OpenJDK 64-Bit Server VM (20.0-b01-internal-jvmg compiled mode solaris-amd64 )
# Problematic frame:
# J  java.util.concurrent.ConcurrentHashMap$Segment.put(Ljava/lang/Object;ILjava/lang/Object;Z)Ljava/lang/Object;
#
# An error report file with more information is saved as:
# /home/ct232829/jdk7/jdk7/jdk/hs_err_pid8672.log
#
# If you would like to submit a bug report, please visit:
#   http://java.sun.com/webapps/bugreport/crash.jsp
#
Current thread is 1
Dumping core ...