EVALUATION

http://hg.openjdk.java.net/jdk7/hotspot/hotspot/rev/c1a0ede55d6f
                                     

EVALUATION

http://hg.openjdk.java.net/jdk7/hotspot-rt/hotspot/rev/c1a0ede55d6f
                                     

EVALUATION

Yes, we're creating the VM and this is where we are in
src/share/vm/runtime/thread.cpp:

   3102   HandleMark hm;
   3103 
   3104   { MutexLocker mu(Threads_lock);
   3105     Threads::add(main_thread); 
   3106   }
   3107 
   3108   // Any JVMTI raw monitors entered in onload will transition into
   3109   // real raw monitor. VM is setup enough here for raw monitor enter.
   3110   JvmtiExport::transition_pending_onload_raw_monitors();
   3111 
   3112   if (VerifyBeforeGC &&
   3113       Universe::heap()->total_collections() >= VerifyGCStartAt) {
   3114     Universe::heap()->prepare_for_verify();
HERE => 3115     Universe::verify();   // make sure we're starting with a clean slate
   3116   }
   3117 
   3118   // Create the VMThread
   3119   { TraceTime timer("Start VMThread", TraceStartupTime);
   3120     VMThread::create();
   3121     Thread* vmthread = VMThread::vm_thread();

We're calling Universe::verify() on line 3115. Notice that happens
after we created the main thread on line 3105 and before we created
the VMThread on line 3120.

Can't check for a safepoint because we can't have a safepoint
without the VMThread. Threads::number_of_threads() == 0 returns
false because we just created the main thread. We're in a "no mans"
land in the VM bringup world.

I did a quick survey of "oops do" functions in the runtime and
prims directories and only found one that asserted being called at
a safepoint. It seems like there is a general assumption that
"oops do" is called from the "right place".

JvmtiTagMap::weak_oops_do() was originally JvmtiTagMap::gc_epilogue()
and that's where the safepoint assertion came from. In my experience
epilogue functions (and prologue functions) often assert that they
are called from a safepoint.

It seems that I can change the assertion to include:

    Threads::number_of_threads() <= 1

or I can delete the safepoint assertion and go with how most of the
"oops do" functions are written.
                                     

SUGGESTED FIX

Context diffs for the proposed fix:

 $ hg diff src/share/vm/prims/jvmtiTagMap.cpp
diff -r 34d64ad817f4 src/share/vm/prims/jvmtiTagMap.cpp
--- a/src/share/vm/prims/jvmtiTagMap.cpp        Wed Jan 12 13:59:18 2011 -0800
+++ b/src/share/vm/prims/jvmtiTagMap.cpp        Fri Jan 14 18:06:31 2011 -0700
@@ -3290,7 +3290,11 @@ void JvmtiTagMap::follow_references(jint
 
 
 void JvmtiTagMap::weak_oops_do(BoolObjectClosure* is_alive, OopClosure* f) {
-  assert(SafepointSynchronize::is_at_safepoint(),
+  // No locks during VM bring-up (0 threads) and no safepoints after main
+  // thread creation and before VMThread creation (1 thread); initial GC
+  // verification can happen in that window which gets to here.
+  assert(Threads::number_of_threads() <= 1 ||
+         SafepointSynchronize::is_at_safepoint(),
          "must be executed at a safepoint");
   if (JvmtiEnv::environments_might_exist()) {
     JvmtiEnvIterator it;
                                     

EVALUATION

The fix for the following bug appears to be the cause of this
assertion failure:

    6458402 4/3 3 jvmti tests fail with CMS and +ExplicitGCInvokesConcurrent

Looks like the fix for 6458402 changed the code path on
which the assertion is checked. JNIHandleBlock::weak_oops_do()
was modified to add a call to JvmtiTagMap::weak_oops_do(is_alive, f)
which checks this assertion.

The following stack snippet:

V  [libjvm.so+0x1398edd]  void JvmtiTagMap::weak_oops_do(BoolObjectClosure*,OopClosure*)+0x69d;;  void JvmtiTagMap::weak_oops_do(BoolObjectClosure*,OopClosure*)+0x69d
V  [libjvm.so+0x10c590a]  void JNIHandles::verify()+0x346;;  void JNIHandles::verify()+0x346
V  [libjvm.so+0x1c10a77]  void Universe::verify(bool,bool,bool)+0x29f;;  void Universe::verify(bool,bool,bool)+0x29f
V  [libjvm.so+0x1b9d09a]  int Threads::create_vm(JavaVMInitArgs*,bool*)+0x2aa;;  int Threads::create_vm(JavaVMInitArgs*,bool*)+0x2aa
V  [libjvm.so+0x100aeaf]  JNI_CreateJavaVM+0x77;;  JNI_CreateJavaVM+0x77

shows that JNIHandles::verify() can be called very early in VM
startup which results in a call to JNIHandleBlock::weak_oops_do()
which results in a call to JvmtiTagMap::weak_oops_do().
                                     

EVALUATION

The JVM/TI implementation often has to live in the early VM
bringup world so there's an easy fix for this problem that is
used in other parts of JVM/TI. The assertion needs to be
modified like this:

  // no locks during VM bring-up
  assert(Threads::number_of_threads() == 0 ||
         SafepointSynchronize::is_at_safepoint(),
         "must be executed at a safepoint");
                                     

WORK AROUND

Use the following option to supress this assertion failure:

    -XX:SuppressErrorAt=/jvmtiTagMap.cpp:3294