Java Solaris Communities Sun Store Join SDN My Profile Why Join?
 
Bug Database
Bug Detail
Quick Lists
Top 25 Bugs
Top 25 RFE's
Recently Closed Bugs
Printable Page Printable Page


Bug Database
Bug ID: 6612102
Votes 1
Synopsis (coll) IdentityHashMap.iterator().remove() might decrement size twice
Category java:classes_util
Reported Against
Release Fixed 7(b25)
State 10-Fix Delivered, bug
Priority: 3-Medium
Related Bugs
Submit Date 02-OCT-2007
Description
J2SE Version (please include all output from java -version flag):

java version "1.6.0"
Java(TM) SE Runtime Environment (build 1.6.0-b105)
Java HotSpot(TM) Client VM (build 1.6.0-b105, mixed mode)

Does this problem occur on J2SE 5.0.x or 6.0?  Yes / No (pick one)
Yes - reproduced on 1.5.0_12 and 6.0.

Operating System Configuration Information (be specific):
Any but tested on:  customer  Windows XP [Version 5.1.2600]

Bug Description:

There is a bug in java.util.IdentityHashMap where it can run into an
infinite loop when calling the put() method. This happens when also using
an iterator() on the same  customer  and calling remove selectively on some
elements. The problem appears to be that the map gets confused about the
free space in the map and eventually tries to find a slot for an item even
though there is no free space. Since the IHM uses linear probing rather
than chaining, it runs round the underlying array looking for a free slot
but never finds one.

I confess I have not examined the IdentityHashMapIterator.remove() method
in detail myself to identity the bug since it is quite involved.

I have however developed a test case that keeps hammering the IHM until it
runs into the bug. It then outputs the set of operations that caused the
problem. Someone could then write a program that just applied these
operations to help nail the bug.

The test case I have got is artifical in the sense that it uses only a
single thread and some random probabilities to find the bug. The albeit
simplified scenario where someone might run into this is like this:

private IdentityHashMap _myMap = new IdentityHashMap();

public synchronized void addItem(ObjectWithState o) { _mapMap.put(o, o); //
note using the IHM as a set hence same key and value but not significant}

public synchronized void removeSomeIterms()
{
     Iterator it = idMap.keySet().iterator();
     while (it.hasNext())
     {
         ObjectWithState s = (ObjectWithState) it.next();

         if (s.isDead())
         {
              it.remove();
         }
    }
}

If you run the above there are certain scenarios where the call to addItem
will just go into an infinite loop.

I have attached a class that illustrates the problem. Just run it and it
detects the infinite loop and prints out the series of operations that
caused the infinite loop. If any part of the code is unclear get back to
me.

Workaroud

The only workaround I could see is "don't use IdentityHashMap if you want
to use an iterator like this". We found this in a third party library but
we can get that changed.

(See attached file: TestIdentityHashMap.java)
Posted Date : 2007-10-02 18:08:01.0
Work Around
N/A
Evaluation
iterator().remove() is sometimes causing size to be decremented twice,
leading to corrupted internal data structures.
Posted Date : 2007-10-03 08:24:48.0

Here's a simpler variant of the test case:

import java.util.*;

public class Bug6 {
    static void remove(Map m, Iterator it) {
	int size = m.size();
	it.remove();
	if (m.size() != size-1)
	    throw new Error(String.format("Incorrect size!%nmap=%s, size=%d%n",
					  m.toString(), m.size()));
    }

    public static void main(String[] args) {
        while (true) {
	    final Map<Object, Object> m
		= new IdentityHashMap<Object, Object>(16);
	    final Random r = new Random();
	    for (int i = 0; i < 10; i++)
		m.put(r.nextInt(100), null);
	    Iterator it = m.keySet().iterator();
	    int size = m.size();
	    for (int j = 0; j < size - 2; j++)
		it.next();
	    remove(m, it);
	    it.next();
	    remove(m, it);
	}
    }
}
Posted Date : 2007-10-03 19:06:34.0

Here's the obvious fix:

diff --git a/src/share/classes/java/util/IdentityHashMap.java b/src/share/classes/java/util/IdentityHashMap.java
--- a/src/share/classes/java/util/IdentityHashMap.java
+++ b/src/share/classes/java/util/IdentityHashMap.java
@@ -1,5 +1,5 @@
 /*
- * Copyright 2000-2006 Sun Microsystems, Inc.  All Rights Reserved.
+ * Copyright 2000-2008 Sun Microsystems, Inc.  All Rights Reserved.
  * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
  *
  * This code is free software; you can redistribute it and/or modify it
@@ -749,7 +749,6 @@ public class IdentityHashMap<K,V>
             expectedModCount = ++modCount;
             int deletedSlot = lastReturnedIndex;
             lastReturnedIndex = -1;
-            size--;
             // back up index to revisit new contents after deletion
             index = deletedSlot;
             indexValid = false;
@@ -781,6 +780,8 @@ public class IdentityHashMap<K,V>
                 expectedModCount = modCount;
                 return;
             }
+
+            size--;
 
             Object item;
             for (int i = nextKeyIndex(d, len); (item = tab[i]) != null;
diff --git a/test/java/util/Map/LockStep.java b/test/java/util/Map/LockStep.java
new file mode 100644
--- /dev/null
+++ b/test/java/util/Map/LockStep.java
@@ -0,0 +1,127 @@
+/*
+ * Copyright 2008 Sun Microsystems, Inc.  All Rights Reserved.
+ * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
+ *
+ * This code is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 2 only, as
+ * published by the Free Software Foundation.
+ *
+ * This code is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
+ * version 2 for more details (a copy is included in the LICENSE file that
+ * accompanied this code).
+ *
+ * You should have received a copy of the GNU General Public License version
+ * 2 along with this work; if not, write to the Free Software Foundation,
+ * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
+ *
+ * Please contact Sun Microsystems, Inc., 4150 Network Circle, Santa Clara,
+ * CA 95054 USA or visit www.sun.com if you need additional information or
+ * have any questions.
+ */
+
+/*
+ * @test
+ * @bug 6612102
+ * @summary Test Map implementations for mutual compatibility
+ */
+
+import java.util.*;
+import java.util.concurrent.*;
+
+/**
+ * Based on the strange scenario required to reproduce
+ * (coll) IdentityHashMap.iterator().remove() might decrement size twice
+ *
+ * It would be good to add more "Lockstep-style" tests to this file.
+ */
+public class LockStep {
+    void mapsEqual(Map m1, Map m2) {
+        equal(m1, m2);
+        equal(m2, m1);
+        equal(m1.size(), m2.size());
+        equal(m1.isEmpty(), m2.isEmpty());
+        equal(m1.keySet(), m2.keySet());
+        equal(m2.keySet(), m1.keySet());
+    }
+
+    void mapsEqual(List<Map> maps) {
+        Map first = maps.get(0);
+        for (Map map : maps)
+            mapsEqual(first, map);
+    }
+
+    void put(List<Map> maps, Object key, Object val) {
+        for (Map map : maps)
+            map.put(key, val);
+        mapsEqual(maps);
+    }
+
+    void removeLastTwo(List<Map> maps) {
+        Map first = maps.get(0);
+        int size = first.size();
+        Iterator fit = first.keySet().iterator();
+        for (int j = 0; j < size - 2; j++)
+            fit.next();
+        Object x1 = fit.next();
+        Object x2 = fit.next();
+
+        for (Map map : maps) {
+            Iterator it = map.keySet().iterator();
+            while (it.hasNext()) {
+                Object x = it.next();
+                if (x == x1 || x == x2)
+                    it.remove();
+            }
+        }
+        mapsEqual(maps);
+    }
+
+    void remove(Map m, Iterator it) {
+        int size = m.size();
+        it.remove();
+        if (m.size() != size-1)
+            throw new Error(String.format("Incorrect size!%nmap=%s, size=%d%n",
+                                          m.toString(), m.size()));
+    }
+
+    void test(String[] args) throws Throwable {
+        final int iterations = 100;
+        final Random r = new Random();
+
+        for (int i = 0; i < iterations; i++) {
+            List<Map> maps = Arrays.asList(
+                new Map[] {
+                    new IdentityHashMap(11),
+                    new HashMap(16),
+                    new LinkedHashMap(16),
+                    new WeakHashMap(16),
+                    new Hashtable(16),
+                    new TreeMap(),
+                    new ConcurrentHashMap(16),
+                    new ConcurrentSkipListMap() });
+
+            for (int j = 0; j < 10; j++)
+                put(maps, r.nextInt(100), r.nextInt(100));
+            removeLastTwo(maps);
+        }
+    }
+
+    //--------------------- Infrastructure ---------------------------
+    volatile int passed = 0, failed = 0;
+    void pass() {passed++;}
+    void fail() {failed++; Thread.dumpStack();}
+    void fail(String msg) {System.err.println(msg); fail();}
+    void unexpected(Throwable t) {failed++; t.printStackTrace();}
+    void check(boolean cond) {if (cond) pass(); else fail();}
+    void equal(Object x, Object y) {
+        if (x == null ? y == null : x.equals(y)) pass();
+        else fail(x + " not equal to " + y);}
+    public static void main(String[] args) throws Throwable {
+        new LockStep().instanceMain(args);}
+    void instanceMain(String[] args) throws Throwable {
+        try {test(args);} catch (Throwable t) {unexpected(t);}
+        System.out.printf("%nPassed = %d, failed = %d%n%n", passed, failed);
+        if (failed > 0) throw new AssertionError("Some tests failed");}
+}
Posted Date : 2008-03-06 23:45:59.0
Comments
  
  Include a link with my name & email   


PLEASE NOTE: JDK6 is formerly known as Project Mustang