This behavior is caused by the fact that JTextPane (and JEditorPane) don't honor composed text in their replaceSelection() methods that they override from JTextComponent. The JTextComponent itself properly handles composed text, this is why, for example, this bug doesn't show in JTextAred (which doesn't override replaceSelection().)
Unfortunately JTextComponent doesn't expose its methods saveComposedText() and restoreComposedText() to subclasses, they are private.
The "right" way is to make these methods protected and thus available to subclasses. Unfortunately, this isn't possible in the update releases. This means that subclasses should either re-implement all the composed text machinery, which is pretty complex, or find some ways to access these private methods.
The obvious solution is to use reflection, but this is only a partial solution and won't work under security manager.
Another solution is to register some publicly accessible methods that "call back" into JTextComponent and invoke the corresponding private methods. I decided to use this way.
The publicly accessible methods live in SwingUtilities2, and JTextComponent registers a couple of closures there that just call back and invoke the required methods. JTextPane and JEditorPane then are able to use the methods from SwingUtilities2.