A DESCRIPTION OF THE REQUEST :
This proposal is a mechanism to allow API’s to be written that contain Java keywords that can still be called from Java, e.g. an API not written in Java. In addition new keywords can be added to Java without braking old code. The proposal is to name mangle a name that is a keyword if it is imported into the name space of a compilation unit (file) by either a "package" or "import" statement.

The idea is simple:

1. Add a new keyword “source” that gives the Java source version, e.g. “source 6;” would mean this file is written in Java 6.

2. The “source” keyword goes before the “package” statement and therefore  doesn’t break old code, since no statement could previously have been at this position in the file.

3. Any “package” or “import” statements that import a name into the files name space gets name mangled by putting an underscore either side, e.g. “assert” would become “_assert_”.

4. If no source statement is given in a file then the source version is taken from the source switch on the command line, as it is at present. But note that name mangling still occurs and a source statement overrides a source switch.

5. The name mangling is two way, so that an API can be written in Java that uses a mangled keyword as a name that appears in the compiled class file but in the class file  the non-mangled name is used. Hence the JVM doesn’t see mangled names.

So for example suppose an API, from either an early version of Java or from some other language contained the name “assert” and this API was to be used in a Java 6 program, then:

  source 6;

  package …;

  import …; // imports something that contains "assert", e.g. an early JUnit API

    …

    _assert_( … );

JUSTIFICATION :
With increasing use of languages other than Java on the JVM it is inevitable that some Java keywords will be used by APIs written in these non-Java languages and therefore these APIs will be unusable from Java, e.g. an API with a method called “final”. Also there are increasing calls to enhance Java and because of backward compatibility concerns these proposed enhancements are handicapped because they cannot introduce new keywords to support the enhancements.


CUSTOMER SUBMITTED WORKAROUND :
1. For an API that contains a Java keyword you can write a wrapper, in a language other than Java that does not use that particular keyword, which translates the API into something that Java can use.

2. If the source and a suitable compiler is available the API can be refactored to avoid the keyword, this is what happened to JUnit, which originally used “assert”.

3. In the case of extensions to Java symbols can be used instead of keywords. This looks odd in Java that has a tradition of unabbreviated keywords and is a barrier to readability. The readability issue is why Java uses unabbreviated keywords. E.G. “extends” is much clearer than “:” (as used in some other OO languages).

All these options are messy and second best.
Posted Date : 2007-01-30 11:47:15.0

N/A

(As an aside, #3 seems a bit unclear:
  3. Any “package” or “import” statements that import a name into the files name space gets name
  mangled by putting an underscore either side, e.g. “assert” would become “_assert_”.
I believe the intention is that if an import statement causes a type to be imported whose simple name clashes with a keyword, then the type's name is manged via _..._)

This part of the justification is reasonable:
  With increasing use of languages other than Java on the JVM it is inevitable that some Java
  keywords will be used by APIs written in these non-Java languages and therefore these APIs
  will be unusable from Java, e.g. an API with a method called “final”.

I think #5 supports it by allowing an invocation _assert_(...), _final_(...), etc, to be written in Java source, and appear in the classfile as method descriptors called assert(), final(), etc. Two points spring to mind:
1) Source code could already define methods called _assert_, and using 'source' would mangle it into assert() in the classfile, and that would be very odd. The defining code could find that calls to its pre-existing _assert_ now invoke the imported JUnit version. Other, un-recompiled, code that calls _assert_ will now fail to link, so 'source' essentially breaks binary compatibility.
2) What 'source' level would cause _final_(...) to be translated to final() ? If we make it 'source 7', someone will want it to be enabled by 'source 6'. Furthermore, Sun cannot possibly know which keywords cause difficulty, so we will constantly get requests to add keywords to the list of escapable (via _..._) keywords.

Now, this part of the justification is less reasonable:
  Also there are increasing calls to enhance Java and because of backward compatibility concerns
  these proposed enhancements are handicapped because they cannot introduce new keywords to
  support the enhancements.
because we CAN introduce new keywords provided they are in a context-sensitive position. For example, in a proposal for type aliasing - import ... as ...; - 'as' is a context-sensitive keyword and doesn't preclude 'as' being used as an identifier elsewhere.

Overall, this request identifies a reasonable problem, but I am not sure it's the right way to solve it. But I will leave it open for the time being.
Posted Date : 2007-02-07 14:42:43.0

The posting to the bug database lost some of the formatting in the original, e.g. the & in the above code was originally three dots to signify something omitted for brevity.

The evaluation says "we CAN introduce new keywords provided they are in a context-sensitive position".

That's true but the problem is that Sun don't seem to want to do that. At least, 'assert' was the last keyword added, and I get the impression that this caused some flak, since it broke the occasional bit of source code, and this has scared Sun into adding no more keywords.

Case in point: in Java 5 the use of '? extends Foo' and '? super Foo', though an ingenious bit of keyword overloading, is pretty desperate. The normal uses of 'extends' and 'super' don't have the relationship that they do in generics, so surely they were driven to this by not wanting to add new keywords.

(FWIW I might go further and suggest that the many complaints that Java generics is 'too difficult' or 'turgid' are due to this clunky syntax. List<+Foo> and List<-Foo> were floated at one stage, instead of List<? extends Foo> and List<? super Foo>, which IMHO is easy enough to understand and soooo much more readable. Granted, this wouldn't require new keywords.)

Another example from Java 5 is 'for (x : y)' which would have read much better as 'foreach (x in y)'. Arbitrary punctuation is exactly what Java was originally trying to get away from, which is why we have 'extends' and not '::'.

The evaluation says "we CAN introduce new keywords provided they are in a context-sensitive position".

That's true but the problem is that Sun don't seem to want to do that. At least, 'assert' was the last keyword added, and I get the impression that this caused some flak, since it broke the occasional bit of source code, and this has scared Sun into adding no more keywords.

Case in point: in Java 5 the use of '? extends Foo' and '? super Foo', though an ingenious bit of keyword overloading, is pretty desperate. The normal uses of 'extends' and 'super' don't have the relationship that they do in generics, so surely they were driven to this by not wanting to add new keywords.

Another example from Java 5 is 'for (x : y)' which would have read much better as 'foreach (x in y)'. Arbitrary punctuation is exactly what Java was originally trying to get away from, which is why we have 'extends' and not '::'.

(FWIW I might go further and suggest that some of the complaints that Java generics is 'too difficult' or 'turgid' are due to the clunky extends/super syntax. List<+Foo> and List<-Foo> were floated at one stage, instead of List<? extends Foo> and List<? super Foo>, and IMHO that would be easy enough to understand and soooo much more readable. Granted, this wouldn't require new keywords.)

First - sorry for not responding sooner - watches don't work for me - I don't receive an email telling me of a change :(

You are correct in your assumption about point 3 which isn't clear - should have proof read it better :(

With regard to your points labelled 1 and 2 above:

1) I thought about the problem of source code that already contained _assert_ and decided that this was likely to be very rare and therefore not really a problem. Your call could be different and you could tackle this with a unique name mangle, e.g. #assert or a uniquely numbered name mangle assert_1 (like dos does to deal with long file names). On balance I decided that _assert_ was likely to be sufficiently unique - its a call as to which option is best and doesn't effect the overall proposal significantly and I suggest that the majority opinion should rule :)

2) For source levels I was thinking that all the major numbers would be supported, i.e. source 1 to 7 would be valid. All the relevant keywords at each source level would be mangeled but not any subsequent ones. EG source 5 and above would mangle assert but 4 and below wouldn't.

With regard to context-sensitive keywords (CSK) the two proposals are not mutually exclusive, but it probably isn't worth doing both. However I think it is definately worth doing one or the other; the proposed syntax for extensions like closures and implicitly typed variables is a long way from standard Java syntax and I think that both these proposals would be improved by using unabbreviated keywords like the rest of Java uses. Java uses extends rather than : for example.

The advantages of the source proposal over CSK proposal include:

1. Supports 'foreign' languages

2. It is the way Java currently works

3. It is easier for tools, e.g. IDEs and pretty printers

4. Sometimes the CSK can be difficult to follow. If implicitly typed variable declarations were introduced with the CSK var then the following would be possible:

var value = ...; // implicitly typed value
var.value = ...; // assignment to field value of instance var

5. Experience with CSK in FORTRAN isn't great!

Some further thoughts on the advantages of the source statement over context-sensitive keywords.

1. source ensures that the programmer and the compiler both agree on Java version number :)

2. source allows greater freedom with changes:

2a. package could be made compulsory in 7

2b. switch could detect fall through in 7

Would some usage of annotation be more useful in this context rather than adding new keywords?  

I think you would use an annotation were you can, but there are many places where an annotation isn't suitable. For example the module keyword proposed for Java 7 or to avoid the very un-Java like symbols used in some closure proposals.