DESCRIPTION OF THE PROBLEM :
The Java language lacks "structs".

The lack of structs gives rise to serious code complexity
when dealing with external systems, and serious performance
implications for code that performs a large amount of I/O
with external systems.

Because of the way Java and Objects are intertwined it would
probably not be entirely practical to insist on the same
struct semantics that are found in C. In particular the lack
of pointers and pass-by-value semantics for a trivial
implementation of structs makes an apples-to-apples theft
from the C language impossible.

However, as the particular problem I am trying to crack is
dealing with high-performance I/O operations (in fact,
OpenGL graphics rendering in this case), I believe there is
an elegant solution based on NIO buffers which should
provide wonderful syntax, high performance, and 100% robust
operation with minimal impact.

The solution I propose is to associate a "struct" with a
Buffer. The buffer's primitive type then becomes the struct,
i.e a position(n) command on the buffer moves the buffer to
access the nth struct in a buffer of these structs.

A struct should syntactically accessed the same as a normal
 customer , ie. passed by reference, and so on. Struct field
assignments and reads take place directly to and from memory
with no intermediate copy.

A struct should be defined in Java using simple C-like
syntax, and generate a .class file like any other Java class
construct. The difference is that the fields may only be
primitive Java types or themselves structs. Each field is
not stored in an instance of the  customer  itself; instead they
are mapped to offsets in memory from a Buffer. A struct
therefore must have a reference to its parent Buffer to
prevent the buffer from being garbage collected and structs
being allowed to access unmanaged memory.

public struct Vertex {
private float x, y, z, nx, ny, nz;
private byte r, g, b, a;

void normalize() {
//etc
}

};

A struct descends from the class javax.nio.Struct, which has
a package-visible constructor which takes the parent
StructBuffer as a parameter:

public abstract class Struct {

private final Buffer buffer;
private final int position;

Struct(Buffer buffer) {
this.buffer = buffer;
}

}

The usage of the new "struct" keyword, which is
syntactically similar to "class" and "interface", instructs
the runtime not to reserve space for the structs' fields
because they will instead be referenced directly from a
buffer at the specified position.

Then a buffer of this struct should be initialized, perhaps
using the new generics feature to simplify the syntax:

// Allocates a struct buffer with 10 structs in it at the
specified address, which can be a long
StructBuffer<Vertex> buf = new StructBuffer<Vertex>(10,
address);

// or from a ByteBuffer, which will check that the
bytebuffer's capacity is an exact multiple of the struct's size
StructBuffer<Vertex> buf = new StructBuffer(byteBuf);

Then values from the buffer can be referenced directly as
Vertexes:

Vertex v0 = buf.get(0);
v0.normalize();
Vertex v1 = buf.get(1, v1); // Constructs v1 if necessary

The underlying implementation in Java should be identical in
speed to C for the common case (with bounds check
elimination and native byte ordering), yet because a struct
can only be constructed by an existing StructBuffer, it will
have all the safety of bounds-checking and garbage
collection which will prevent illegal writes to memory.

The two problems solved by this RFE would be the sheer
complexity of maintaining multiple buffers to access
different parts of a struct with different primitive types;
and the massive increase in performance afforded by being
able to reference memory directly and bracket accesses to
members of a struct with one bounds check.

You will understand if you do graphics programming; it is a
key requirement of games programmers to have greatly
simplified and efficient access to these operations or the
benefits of C++ simply outweigh the extra complexity and
overhead inherent in Java.

(Review ID: 181371) 
======================================================================
Posted Date : 2005-09-13 18:27:04.0

N/A

There are some interesting ideas in this proposal,
however, we do not see the need for a new language
feature.  We believe this can be solved as a library
without changes to the language and VM.

As a consequence, we have moved this RFE from
specification to java.util.

We are currently investigating various solutions
and if it should be part of the JDK.  In any event,
it will not be in time for Mustang.

More details will follow.
Posted Date : 2005-09-29 21:14:50.0

We are currently considering if this should be addressed in Dolphin.

A combination of library methods and annotation processors should
make it possible to write something like this:

@Struct
interface CK_SLOT_INFO {
  @PaddedString(' ', "utf8", 64)
  String getSlotDescription();
  @PaddedString(' ', "utf8", 32)
  String getManufacturerID();
  @Unsigned(32)
  int getFlags();
  CK_VERSION getHardwareVersion();
  CK_VERSION getFirmwareVersion();
}

(Using an example from PKCS#11)
Posted Date : 2006-02-10 01:09:44.0

Response to SDN comment from Cas: you're right about the category.
I have moved this RFE to java.nio.
Posted Date : 2006-02-14 21:08:38.0

Responding to SDN comments:

We are considering this RFE as a request to access low-level/native
data structures as can be done with C structs (and C++ classes).

If you're looking for stack allocated objects or value types, you
should vote for RFE 4213096 or RFE 4809540.

Clearly performance will be paramount to the success of any solution
(library based or language syntax) but claiming that it can't be done
with classes is unfounded.  Modern JVMs, such as HotSpot, are very good
at optimizing Java byte code.

The only real difference between a class and a C struct is who generates
the code for accessing members.  Consider a C struct like this:

struct Foo {
    int a;
    int b;
} Foo;

If you have an array of Foo, foos, then a C compiler can compile
an expression like:

foos[6].b

to i386 machine code like:

        movl    foos, %edx
        movl    52(%edx), %eax

where 52 = 6*sizeof(Foo)+sizeof(int) 

You can accomplish something similar in Java:

static int SIZEOF_FOO = 2;
static int SIZEOF_INT = 1;
int getFoo_b(IntBuffer foos, int index) {
    return foos.get(index*SIZEOF_FOO+SIZEOF_INT);
}

The expression similar to foos[6].b would be:

getFoo_b(foos, 6)

HotSpot can inline this:

foos.get(6*SIZEOF_FOO+SIZEOF_INT)

which can be further optimized:

foos.get(13)

This then all boils down to how efficiently HotSpot can handle
IntBuffer.get.  However, as always, C has a performance advantage over Java:
no range checks.  This means that the best we can hope for in Java is
something like this:

        cmpl    $56, size
        jbe     .OUT_OF_BOUNDS_FAILED
        movl    foos, %edx
        movl    52(%edx), %eax

If the Java platform can't handle foos.get(13) efficiently, then you will
get bad performance no matter what solution you dream up.  I don't see how
saying "structs" solves this problem.

The really tedious part of accessing low-level data structures is writing
methods like getFoo_b and constants like SIZEOF_FOO.  This can be automated.
The proposal with @Struct is to have an annotation processor generate such
methods and constants based on a higher abstraction.

So do us a favor and let us know what is wrong with the performance of the
java.nio buffers if it doesn't meet your needs.  That is a realistic way of
solving your problems.  Asking for C structs is not.
Posted Date : 2006-02-18 01:55:38.0

Apparently, my previous evaluation could be misunderstood.

I was responding to this SDN comment:

"I'm for structs. Classes are way too expensive to create
 in many situations, and are sometimes semantically invalid
 in various contexts."

So just to be clear:

* We are considering addressing the requirement of accessing
  low-level data structures in Java.

* We are not considering adding C-like structs to Java at this
  time.

* If you're interested in C-like structs, vote for other bugs.
  (But don't get your hopes up).

* If you don't think the @Struct proposal will solve your
  problem, please let us know.  A precondition for the
  @Struct proposal is support from NIO, for example, as Cas
  suggests, "memory mapped objects as a sliding window over
  ByteBuffers".
Posted Date : 2006-02-22 21:24:56.0

More discussion on this RFE can be found at JavaGaming.org
in the Community Process forum.

Cas :)

I'm against this RFE because it's very seldom used,
complicated and can be solved in other ways.

Interesting.
I've just written some C++ to generate java code directly
from a structure, and a short description of what part
of the struct is passed to java.

This proposal is exactly what I was looking for before I
wrote the code.

100% support this enhancement.

megagurka,
it's only seldom used if you don't do these kinds of things. 
However, if you're doing a) realtime graphics or b) 
high-performance IO to external systems you will appreciate 
what it brings to the table.

It cannot be solved in any other way; this is a language and 
VM enhancement to make a complex and inefficient task 
easier.

And it's a very simple RFE. Map Java fields directly to memory 
in Buffers.


Cas :)

This is also useful with networking code.  In C/C++, one
technique for fast datagram packets is to get back the
pointer to the buffer that the datagram packet writes to,
then simply cast that to the struct type that is
appropriate.  That way you can access whatever data you want
without the extra copying. 
So this is generally more useful than graphics programming
only.  I imagine many other great uses will arise.

As I see it, this is only a performance issue. A similiar
feature can easily be implemented using interfaces and
specialized vector classes, although the performance might
be a bit worse. I'm against changing the Java language just
for performance reasons. In the end we will have a language
like C, and nobody wants that. Java is designed to be safe
and simple.

Do you agree with ByteBuffers? 
"A similiar feature can easily be implemented using
interfaces and specialized vector classes"  Java didn't
"need" ByteBuffers for functionality, you can do the same
thing with the regular JNI method passing.  But it's
incredibly ugly and inefficient for large data, so they
added ByteBuffers, the needed improvement.  This RFE just
takes that one step further (or finishes the job...) by
making automatic type mapping, to make code much cleaner and
more efficient.

ByteBuffers are not a language change.

This feature is absolutely NOT needed. 

And will restrict structure of ALL implementations of JVM. 
What if I would like to create my own JVM -- I must 
implement this feature?

If you need specific binary layout, you may:

a) Compile custom JVM, at least own hotspot version;
b) Use byte arrays;
c) Use JNI.

I understand why structs would be needed, but for the sake 
of consistency, a clean object-oriented language like Java 
should stick to its paradigm: "objects are smart structs". To 
deal with code that is intensive in something that Java does 
not provide, the good solution has always been JNI.

That's my two cents...

Thib ;-)

If I write an ordinary class wrapping a byte buffer with 
accessors for 'structure' members, then surely we could have 
a JIT which recognised this pattern and where the byte order 
matched the native byte order could generate code 
equivalent to C struct access. In practice it would also have 
to insert checks for the current byte order (which can be 
changed at any time), but many of these tests could later be 
eliminated (as redundant).

So something close to the speed of C should be achievable 
without changing the language at all. Yes you still have to 
use accessor methods instead of direct field access, but this 
is common good practice anyway. It may also help both 
devlopers and JIT implementors to have a standard class 
pattern for this purpose.

I don't understand why this RFE proposes a language change
instead of a JVM enhancement for classes which only contain
primitive fields. Could someone explain why adding another
data type would be better overall than simply encouraging
Sun's HotSpot people to optimize for classes with a certain
structure?

After all, this is what is done for many other aspects of
the language. There's no keyword to describe that a "getter"
only returns the value of a final field, and thus should be
inlined, and yet that happens, for example. I think that
optimizing the VM is the best bet (but I'd love to hear
arguments to the contrary).

-1

http://developer.java.sun.com/developer/bugParade/bugs/461
7197.html

Provides a more elegant solution, it proposes immutable 
objects which can be packed into arrays, buffers, etc. Thus 
immutables would provide the performance you require but 
would also provide many other useful features in addition.

@megagurka: A few other enhancements for you to ponder 
that have changed the Java language but which could have 
been implemented in other ways:

Inner classes
Enumerations
Generics support
Assertions
Static import

As an aside: Gosh, six months on and no Sun evaluation yet?

Charlie.

re: crmdobbie WED AUG 13 04:57 A.M. 2003

> As an aside: Gosh, six months on and no Sun evaluation 
yet?

Nope, they're probably still laughing

>>Inner classes
>>Enumerations
>>Generics support
>>Assertions
>>Static import

Actually changed the Java Compiler and not JVM...

Very interesting.  I think Sun should look at this as a
problem, then find a best solution the solves the problem. 
This reminds me of engineering problem.  You define a
problem (not a solution), then list all possibilities, then
evaluate each one, then finally pick a solution.  That will
hopefully satisfy many of the posts I see above.

Struct/Union now exist and are public domain.
You may check them at http://jade.dautelle.com

Note: There is no need for a new keyword.

bestsss (SAT SEP 06 02:05 P.M. 2003) wrote:
> Actually changed the Java Compiler and not JVM.

Yeah, it was a language change he mentioned, not a 
bytecode change.  Although we just had a bytecode change 
with 1.4, didn't we?  Must look that up...

Struct/Union classes provided by the JADE library
(http://jade.dautelle.com) are PUBLIC DOMAIN.
Therefore, nothing prevent SUN to include them into their
next distribution (in the java.nio package) !

If SUN is listening, please let us know your intention on
the matter.

Regards.

We need structs to beat C#

I don't think we need this as a new language 
extension. For me the problem is directly related to 
boxing of native types into Object's, which take too 
much memory due to the Object header.
I think there's a place for a top-level Primitive class with 
zero headers, and from which Object would be derived. 
So all primitive types would become descendent of 
Primitve as well as Object, and auto-boxing of primitive 
types would consist primarily in associating a primitive 
type with its associated Class reference.
Once primitive types can be handled with auto-
boxing/unboxing in a clever emulation where a 
PrimitiveHolder descendant of Object is used to 
reference the native type with a pointer to the internal 
storage of the primitive type, this PrimitiveHolder can 
become the parent of Byte, Integer classes and so on.
The interest here is that PrimitiveHolder is a fixed-size 
Object, handled internally by the JVM, and that will be 
able to be used for lots of buffer-oriented structures to 
add them, if needed, the appearance of an Object. All 
PrimitiveHolders could be allocated in a specific and 
very efficient memory pool as they will all have fixed 
size.
Let's remember that the interest of Object is its ability 
to manage references that can be counted, garbage 
collected, or locked, or identified with dynamic types at 
run-time with the .class pseudo-attribute. Most objects 
in Java apps do not need these features and adding 
this extra Object header to all objects is too expensive.

So a first step is to reduce the memory cost per Object: 
this is where too much memory is consumed. This is 
not a language problem but a JVM implementation 
problem: in 32-bit environments, we basically need 4 
additional bytes per Object instance to store the .class 
reference, plus may be 4 bytes to reference a locking 
queue (this extra cost could be allocated later in 
instances by relocating them in a lockable memory 
pool at first use of a lock). Nothing more. All the rest 
should be stored out of object instances. Boxed 
primitive types could also be allocated in specific 
pools, where they would be grouped together in 
aligned pages starting by the .class descriptor (this 
would save the cost of the .class reference pointer per 
instance, as a single page of primitive instances could 
store several thousands of them). Also the VM can play 
on alignment of internal pointers to use the lowest bits 
to indicate how the referenced object is aligned in its 
page.

A single lowest in the pointer can indicate whever the 
instance is paged or not (this should be the case for 
all primitive types that will be grouped in pages 
containining instances with the same .class reference 
stored at the begining of the page) else another bit will 
specify if the aligned instance has a leading locking-
queue pointer, etc...
There's lots of storage optimization here that will allow 
Objects to save memory and still preserve the capacity 
of being type-identified at run-time, garbage-collected 
and locked/monitored.

The next step is to allow boxed primitive types to be 
used as if they were Objects through emulation, 
without requiring this header (There's certainly a way to 
conciliate primitive types and the Object hierachy in a 
single hierarchy that will have an implicit API supported 
by the language compiler and the VM runtime class 
verifier, rather than through explicit labelling of types at 
run-time within each instance).

We should be able to vote _against_ bugs (like this one) as
well as vote _for_ bugs.

You would only vote against this particular enhancement if you 
had absolutely no understanding of what it was for. In which 
case it would be wiser if you left it alone, don't you think?

This sort of thing also comes up in network 
programming. In C, you're used to having a char bigbuf
[1024] and then saying struct ip_header *iphdr = bigbuf 
+ n.

Some way of taking an unstructured buffer and then 
imposing structure on top of it without having to 
perform copies would be a good thing.

Could Sun include the public domain classes Struct/Union
from the open-source JADE library (http://jade.dautelle.com)
into the upcoming Tiger JDK1.5 release (the best location
would be in the java.nio package, e.g. java.nio.Struct and
java.nio.Union).
Thanks.
Note: Here is a link to the Struct class description:
http://jade.dautelle.com/api/com/dautelle/util/Struct.html

I recently had to create an interface between a Java 
application and a C++ library that maintains a very 
large network data structure in memory (approx, 100 
milliion objects).

I expose this data to Java through DirectByteBuffers 
(which were a great improvement over 1.3). Cas's 
proposal is a nice improvement to the language that 
would have significantly reduced the effort I had to 
expend to put the interface together, plus make it very 
easy for the VM to optimize away the very large number 
of bounds check which currently occur.

God bless,
-Toby Reyelts

Here is the simple.

Regardless of the struct member types, the data is not
accessible to you from somewhere other than block memory
without performing an IO operation. Thus, your example
[code]

public struct Vertex {
private float x, y, z, nx, ny, nz;
private byte r, g, b, a;
}
[/code]
requires (assuming 4byte floats) 28 bytes. A c programmer
has the advantage of being able to do a single 28 byte read
and then use the struct offsets to access the second byte of
the third float without performing any additional reads.

However, the java programmer writes,
[code]
public interface Vertex {
float getx();
float gety();
float getz();
float getnx();
float getny();
float getnz();

byte getr() //{ buf.get( position + 24 );};
byte getg();
byte getb();
byte geta();
}

and only has to read one byte per record while iterating
over the alpha values in an array of vertexes.

Let Java be Java. We dont need legacy C ideas, what 
we need is for the overhead of objects to be less and 
to remove the problem of boxing primitive types. 

Define a Primitive class (defining no methods or 
structures) that object extends and all the primitives 
implement as verdy_p suggests.

What we really need is a KitchenSink object.
I have suffered needlessly for years with out.

Too many people can still read Java code. We
need to compicate it more by adding useless crap.

Any kind of serious parser will benefit from this RFE.
100% support this enhancement !!

Regardless of the solution, this RFE hasn't even been
EVALUATED in over a year....
Can some JDK rep reply ANY kind of opinion please?

Since submitting the proposal I think there are a few small changes I'd like to make to the design.

Firstly I'd like to state that a struct should have a byte offset rather than a "struct-sized" offset, which allows you to read data from a struct from an arbitrary point. Of course this means that the VM will have to validate the memory alignment of all internal members of the struct; if all members fall upon an alignment allowed by the underlying architecture then the fully-optimized memory path would be allowed (the VM should probably mark a struct instance as being byte-alignment valid or not). On x86 architectures at least merely checking for 8 byte alignment of the struct's offset will probably be sufficient.

A further simplifaction is to remove the concept of a "StructBuffer" and simply use ByteBuffers and also use the narrowed type ByteBuffer rather than the superclass Buffer in the Struct class.

And another simplification is to allow Object fields in Structs but to simply initialize these to null whenever the offset is changed to insist that the developer take care of Object references. 

Again it must be iterated that the entire functionality of Structs can be simulated by the VM in pure normal bytecode by manufacturing adapter code as necessary to read and write mapped fields out of a ByteBuffer in the manner that the normal Java developer must currently labouriously code by hand. But this framework allows the VM to make some extremely powerful optimizations.

It is high time that this RFE received a look in from the VM engineers as one of the critical blocking features to using Java for high-performance video gaming is the current huge overheads in storing huge in-memory structures such as BSPs and octrees and the memory bandwidth / bounds checking overheads of blasting data in and out of buffers for processing by rendering engines.

Cas :)

trobauer:  Heh!  I concur with that.

Seriously, if you want to really mess around with stuff at the concrete machine level, you need to use a language other than Java.  You'll be gaining speed (if you know what you're doing) and losing other features of Java, but that's your choice.  But please don't force the rest of the Java community (where, for example, high degrees of robustness are very valuable) to have things that are highly crackrock for reliability.

(FWIW, I'd love to see all GUI support removed from Java, as it is diverting a lot of developer effort from other tasks which are much more useful to me, and there are other languages and toolkits which are far better for GUI development. You're welcome to disagree. ;^)

I received an e-mail from SUN developers last month.
They ARE using the public domain JADE's Struct/Union (http://jade.dautelle.com) for interfacing with C/C++ code!!!
If  EVEN developers working for SUN are using these utility classes, why not include them into the JDK ?

This RFE is evidence that Sun should allow us to vote *against* RFEs,
not just for them.

Please, don't add this.  If you want to write C code, use C.

We're talking about an OO language, Structs are definitely NOT OO and therefore have no place in Java.

If you want structs, use C or another procedural language (even in C++ they're retained only for backwards compatibility with existing C code for people having mixed projects).

Do you guys actually understand what this proposal is about or do you just write web applications all day? Look outside of your own application spaces and understand what structs actually *is* and what it *is not*.
1. It is *not* a C-struct. These are 1st class Java objects just like any other.
2. It is *not* non-object oriented programming; structs are objects
3. Structs do *not* have to include any change to the language spec; the implementation can be entirely VM specific and can be done by writing optimized machine code or by generating bytecode.
4. Structs are *not* for people writing web applications and other trivial Java; they are for high performance I/O programmers. There are a lot of us. They are for developers who write application servers, high performance interfaces to legacy systems and games. If you don't do anything like this structs is probably irrelevant to you. As is NIO and OpenGL.

There are *no* valid reasons to vote against an RFE. If you don't care for something, simply ignore it. If you want to vote against something then it is likely because you got out of the wrong side of bed this morning. Go back to bed and think about other developers with other needs for a bit. Come back with a more understanding attitude and an open mind. Don't tell us we need to use C. I'm a Java programmer.

Cas :)

I do a lot of GP (genetic programming) work in Java, which requires the management of literally hundreds of millions of objects, with working set behaviour that the JVM hasn't a hope in hell of working out, no matter how smart it's heuristics.

At the moment, I have to write the entire application WITHOUT OOP because of the lack of structs. Why? Because Sun's memory overhead per object is so large when multiplied by 100 million that things go crazy. So I have to simulate structs by writing objects where each object is a byte array, with my own classes to interpret these as objects.

This is crazy. I have to abuse java (and thank God that basic datatypes were included in java 1.0 at the last minute) because the only other option is not to use java. Many of my colleagues use java for their evolutionary work, and many more would use it if we had structs. That's not a reason to add it, but it is an example of how varied the applicability of these things is.

I've met other people with needs for vast numbers of small objects too, usually (as in my case, and in the BSP tree case) of precise fixed size. It seems there are a lot of us with these needs - but, as Cas says, it seems we are being held back by ignorant selfish fools who have too little programming experience to understand the problems of vast numbers of objects, and tar us with insults as if we were incompetent.

I find it outrageous that people post in this bug comments along the lines of "don't use java" or "you are abusing OOP". Structs would ALLOW me to use OOP!

Is it just me being stupid or doesn't this also provide a compile-time check for stuff that at the moment you don't even get checked at runtime? Certainly, my hacked attempts to simulate 100 million objects in memory suffer from plenty of bugs for stuff that could have been checked by a compiler, if it were more aware. And the code would be a lot easier to read.

Basically this is an attempt to have something workable without a new keyword.


struct proposal

Simply do interface like this

public interface gfx implements Layer, Boundary16B{
	int marker;
	byte data2;
	byte pading1
	byte pading2
	byte pading3
	int data4;
}
Basicaly this is a struct. You should lift the restriction for interfaces that implements Layer (or Compiler.Layer) to be in a separate file, and add support from JVM. Also I would like to have support for larger data types like:
package interface gfx2 implements Layer, AutoPading{	
	u512 data512; //512 bit large data.
	i128 RiscPS2; //128 bit large signed integer
	int marker;
}
And yes this struct would be acessible from package level. //Is it smart? Dunno.

implements Layer is something like implements Cloneable. It will simply tell the JVM that this is a different object than common class.
(Of course another possible keywords instead of Layer are Layered, or Structure.)
Boundary16B means a structure in memory aligned to 16 Byte boundary and prepared for SSE2 instructions.

Raghar

Congratulations, RFE-4820062!  18 months old, and still managing to avoid notice by any Sun employees.  An inspiration to us all.

More seriously - people, please put forward more intelligent arguments than "C-structs don't belong in Java"!  This RFE isn't about making Java less-OO, it's not about adding pointers to the language, and it doesn't require a language change.  It's about efficient IO and elimination of bounds checks.  Is that really such a bad thing?

I think it is brain-damaged to include struct in Java. Java is an OOL, so struct has nothing to do with it. I think if you have a problem with Java/C interfacing, you have to post a RFE for JNI, but please let "struct" be out  Java. Every "smart" and "performance enhancing" practice u use in C/C++ is out of question in Java : it adds complexity and code maintenance costs. "smart pinters" and other tricks hav, I hope, been forever banned from Java !

I like idea of structs very much, there must be an ability to write fast code without native code. I'm sure that JVM should be extended to support structs, but strutcts objects itselt can be written in some other, Java-like language, not Java itself, and these objects can be used from Java as usual objects.

example:

in other language:

strutct Vector {
float[4] coordinates;
public void normalize();
public float getX() { return coordinates[0]; }
// ... and so on
}

In Java:

void run() {
Vector[] vectors = new Vector[100]; // this is array of objects, not array of references
for (int i = 0; i < vectors.length; ++i) {
 // ...
}
}

+1
This is a must for Java. The benefits are just too great to overlook. Don't know if the proposed solution covers all bases for having high performance data structures like structs; nevertheless other RFE’s covering immutability, const and reducing object overhead must be evaluated by Sun and JCP also.
As a side note, maybe indexed access should be included to a struct like object, thinking again these will bring up the operator overloading discussing again..... Maybe something like Groovy's implicit operator overloading will work.

This structs proposal is almost as appealing as cold oatmeal.  Syntactic sugar for eliminating the object header overhead is a good idea.  However, better, demonstrated proposals are tuples and lightweight objects: 

"Adding Tuples to Java" 
http://tinyurl.com/6vksn  

"Kava: A Java Dialect with a Uniform Object Model for Lightweight Classes" 
http://tinyurl.com/7297j

I'm glad no one is advocating C#-style stack allocated objects (structs).  The correct solution for that problem is to use escape analysis.

Efficiently casting raw memory to structured memory is one of the 3 last major performance issues preventing much wider java adoption.  I strongly urge sun to work on some solution to this problem.

Once again let's just hammer home these points:

Tuples & LWOs are all very well and good but they are major changes to the language and the VM and solve a much broader range of problems for mathematicians. We are as likely to see them implemented in the language this decade as we are to see Java on Playstation 2.

The Structs proposal here is a lightweight solution which can be implemented with no syntactical changes. An implementation of the Structs proposal here can be made simply by modifying the classloader to manufacture bytecode initially. Later the VM can be enhanced to generate intrinsic code and remove buffer bounds checks. It's a quick solution, easily implemented, and it does what it says on the tin: provide fast, safe access to memory by superimposing Java objects on it.

Cas :)

Frankly I don't see why there's opposition to this RFE.  If you don't need structs, don't use them.  As far as not wanting C-style constructs in Java, did you even bother to read the description?

+1 for me anyway.

Give it a +1

I know this is probably messing at the edges but how about you opt for a "native class" - perhaps might hammer home that these aren

as ... the rest of my comment was lost ....

public native class MyBufferedStruct

etc

Its not  "really" a struct after all - just a facade onto a buffered array - and facades are proper OO constructs :-)

What isn't clear is how you address the following:

1. endianism - when does this happen ? It ideally has to be dynamic - so I guess that  to make this properly portable (remember that  class files are location agnostic) the bytecodes would need something like:
switch 2, switch 4, switch 8 instructions which would pull the value from the buffer onto the stack where it can then be treated like a proper primitive.

2. Security - this type of access leaves the biggest security holes - as it fundamentally allow you to map any data structure onto a section of fixed memory - even when you shouldn't be doing this - so I guess some stringent access control needs to be placed on the use of this.

3. How does it deal with array types ? I guess you'd have to treat arrays as substructs of the primary struct - then the same operations can be followed. (i.e. any access of the form myStruct[index].a[subindex] would perform the "switch" operation on the subindex ... - although the cases are slightly different (you'd want a[subindex] to actually return the primitive rather than a "native class" object

-1,

Structs represent state without behavior. If you have state, you will need behavior to manupulate it. Co-location of state and behavior is a main tenant of OO.

However, it would be interesting if we were able to use an object (class or interface) to provide a structure to an mmap area of memory.

What we are realy asking for here is control over how fields are mapped to memory. In this case, we wish to create a NIO buffer, allocate Java objects in the buffer (using our own mapping logic) and expose those object references to the outside world.

I think that the logic and proofs needed to handle this kind of things safely have been worked out in the context of real-time Java, and the multiple garbage-collection pools that it supports. Although I don't know the implementation details of real time Java, my gut tells me that  it would be possible to 'slot in' your own heap implementation that effectively gives this behavior.

Would someone like to tell me if I'm wrong?

I agree 100% and vote for this enhancement!

I see overwhelming evidence (and optimism) that structs can solve MANY, VERY important real-world problems, some of which can only currently be solved by either moving away from the Java platform, cutting features, or accepting poor performance. On the other hand, I see no ‘reasonable’ arguments here explaining why it shouldn’t be considered. And I suppose it doesn’t have to be structs, but that’s the term people know. I would consider an alternative implementation as long as this basic problem is solved. 

The Struct/Union classes are availbable for inclusion to the JDK. They have been extracted from the Javolution library (http://javolution.org). See JavaLobby post: 
http://www.javalobby.org/forums/thread.jspa?threadID=17098&tstart=0

-1, stick to C if you're missing structs

Doing direct I/O with structs was generally a bad idea in C 20 years ago. If you REALLY need to optimize I/O this much, you shouldn't be using Java to do the I/O.

I like the JADE library (http://javolution.org/api/index.html) approach of Struct and Union to solving this particular problem, leaving the actual optimization implementation to the JVM.  I would like to see Sun incorporate this code (the author is even clearly volunteering it freely) into the Offical Java APIs.

I am against modifying the language/compiler to implement this.  I think the optimizations can be more safely implmentened with the Java type system staying based around Object and leaving the JVM to do analysis and optimization when it notices the implementing class is essentially fina/immutable and can eliminate bounds checks and use pointer arithmatic to optimize access.

"Doing direct I/O with structs was generally a bad idea in C 20 years ago."

Please remember that I/O is not only about files these days. We are talking about interfacing with GPU, I/O to/from graphic card. Do you want to say that if we want to do 3d games, we should stick to C and forget about java, just because 3d games require non-kosher-OO interface in few places ?

http://javolution.org/api/javolution/io/Struct.html

Looks interesting...

I'm all for this RFE.

I have worked on various projects where we were integrating Java systems with IBM Mainframes.  Most of these projects have been delayed because of the difficulties in getting an Object based system to integrate with a Records based system.  And most of the delays were due ot the amount of coding on the Java side required to move information based on Record Structures into Objects.

Not only should there be structs, making such a change wouldn't impact at all those people that don't use them.  

Yet when you working with integrating different systems structures are a must, other wise you spend far to much time parsing a byte array into Java data types then back.  Especially when you dealing with bidirectional asyncronous transmission of record or message bassed information.

The Corba, Java RPC and XML arcitecures are far to slow to be used on anything but the smallest of systems that have few concurrent useres.

When you're dealing with interfacing Java Clients with large high speed systems, it really is a major need to bypass any middle ware, and transmit a structure that is understood by -both- sides of the fence.

Okay this is now number six on the list and nearly 2 1/2 years since it was submitted and we've still heard nothing from Sun. It's needed, ignoring it won't make it go away!

STRONGLY AGAINST
If you like struct, deal with non-object languages ! Performance tunning should be done by Sun without "struct". Please, Java is not like C++, an object language made from a language (C) that simplified assembly. Java is not "pure" object like smaltalk, but it must not take more non-object syntax. I'm sure Josh Blosh agrees with me.
Breaking OO design principle to get more performance wil *NEVER* be the solution. The fact that Point attributes are public : passes. Using "struct" : no, stop it ! Java is definitively not "J++".

STRONGLY AGAINST TOO

I agree with Kilwch "Breaking OO design principle to get more performance wil *NEVER* be the solution."

Furthermore, I believe structs cannot optimized by a hotspot compiler that much. Further language structures will complicate the language and the simplicity is the reason I use Java instead of C++/C#. (Don't get me wrong I love Java's Generics :-)).

You CANNOT implement this feature with any performance benefits without a VM enhancement. No specific changes need to be made to the language; the change can be purely semantic with the VM recognising descendents of a particular JDK class.

The proposal CAN be implemented without any VM trickery by generating bytecode within the classloader, however this will NOT YIELD ANY PERFORMANCE IMPROVEMENT.

Furthermore this is part of the NIO packages, not a utility class. It specifically depends on java.nio in order to work.

"You CANNOT implement this feature with any performance benefits without a VM enhancement."

Hard to be certain of that without knowing what the current VM does with nio buffers in great detail. For example, what optimisations if any are performed on IntBuffer.get(int position) if the IntBuffer is a direct buffer? Could the existing optimiser hoist the bounds checks?

Please add this feature at the language level and have it properly designed to be pragmatic and efficient, please no more half backed solutions like generics.
If structures are added, think outside the hotspot box and have a spec that can be implemented on any architecture (embedded for example).
For those always complaining about readability and OOPness, please stop, you can’t have efficient code for fast data processing without this, just use Boolean, Integer, Float etc. instead of the primitive data types and come back to say how much readable and OOP is your code.
Structures are not C, they are a computing reality that every decent OOP language just doesn’t ignore it; you can’t force me to use C because you don’t want to learn a language feature, I like java and I want to code as much as possible in Java. JNI it’s just an ugly and inefficient way to integrate with the platform bellow and in this case doesn’t do the job.

Javolution (link: http://javolution.org/) provides struct compatible IO with C. As for light-weights... well it's been discussed and discussed.

If I understand this recommendation, something like Fortan's EQUIVALENCE blocks would also solve the issue, no?

I think I prefer the idea of having it in a library....

I like the idea of structs, both the advantages and disadvantages. Anyway, what I think should be considered is that the structs actually are light-weight objects, and only shall contain primitive types. It is indeed C, but we shall not constrict us to require programmers to use JNI for everything that isn't "Pure" - java.

And for those complaining about byte-order for networking - the byte-order in network packets are always the same, regardless of system.

Whenever a struct has to be passed to other code through JNI then there is an issue of byte ordering, but if that is well-documented it shouldn't be a problem. Of course the struct declaration could provide an optional alternative declaration for byte order, which may or may not be a good thing. It may be a good thing to optimize performance when communicating with external devices, but it may also cause problems when interacting with other components.

Anyway - as I have understood reading all comments - the actual problem resides in the performance and memory cost of creating Java objects. JNI isn't even an alternative since it is rather cumbersome to work with and is not very efficient when it comes to performance for the management of massive arrays.

If Java is going to stand up against competition from other languages it has to evolve on both low-level and high-level while still maintaining it's portability.

@mthornton: ""You CANNOT implement this feature with any performance benefits without a VM enhancement."

Hard to be certain of that without knowing what the current VM does with nio buffers in great detail. For example, what optimisations if any are performed on IntBuffer.get(int position) if the IntBuffer is a direct buffer? Could the existing optimiser hoist the bounds checks?"

I don't think you realize who you are talking to.  Cas has played with the VM probably more than some people one Java at Sun.

To the rest of the people against this.  Before commenting, try to open your eyes and read before replying.  This is *NOT* C style structs.  It is just an idea borrowed from C, as most everything in Java is.  This will be an API change, not a language change.  To anyone who does not use it, it will be no different from java.nio or javax.xml.  Don't like it or want it, ignore it.   But don't vote against something that has added value for lots of other people.

markbernard, perhaps you should look at this thread on JavaGaming.org: http://www.javagaming.org/forums/index.php?topic=11112.0

There are some quite interesting experiments reported there which show significant performance gains with the current JVM. They just use some undocumented (and probably non portable) dirty tricks.

Hm, the reason behind the proposal is sound. It should be somehow addressed.

However I am not sure that:

1. Language change is needed. Modify Java VM to understand some sign in bytecode ("native" bit of class modifier, special interface, special package and so on.) AFAIR source code for the VM is available, can be changed, compiled and redistributed under some legal terms.

2. It is Sun who should work on this? For example, David F.Backon of IBM labs has implemented
- operator overloading;
- matrixes;
- lightweight objects.

I understand that princec would like to get this feature for free and forever. But if there is a $$$ market (Playstations?), then why not pay to Sun? (Or IBM, they produce JVMs too.)

Instead of whining years long, I'd rather implemented this solution in some good VM.

Strongly against it! Changing the Java VM is definitly NOT an option. 

If you want to use structs, go to Javolution.org. 

A point that I don't believe   (on a quick scan) has been made is just how much extra complexity this RFE would impose on the JVM. We're talking about a whole new mode of data access here. That means, at minumum, a new load and fetch bytecode for each data type (signed and unsigned), plus instructions to access the struture's base address. I think you'd be increasing the JVM's instruction repetoire by something like 10% and complicating the register model significantly. References are not pointers. I think they are probably, in the implementations I've used, indexes into an array of Object pointers, but the JVM can safely assume that reference points to a legitimate Object with a well understood structure. So a reference to a structure would have to use a completely separate mechanism. Introducing stuff like enums and annotations would have been a doddle by comparison because they all partake of the class/Object structure. Struts would not.

Come on Sun!  Some progress on this would be nice!

Strongly against it! Changing the Java VM is definitly NOT an option. 

If you want to use structs, use C#. :-)

Annotations is an excellent way to achieve this (predating my original RFE). No language level adjustments need be made, just some clever VM coding and a few classes in java.nio (I'd use NIO, not java.util, as the RFE fundamentally deals with IO)

Cas :)

Right, annotations should be all that is needed.  Then we could hopefully get an ObjectBuffer<T> that works much like the current primitive buffer types like IntBuffer...

ByteBuffer bb = ByteBuffer.allocateDirect(size);
// following thows an exception if StructThing class is missing the needed annotations
ObjectBuffer<StructThing> objBuf = bb.asObjectBuffer(StructThing.class);
...
StructThing s = objBuff.get(0);
...

I'm for structs. Classes are way too expensive to create in many situations, and are sometimes semantically invalid in various contexts.

I don't want C-structs, what I want is memory mapped objects as a sliding window over ByteBuffers. Specifically this addresses these issues:

1. Simplification of Java code syntax and OO cruft.
2. Memory bandwidth
3. Very easy JVM optimisation to remove bounds checks for random member access.
4. Reduction in object overhead and garbage

I will explain this in more detail to Peter von der Ahé via email. I DO NOT WANT C-STRUCTS! I DO NOT WANT A CHANGE IN JAVA SYNTAX! I just want an extra class in java.nio which will be intrinsified by the JVM or that can be synthesised otherwise.

Cas :)

I do lots of opengl programming, I'm interested in increased performance, but I don't live by it. The way I see it, if this feature is implemented with thought and care everyone will be happy.

I vote for @Struct.
It's really important for graphics and games programming.

I work with real-time media applications (OpenGL, 
video etc.) and it has become very obvious that features 
like this struct RFE will be very helpful in both performance 
and reducing code clutter. Sometimes a change like this
is required to achieve both performance and simplicity in
usage. Many people complain that Java is not present on
the desktop, missing in games, beaten by Flash on the
web and doesn't do photo+video well. I suspect that these
people are not aware of the nature of the APIs required to
do the above. Most are native libs that have slim chance of
being ported entirely to Java. Most are fussy about the way
data is fed to them in order to achieve maximum 
performance. This RFE illustrates that. While there're
examples out there that show Java beating C in some
case, the experienced among us (who knows both Java
and C) also knows exactly where Java can never be as
fast as C. Note that JNI is not counted for obvious reasons.
Java has matured in terms of functionality. It is now a good
time to start thinking of performance. It is just like 
optimisation: leave it to the last as early optimisation is the
root of all evil.

see http://javolution.org

have a look at http://javolution.org/

A lesser known feature in structs in c++ is the ability to specify it as a bit field, e.g:
struct MyBitField
{
    BYTE bit1: 1;
    BYTE bit2: 1;
    BYTE bit3: 1;
    BYTE bit01: 1;
    BYTE nibble1: 4;
}

An instance of this example struct will be contained in one byte.

Will this feature be considered if structs are being considered?

And why don't we go a step further an introduce unions:

union MyBitFieldUnion
{
    MyBitField bf;
    BYTE by;
}

In the above example, changing the BYTE (by) will modify the bits, and vice-versa. Will this be considered?

James

This is a good improve for creating a future batch for the java plafform, I think that is better using the stack for the POJOs

This could improve a lot off the Java plaform, only need to define a struct that is a class but with only contains primitive types or struct, this could be use in a like cobol batch processing or in a POJO system. By definition the Struct made be Cloneable direcly.

RE:However, as always, C has a performance advantage over Java:
no range checks.  This means that the best we can hope for in Java is
something like this:

        cmpl    $56, size
        jbe     .OUT_OF_BOUNDS_FAILED
        movl    foos, %edx
        movl    52(%edx), %eax

If the Java platform can't handle foos.get(13) efficiently, then you will
get bad performance no matter what solution you dream up.  I don't see how
saying "structs" solves this problem.


Structs can solve the problem partly by doing the bounds check on a struct-by-struct basis, not on a field-by-field basis.

People asking for this are morons. Sorry. The JVM is actually getting better and better about avoiding bounds checks when it can.

I have done serious network programming without the change.  It actually leads to much more understandable and reliable code.

It may be a bit slower in some cases, but in most cases the speed of the network is the limiting factor anyway.

The various buffer classes handle this efficiently, along with JVM optimizations.

Please don't add this to Java. Thankfully the Sun engineers are more than competent and their replies have been right on.

Please don't let ignorant hackers mess up Java.

Anybody who has a slight idea of object orientied programming will vote against this RFE. Anybody else should read any theoretical book on OO to read about the reasons for that. OO languages are intended for OO developers should not be messed up for some hackers' convenience. Even nowadays OO developers still have a hard job in companies to get some hacking colleagues to write solid OO-compliant code. And this would get even worse if the language supports OO-offending constructions explicitely. Would be a big step back. That's exactly why I was in opposition of the idea that Java should get Open Source. Maybe it is time to set up an RFE to be able to explicitely vote against a certain RFE. I am sure with this RFE that the 341 "yes" votes would face thousands of "no" votes.

As Scott Palmer says: "Structs can solve the problem partly by doing the bounds check on a struct-by-struct basis, not on a field-by-field basis."

The @Struct proposal though neat is still going to need some help from the VM to remove bounds checks. If you can do some internal work so that the generated code which accesses the data does no bounds checking that would do the trick; instead the bounds check needs to be made by the setPosition() method of the Struct which positions it in the ByteBuffer upon which it lies.

robert_engels, liebelt: please read up about the Flyweight pattern. There is nothing non-OO about this proposal. There is still an Object, it still behaves like an Object, the only difference is the values of its fields are determined by the contents of an underlying ByteBuffer and its position in that bytebuffer.

Oh one more thing - would greatly prefer not to use any other annotations inside the definition, just plain old Java primitives byte, char, short, int, float, double. Objects are totally ignored.

I'm curious about what people actually want with this RFE. We seem to have requests for:

1. LWO's
2. Objects with no bounds checking.
3. Memory-mapped objects with bounds checking.
4. Memory-mapped objects with bounds checking only at the array level (?)

I see no problem with Sun's suggested implementation using @Struct, with an undelying NIO buffer.

- bcorb

please no. it's an extra complexity to the language for a feature almost never used.
i, myself, could not find feasible use of struct in C#.
ByteBuffer can be sorta cheap wrapped or in the absolutely very worst cast native asm code to address very specific needs. Completely agree that struct alone may not fix the case.

I'm all for structs in the Java language. Classes and objects are inherently expensive to instantiate, use, and clean-up. Especially when they are not exposed! Think of an object like a machine that holds and processes, and a struct as a simple store of data. Also, provide struct inheritance, allowing for multiple inheritance. Data inheritance is very different from object inheritence, and you can get away with a lot more when there are no objects to deal with! The only problem is that the inheritance system in Java relies on methods rather than fields -- how would you work around this?

This RFE should be rejected, structs are useless in a modern OO-language with automatic memory mangement.

Look at 
http://www.geocities.com/csharpfaq/structs.html
and learn why strcts suck in C#.

@tfa99:
Dont try to mislead people with false information. You didnt even read the RFE. Java structs would not be like C# structs. C# structs are stored on stack and their content is copyed on assignement. Java structs would be stored in ByteBuffer, and assignment would copy the index where the struct content begins in the buffer. This is more like structs in C++, and not in C#. 

Structs are needed when working with OpenGL vertex data. Right now every operation on the vertices involves copying values from ByteBuffer, then after calculation the copying of modified data back into the ByteBuffer. This most of the time leads to duplication of the data: data in ByteBuffer ready to be sent to OpenGL, and same data in lots of small objects, suitable for doing calculations. With structs we would not need to duplicate data, and would not need to copy data each time we access them. Data operations could work on the memory wrapped by the ByteBuffer directly, without any added overhead. Tightly-packed data is needed for physics simulation, AI, networking, just to name few. It is needed almost every time when exchanging data with native libraries. The reason for this enhancement is not to make Java a "nicer" language, or more C like, this is needed because of purely technical reasons to perform real-time graphics (and lots of other stuff) better. Please vote for.

struct in C, is a normal class without any method, but without pointers, it's pointless, and with pointers, JVM security is gone.

No, NO, a thousand times No!

Cas speaks the truth. Structs may be very helpful in numerous cases where dealing w/ large amounts of data simply doesn't require the extra object header (and garbage collection).
+1