A while ago I spent a day and a half fretting over a missing checkcast in a Java/JDK file before I finally solved it. Before I finally mostly solved it.
It didn't take long until checkcast returned to hit me again from another angle.
The Java List<E> includes a method called toArray(), which returns the contents of the list as an array. toArray() is older than Java's generics, so it returns Object[] rather than an E[]. This isn't a problem on its own, because implementations of List<E> are free to return an E[].
The next part of the puzzle is ArrayList, which implements toArray() and returns an Object[]. It doesn't have to do that, but the source code uses Object[] for storage rather than E[]. The constructors could have called new E[], they do call new Object[].
ArrayList.toArray() calls Arrays.copyOf(), so the object it returns actually is an Object[]. The third, and critical, part of the puzzle is… is all over the JDK. ArrayList is used all over the JDK, and code equivalent to String[] a = new ArrayList<String>().toArray()
occurs in many places, and of course works with Sunracle's JVMs.
Sometimes the JDK code includes a checkcast (but no exception handling), sometimes there's no checkcast. Either way it works… except for me.
What to do? On one hand this detail is very, very difficult for me, because it's a minor side effect of extremely important invariants in my compiler. On the other, I cannot very well refuse to compile the JDK and its two thousand instances of ArrayList, and right now this issue is making every application fail during startup.
I decided (after writing this and pacing the corridor for a good long while) to make checkcast copy the array, if the array's contents are acceptable to the new type. That seems to be the least bad option I have, but of course it means that == behaves differently from real JVMs, because the cast actually returns a copy.
At present the Object[] and the copy share the same hashCode(). I'm not sure whether that's a good idea.
What is java? In a way, my compiler defines java as the language used to write the twenty thousand classes in the JDK library, Jsoup or whatever else gradle fetches to build a test. While I do read the specification, test-driven development is called test-driven for a reason.
A test drove me into a problem yesterday and my head hurts. I've encountered that problem before but escaped for various reasons, this time I have to confront it. The problem involves a one-line function that just accepts an argument, casts it to a subclass and returns it. Javac has compiled that as aload_1; areturn
, which means push the first argument onto the stack and then return it
. Javac would ordinarily include a checkcast
to make sure that that first argument actually has the function's return type, but didn't in this particular case.
Taken together, this tiny function and its callers convert an Object to more-specific class without type checking. The caller gets an arbitrary Object from a call that returns String, and this is legal java. I'll skip the details of why this is legal, they just make me sad and angry. […More…]
It's likely that my compiler treats foo as though it were just static final
, and if you do not understand how a final volatile
variable differs from a plain final
, then congratulations.
The problem is that although people generally think of static final variables as constants, they aren't quite constant: foo is assigned its value early in the life of the application and can never be reassigned and, but if two or more classes reference each other, so that each class has to be initialised before the other, then code involved in that initialisation may see foo still being 0. This is usually an unpleasant surprise, found while debugging. volatile
affects the visibility of the foo=1
assignment, and it probably forbids one of the things my compiler does to handle those loops.
I can fix it and I probably will, although I will not be highly motivated: It's a minor problem, and quite frankly, people whose code depends on the semantics of static final volatile
ought to go for a walk in the park and reconsider that aspect of their design.
Today I need to write a benchmark.
I have decided to reimplement HTML Tidy in a simple, approximate way. I chose this because:
Of course I realise that this won't be a very good as a benchmark. I chose it because I believe that Jsoup is the kind of code I need to handle well. If my code doesn't do well on Jsoup, then I need to fix it. I don't have the same feeling about specint and other well-known benchmarks. They try to be good, precise benchmarks, and they don't explicitly try to be typical java.
As a result this benchmark will be useful for me, and the cost of that is that its measured performance is imprecise for you.
I've been working on something that, I think, ought to perform 30%-200% better than the current solutions, depending on the workload and how the measurement mixes values for average/typical/best-case/worst-case time and space. Some workloads might get more than 200% improvement, and I hope less than 30% will be rare.
The very first time I tried to measure the entire system, however, my measured result wasn't 30% improvement or 200%, it was a little over 15000%. Three zeroes. And I wasn't even trying to make a misleading benchmark — I just wanted to measure how well my code worked and I suppose I unconsciously concentrated on parts where the differences should show up clearly.
15000% certainly is a clear difference.
It's also totally meaningless. It measures something you'll never, ever do. However, it's a fantastically big number, I know that I was honest, and it has taught me that even if benchmark results are laughably unrealistic, it's not always because someone tried to brag or mislead. Maybe they are just myopic. Focused on the details of their work.
I've spoken harsh words about other people's benchmarks in the past. I don't think I will do that any more.
Update: 1500000‱.
Update: And the best way to represent it is with a logarithmic bar graph. Most people do not really understand a log scale, but the difference still looks very large and so the meaning is preserved. The label on the y axis makes it formally correct.
This is the simplest possible java main(): static int main(String[] argv){return 0;}
. How many classes, constructors and methods does it require?
The simple answer, of course, is the String constructor and the String[] constructor, so that's two functions.
But it also requires whatever those two functions require, which is where the fun begins. […More…]
A year ago I talked at some length and frequency about the evils of Microsoft's reference application for the Xbox. One of the points I mentioned most often is that the thing links in four different JSON libraries, all deficient in some serious manner.
Today I added a third JSON library to an application, despite knowing that it already used two different ones.
Modula 3 is perhaps my favourite language. It has (had — it's practically dead now) most of what I like about java, most of what I like about c++, most of what I like about modula 2, and some unique features.
One of its little greatnesses is in the integer type system.
In modula 3, an unsigned integer is 31 or 63 bits long (as I recall, there are two unsigned integer types, although a tutorial I found on the web now mentions only one). Signed integers are 32 or 64 bits, so if a is a signed integer, a=b always works, regardless of whether b is signed or unsigned.
a=b does not throw exceptions, a=b; b=a does not change the value of b, and the cost of that is merely that you have to start using 64-bit variables at 2147483648 instead of at 4294967296.
Update: I want to expand on that, and compare it to java. The language designers of both java and modula3 understood that confusion or sloppiness with regard to signed and unsigned integers was a significant source of bugs in c/c++. Java solved it by not having unsigned integers, modula3 solved it by reducing their bit width by 1.5-3%.
I have seen many java programs that either output long to formats or protocols where only unsigned numbers are legal, or that read into long when the format clearly says 64 bits unsigned
, so I think the java solution isn't very good. They chopped off the problematic feature and instead people use a misfit type. Sometimes it works: In this example it likely works because the sender too, uses a java long, so the so-called 64 bits unsigned ID is really 63-bit. I am not sure whether this kind of bug is preferable to the kind of signed/unsigned bugs in classic c.
Modula3, on the other hand, made the less obvious choice of leaving one bit at zero. The CPU registers are 32 or 64 bits wide, modula3 restrains programs to using 31 or 63 bits. As a result, programmers can still express the unsigned nature of many numbers using the type system, without sign problems. Subtle, well-considered, 97% good.
Archiveopteryx git head crashes a bit. Not every day, but some people reports that it crashes every week or month, at random times. Clearly there is a bug. Abhijit and I have discussed it and found a way to contain it, and I've written the code.
But I haven't found a way to push the fix to the master tree. I seem unable to commit and push that code. My soul wants to find the bug and fix it, not contain it.
Meanwhile, I had an appointment with the dentist this morning.
In the waiting room I read a fascinating blog post about a Chromium exploit. Sergey Glazunov, clearly an admirably clever hacker, stitched together fourteen bugs, quirks and missed hardening opportunities
to form a critical exploit. The bugtracking information for one of the bugs shows that it was reported, discussed for a few days, then it was idle until Sergey leveraged it, and then it was fixed.
Chromium is a nice browser, and I appreciate the hardening and exploit resistance the team has added. I particularly appreciate the team's honesty: They run their pwnium contests and are frank about the results.
But now I am even less happy about making fault tolerant code. I feel that it may be mentally difficult to make a program tolerate faults and at the same time make a programmer not tolerate faults.
Consider the function Message::acceptableBoundary(). That function's reading order is exactly the same as the its execution order. This is not unusual in C and C++ (and more or less in Java), but there is a significant exception, for(). […More…]
Some Java book I read long ago, I think it was Thinking in Java, explains that one of the benefits of Java exceptions is that you can shift error handling away from the normal path, leaving the implementation of the common case clearer and better.
Fine. There's just one catch: You have to catch the exceptions and handle the error somewhere. […More…]
jam -g is the best make system I've ever used. Best for the simple reason that when the build fails, it usually fails quickly. I start the build, and a second later I'm already looking at my mistake. That feature outweighs any and all drawbacks.
Sadly, I don't use jam very often at the moment. I mostly use maven 2, which starts the build by determining from first principles which source files to use and which libraries to download. In practice the set needed hasn't changed in the past minutes, […More…]
Javadoc is built in to Java, but I think they botched it. It's clear that they didn't care deeply: JLS3 grammar doesn't mention javadoc at all, and the JLS doesn't specify it, hardly even mentions it… the word stepchild
sounds more appropriate than does builtin
, in my oh-so-humble opinion.
There are many things I don't like about the result, and few things I do like.
There's too much typing for not enough benefit. […More…]
Exceptions are such a pain. They look as if someone thought they could solve many disparate problems with one tool, and in the end, the tool doesn't look terribly elegant, and people don't use it terribly well. […More…]