OSCON, anyone?
Anyone out there going to OSCON? I'm located in Portland, so let me know and maybe we can meet up.
« May 2005 | Main | August 2005 »
July 26, 2005
July 25, 2005
HashMap.get() can cause an infinite loop!
Yes, it is true. HashMap.get() can cause an infinite loop. Everyone I've talked to didn't believe it either, but yet there it is -- right in front of my very eyes. Now, before anyone jumps up and shouts that HashMap isn't synchronized, I want to make it clear that I know that. In fact, here is the paragraph from the JavaDocs:
Note that this implementation is not synchronized. If multiple threads access this map concurrently, and at least one of the threads modifies the map structurally, it must be synchronized externally. (A structural modification is any operation that adds or deletes one or more mappings; merely changing the value associated with a key that an instance already contains is not a structural modification.) This is typically accomplished by synchronizing on some object that naturally encapsulates the map.
Now, I've always taken that standard warning in all the Collections classes as simply stating the obvious: these data structures are synchronized and if you aren't doing your own synchronization you'll get dirty/unreliable/unknown data and/or behavior.
But never in my wildest dreams would I have thought that without proper synchronization that an infinite loop could occur! Let me explain. First, it helps to take a look at the get() implementation in HashMap. Right off the bat it becomes easy to see how an infinite loop could occur:
public Object get(Object key) {
Object k = maskNull(key);
int hash = hash(k);
int i = indexFor(hash, table.length);
Entry e = table[i];
while (true) {
if (e == null)
return e;
if (e.hash == hash && eq(k, e.key))
return e.value;
e = e.next;
}
}
Just seeing that while(true) block simply shocked me! Looking at it a little further, it seems that it is possible for e.next to point back to e (either directly in a tight loop or indirectly in a longer loop), causing an infinite loop and effectively locking up your thread.
I looked at HashMap a bit more and found the only place where e.next is assigned and could possibly cause this problem:
void transfer(Entry[] newTable) {
Entry[] src = table;
int newCapacity = newTable.length;
for (int j = 0; j < src.length; j++) {
Entry e = src[j];
if (e != null) {
src[j] = null;
do {
Entry next = e.next;
int i = indexFor(e.hash, newCapacity);
e.next = newTable[i];
newTable[i] = e;
e = next;
} while (e != null);
}
}
}
This method is called when the internal hash tables are resized. Now, at this point I don't know how it happens, but somehow the table entries get in a state where an element points back to itself, causing the next get() call to loop forever.
Did anyone else know this? Does anyone else think that this is "OK" or "not OK"? Like I said, I always assumed that the JavaDocs warning was in regards to data consistency -- not actual program failure or "lock up". I'm curious to hear what other people think, as I know I was definitely shocked to learn this.
PS: The original reason I didn't synchronize the map was because I really didn't care about data consistency. But I certainly care about avoiding infinite loops, so I've since added some synchronization.
July 18, 2005
Going public
Mark Cuban remembers Broadcast.com's IPO on its 7th aniversary today. I think the most telling quote is this...
Going public wasnt a reason to relax, it was as stressful a moment as I had ever experienced. I was now responsible to people who I had never met. People were buying our stock as a foundation for their future. That was scary.
... and this...
There was no doubt that Todd and I had to get back to work the next day. We had to make it clear that the IPO wasnt a reason to celebrate, but rather a reason to work harder for all of our new shareholders.
I think these words really underscore the mindset that an entrepreneur must have.
July 16, 2005
New toy: 24" Widescreen LCD
So last week I got my new toy (Dell Widescreen UltraSharp 2405FPW) and I'm now pretty much settled down with it. I had a few problems (like turning on features I shouldn't have with my hacked iBook that uses Screen Spanning Doctor), but now everything is running great:
Anyone looking for a new monitor and can afford the $950 price tag (I got mine for $700 via a gift certificate), I cannot recommend this enough. It even comes with various video inputs, including component, which allows me to hook up my XBOX to it and play with picture-in-picture. However, don't buy this with the assumption the video inputs, including HD ones, will look decent. The XBOX signal seems to be hit or miss -- sometimes it looks great and other times it has a very "wavy" picture and is unplayable.
Now, if the XBOX 360 does DVI-D ouput, then we'll really be in business...
David: putting words in my mouth
In the comments of my last post, David said:
"It's great to see that you're warming up to the importance of productivity."
Wait one second -- I never said I was "warning up" to it, nor even implied to it I have been "cool" on it in the past. In fact, productivity has always been the critical factor in everything I do in business, whether it be selecting a language, hiring choices, or development methodologies.
My post never claimed that Ruby is more productive, overall, than using Java. What I acknowledge is that Ruby is more productive for creating some applications, but that Java is more productive for maintaining most applications. And in my job role I maintain software much more than I create it, so Ruby is still not even an option.
Please don't selectively read my post and think that I have "changed my mind" or came to some grand realization that parts of Java weren't as productive as I thought they were. I started my career as a perl and PHP developer, and throughout my career I've always looked for tools (from JDE on Emacs, to IDEA, from web frameworks to bug trackers) that make me more productive. Nor do I pretend that Java is the holy grail for productivity, or fool myself that without IDEA that I'd still be using Java today. However, I am not going to pretend that the world operates the way I wish it would, but rather I will work with the world the way it is and do my best to change it for the better. But to think that Rails has changed my thoughts on productivity is, at best, foolish, or at worst, an insult.
July 14, 2005
It's the tools, stupid!
Patrick Peak had some thoughts on a Rails presentation at the Denver Java Users Group (why is there no Ruby User Group to talk about this?). Namely, he comes to the conclusion that the main benefit Ruby brings to the table over Java is that the development workflow is simply code, alt-tab, reload browser. In Java, it is typically code, compile, package, deploy, alt-tab, reload browser. Dave didn't really like Patrick's thoughts, but I think this is just a case of someone being overly protective (I know I have been in the past with WebWork).
This is very true and, in my opinion, is the heart of the Rails vs J2EE/Java/WebWork/Whatever argument. The key thing is, in fact, not Rails, but Ruby. And that's OK.
Rails benefits...
Before I get in to the different development workflows, I want to mention some of the cool things I've seen from Rails. I've been playing with Rails for the last couple of days (hands on, this time), and it does some excellent things:
- I was worried about configuring Apache with Rails, as I am far from an expert. Much to my surprise, simply invoking scripts/server booted up a mini server ready to develop with. This is comparable to WebWork shipping with Jetty, and ready to run sans external Servlet container. I like it. A lot.
- Gems is just like CPAN and made installing Rails extremely easy. Java needs something like this... that isn't Maven. Ivy could be it... though the usage patterns are slightly different.
- I liked the idea that the rails command creates a fully working environment for you, complete with a makefile (Rakefile), unit tests, and customized scripts to generate more views, models, controllers, and other scaffolding. This is very similar to AppFuse, though much lighter weight and more focused (only one technology stack to serve).
Is it a revolution?
However, nothing in Rails floored me, or otherwise indicated a revolution was taking place in front of me. The design principals are very similar to Struts and WebWork and most other web frameworks. While it is true that it does more implicit configuration, that isn't the key benefit. The key benefit, while partly that, is bigger: the development workflow is several orders of magnitude faster. This, combined with a fairly modern web framework, is the true winning combo. As such, I suppose some might call this pairing a revolution. I would call it an evolution.
Software maintenance
Now, development workflow isn't everything. Code maintenance is by far more important, and Ruby really falls down here, mostly due to the fact that there just aren't excellent tools out there similar to those found in Java. Say what you will about various programming methodologies (test first, etc... remember, I did write a book on TDD), but tools like IDEA make any methodology easier, test-first, test-after, or no-test, it doesn't matter. Some on Slashdot have claimed that there is no need for advanced tools (especially when paired with Extreme Programming or TDD), and that they are simply crutches. While some may be (Xdoclet comes to mind), I really can't see how one could say such things about IDEA.
It's the tools, stupid!
Could someone make a Ruby-enabled IDEA? Sure. If they do, and there are good libraries available (such as Rails), I have no problem using it for my projects. However, I find browsing existing code, quickly seeing the API docs inline, code completion, refactoring support, etc all make the code maintenance much easier in Java. Ruby already has some of those tools (namely, libraries like Rails). But given that I spend most of my time writing code and not doing framework plumbing (as is touted heavily by the Rails crew, and rightfully so), I need my IDE to be of the highest standard. And I need all my coworkers, support engineers, and services engineers who work on the same code to also be using these excellent tools.
Can Java compete?
So the question is: is an IDEA-like IDE for Ruby the only thing keeping it back from conquering the software world? Maybe. But Java isn't as far away as some might think. Sure, Java wasn't developed like Ruby and SmallTalk: dynamically changing class, method, and field signatures isn't currently possible in Java (though HotSwap is close, it isn't good enough). This means that Java currently can't compete with the simplified developer workflow that Ruby offers.
What can the Java tools do?
But that doesn't mean Java is out of the game. Not by a long shot. First and foremost, Sun should recognize the fact that this extended developer workflow is unnecessary and a waste of time. They can fix this by offering a "development mode" in the JVM where full class replacement is possible, even to objects already initialized. Smart defaults, just as Ruby and SmallTalk have, would revitalized a slowing Java community.
Tools like IDEA are doing an excellent job and continue to innovate. IDEA supports HotSwap, meaning I can inject method-level changes (though not method signature changes) at run-time. This has been a huge productivity boost for support and development. Just remember: Ruby has this and more.
While IDEA is doing great, the specifications like J2EE are not keeping pace, and as such the vendors implementing those specs are not keeping up either. Tomcat and Resin, while fast, still take several seconds to start up. Until the JVM allows for full class replacement, quick startup times are critically important.
What can the Java frameworks do?
And speaking of startup times, the app servers aren't the only ones to blame. WebWork takes about 1 to 2 seconds to reload its configuration -- we could shrink this down. Worse, anyone who has used Hibernate can attest to its abysmal startup times. These things must be fixed, regardless of the direction of Sun and other JVM vendors take in the future.
On top of startup times, most frameworks in the Java land should begin to get out of the mindset that code is "deployed" and state can be set up once. If possible, frameworks should embrace a "just in time" execution policy. WebWork did this when Struts did not: Actions are created on every new request rather than being re-used.
Is there anyone we can do NOW?
Lastly, there are things that are happening right now that might make this whole discussion a moot point. With projects like BeanShell and Groovy close to supporting 100% Java syntax, there is potential for Java programmers to have all the development workflow benefits Ruby has while still keeping the excellent tools close to them.
Specifically, I've been playing with teaming BeanShell up with Jetty and configuring BeanShell's classloader to be used as the webapp classloader for Jetty. The result: I can change my code, alt-tab, and refresh the browser. Viola! Changes are there, no compile, no packaging, no nothing. Right now, the changes are only available for objects initialized after the changes, meaning the class-level signature changes don't get injected in to existing objects. However, I'm told by the BeanShell guys that this is possible to do, so there is a lot of excitement to get a working prototype of this.
Remember: as long as BeanShell supports 100% of the Java syntax, you could still use IDEA, refactor all you want, and develop quickly. Then in production, just like in Rails, you'd compile, package, and deploy, locking the release down.
Anything less scary on the menu?
Of course, this is a scary alternative and may take years to get working perfectly. BeanShell surely has bugs it it's implementation that are not in the real JVM. And with dynamic class replacement, more problems could arise. And don't even start to think about all the scary classloading that might happen. Fortunately, there are other options you can use today.
Shockingly, many Java developers (and maybe Rails "fanboys", who are ex-Java developers now) painfully build and test their Java web apps by creating a war and deploying it. If you're doing that: stop, there are better ways. In the simplest form, you can structure your source to essentially be an expanded war, letting you at least edit JSPs on the fly. Most people know about this.
Beyond that, use your debugger and use HotSwap. It is sad that it is such as little known feature. It effectively breaks down some development to at least code, compile, alt-tab, refresh.
On top of that, IDEs like IDEA know how to take your source control that isn't structured like a webapp and copy the files in to an exploded war directory very quickly. The end result is a quick make which breaks down the "compile, package, deploy" steps to simply "make".
Finally, if you're not using an advanced IDE (you should!) and your code isn't in the form of a webapp, there are still options. Namely, Resin and Jetty allow for very advanced configuration of webapps, allowing you to specify JSPs/resources, classes, and libraries from all over the place. Resin even has a very little known feature where you can map two directories to a single webapp path using a "merge:" directive, which is great if you ever extend existing webapps, such as we do in Jive Software's professional services department almost daily.
What's the point?
This definitely has to be one of my longest posts I've ever done. And it should be: this is something I'm very passionate about. Developer productivity is not to be dismissed lightly. Ruby has a leg up on Java right now because it is inherently a better language. But it is far from a better platform right now, which is unfortunate. So the question is: Which will happen first? Will Ruby get the tools, or will Java get its act together and catch up to Ruby? One, if not both, are bound to happen soon, and no matter what the developers will win.
PS: For full disclosure, I've said what some might view as negative comments towards Rails in the past (here and here). Feel free to read them and take them in to context with this article. I hope that I have explained myself even better this time, but my underlying message remains the same: "it's the tools, stupid!"