the ant of performance testing

Much has been made of the recent performance improvements of web browsers – Mozilla, Microsoft, Opera and Google all lining up to show how their browser is the fastest. Similar performance focused shot outs have taken place for databases (Oracle, IBM, Microsoft) or application servers (BEA, Microsoft, IBM, JBoss etc). Probably any software technology has been subjected to some form of performance comparison.

Performance is important – don’t get me wrong – but it is really very subjective and not as objective as the technology vendors imply.

I ran Mozilla’s ant test site with the new firefox 4 beta 7 and the new Google Chrome 8/9. Chrome won hands down. The test ran much faster in Chrome according to the numbers displayed. And the ants do seem to run smoother in Chrome.

But the problem for firefox isn’t its core code implementation, but its integration with the operating system. When Chrome is running the ant test, the window manager ( consumes about 20% of the CPU leaving Chrome to consume the remaining 80%. But Firefox causes the window manager to consume more than 60% leaving only about 40% for the browser itself. Fix the integration and it should in theory be about 40 to 50% faster than Chrome.

But then again, maybe not. Performance projections are very rarely linear, but far more likely to be indeterminable curves riddled with bottlenecks.

The debate will probably die out soon, once we realise that even 100% improvements of a few milliseconds are still just a few milliseconds – but thanks to Google, all browser users win.

You can try the test for yourself here

Leave a Reply

Please log in using one of these methods to post your comment: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s