Betanews tested the latest available development and stable builds of all five brands of Web browser, on all three modern Windows platforms — XP SP3, Vista SP2, and Windows 7 RTM. Once again, we threw the kitchen sink at them: our new and stronger performance benchmark suite, consisting of experiments in all facets of rendering, mathematics, control, and even geometry. Like before, we use a slow browser (Microsoft Internet Explorer 7, the previous version, running in Windows Vista SP2) as our 1.00 baseline, and we produce performance indices representing browsers’ relative speed compared to IE7.
Averaging all three Windows platforms, the latest Beta 1 preview of Firefox 3.6 posts a Betanews CRPI score of 9.02 — just better than nine times the performance of IE7. Compare this against a 7.34 score for the latest stable version of Firefox 3.5.3, and 7.81 for the latest daily build of the Firefox 3.5.4 bug fix.
When looking at the results broken down by platform, we’ve noticed that in the CRPI scores for Firefox’s development builds, moving up through the latest Alpha of version 3.7, the Windows 7 score has been pulling closer to the XP score. Typically, XP has been the fastest platform by a noticeable margin; that hasn’t been the case in the latest builds. This trend has definitely permeated Opera 10, whose stable and development builds on XP and Win7 are almost dead-even.
Still, Firefox 3.6’s score on XP crossed over the 10 mark on our new index, with a 10.44 on that platform versus 9.09 for the current stable version 3.5.3, and 7.38 for version 3.6 on Vista.
For our long-time readers who watched browsers’ performance scores before we added new tests to our comprehensive suite: You’ll recall how during the development of Firefox 3.5, test builds flirted with a 10.00 score on the old index in Windows XP SP3, and only occasionally crossed that mark. Right now, using the old scoring system, Firefox 3.6 Beta 1 is scoring a 13.59 on XP — moving into the territory that Google Chrome blazed when it first hit the scene.
For all its forward strides, the pace of Firefox’s acceleration is finally approaching that of Chrome. That is, Firefox is getting faster at the rate Chrome is getting faster. Of course, Chrome’s acceleration has been, and continues to be, meteoric, absorbing about one IE7’s worth of performance in its development builds every two months. Having posted a CRPI score of 15.37 just one month ago, the latest stable build of Chrome 220.127.116.11 posted a score of 15.85 this week. (In strange but typical Google fashion, build 195.25 was posted to both the stable and beta channels simultaneously, for reasons known only to Google at the moment.)
While the Chrome 4 development builds had been lagging behind Chrome 3 somewhat, they’re catching up nicely now with a CRPI score of 15.35. On Windows 7, Chrome 3 and Chrome 4 are almost dead even in performance (15.27 versus 15.23), and only two tenths of a point separate the stable and dev builds on Vista.
Next: Performance breakdowns by individual test…
Once again, our rendering test shows that Google Chrome 3 appears to have been intentionally developed with Windows XP in mind — the version of Windows many believe best suits netbooks, which is where Google is still seeking to break through. But Chrome 4 does not follow suit; on average, it actually falls behind the latest stable Apple Safari 4 build.
The Celtic Kane test is the one that last year called out Opera as the “fastest browser,” much faster than Firefox. If the CK test alone designated speed, then Firefox would actually be the champion over even Chrome. Firefox tends to be faster at simpler tests; when the loads get tougher, Firefox gets weaker and Chrome and Safari march right on.
Acid3 compliance scores are unchanged for the last several months. We may not see Mozilla attempt to close that last standards compliance gap until the first daily builds of Firefox 4, which may not come until next year.
Chrome loves heavier workloads; and here in the SunSpider battery, we see it surging ahead, better than doubling Firefox’s scores. The dev build of Chrome 4 has closed the gap here somewhat against the stable/beta Chrome 3.
Next: The four new tests in our CRPI battery…
We found the TestCube geometry test, which was created to address a bug that a Mozilla developer had discovered in Firefox. After we were told there were several areas we hadn’t discovered where Opera is clearly the faster browser, we found our readers to be correct on that count, at least with respect to DHTML graphics rendering. Here not only is Opera the clear leader, but Firefox 3.6 Beta 1 is right on the heels of Chrome 3.
When a Web page doesn’t use CSS to divide its elements, it uses old-style <TABLE> elements. And just because developers…deprecate their use doesn’t mean they’re not everywhere. In concentrating on developing for newer standards, Firefox actually performs relatively slowly in the table handling category, though the standards-compliant Chrome handles old-style tables with ease. What’s amazing here, though, is the gap between performance scores in this battery for both stable/beta and dev editions of Chrome, on Windows 7 (12.60) and Vista (6.69). Chrome is also faster on this test on Win7 than on XP, which is unusual given its usual preference for XP.