Firefox 3.6 is out, Chrome's stable version got a big upgrade, and Opera 10.5 is inching toward release. It's a great time for us to break out the timer, process manager, and code tracker for some up-to-date browser speed tests.
These tests are now out of date. As usual, you can see our most recent batch of speed tests at lifehacker.com/browserspeedtests.
This go-round, we did things just slightly different from our last batch of browser tests.
We're still timing the human experience of clicking to start a browser and waiting for it to reach a (locally saved) home page, both "cold" (right after boot-up) and "warm" (after running at least once already), and having it load nine tabs at once, using a millisecond, keyboard-activated timer app from Rob Keir. To test JavaScript speeds and, new to these tests, DOM and CSS querying, we're using Mozilla's Dromaeo online suite, which loads offline and implements both Apple's SunSpider and Google's V8 tests, along with others. To measure memory use, we're using Windows 7's process manager, except in the case of Chrome, which has its own system. The tests are, as ever, performed on my trusty ThinkPad T61p, running a 2.0 GHz Intel Centrino Duo processor and sporting 2 GB of RAM, and currently running Windows 7.
What's new this go-round is, as mentioned, the inclusion of Dromaeo's DOM/CSS tests, as the use of jQuery and other JavaScript-based code will be increasingly important as HTML5 starts changing the web game. We also ran a little side test involving extensions, as many readers (and, truth be told, editors here) have noted that Firefox runs tight on memory when it's brand new, but it's long-term extension use that seems to slow things down. So we loaded up Firefox 3.6, and both the "stable" and "development" versions of Chrome, with extensions found on both browsers:
Xmarks bookmark sync
AdBlock (known as AdBlock Plus on Firefox)
CoolIris photo viewer
Gmail-based mail checkers: Gmail checker for Firefox, Google Mail Checker for Chrome
LastPass password manager
Are these extensions exactly the same between Firefox and Chrome? Not really. Are they generally providing the same functionality? We hope so. Either way, we wanted to see what a fan of these five fairly popular extensions would experience, in terms of memory use, across all three browsers (for Windows) that have a robust extension platform.
On to the tests!
Update: Originally this post referred to Opera's 10.5 release as a beta, but it's actually still a pre-alpha release. We've updated the text; our apologies for the confusion.
Click any of the images below for a clearer and wider look at the results.
Boot-up and warm loading; Winner: Opera!
Opera 10.01 and 10.5 Pre-Alpha are so close together in cold and warm starts, you could write off the difference to the speed of your testing editor's finger on the timer button. Close behind is Chrome's stable and development versions, similarly close in standing. Firefox 3.6 was surprisingly just a tad slower than 3.5 in starting up, at least from first boot-up, and Safari remains a fairly slow starter.
Tab Loading; Winner: Chrome (Stable)!
We were more than a little surprised at this result, so we triple-checked that we'd cleaned out Chrome's cache, cookies, and browsing data to ensure it wasn't getting a head start on loading each browser's home page, plus Gizmodo, Lifehacker, Hulu, and the Google home page. But the numbers bore out the new Chrome release's prowess on the "Open all in bookmarks" function, which bodes well for its individual tab loading:
JavaScript; Winner: Opera 10.5 Pre-Alpha! (Somehow)
Browser developers have told us before that JavaScript engine coders can, intentionally or subconsciously, write their frameworks "to the test," giving them an artificial advantage when placed against the competition in certain tests. And pure runs per second is, as Mozilla's Mike Beltzner put it, like a horsepower rating on a car—it doesn't tell the whole story of its performance.
Still, to look at Opera 10.5 in the Dromaeo aggregate results, you'd think that either Opera has developers who have learned not to sleep, or that something funny is going on. Still, we have to call them the winner, with Chrome's development channel, stable release, and Safari 4 picking up the remaining winner's circle slots.
DOM/CSS; Winner: Chrome (Dev)!
Google's browser mostly walks away with this one. Safari puts in a strong showing, and Firefox pushes past Opera.
Memory use, no extensions; Winner: Firefox 3.6!
For all the emphasis Firefox put on its themes, security, and plugin management for the 3.6 release, they could have mentioned that it's even more efficient with memory, at least soon after start-up, than the already impressive 3.5 releases. Opera 10.01 and Safari 4 aren't far behind, Opera 10.5 shows a pretty hefty jump in tab holding, and if you're using Chrome with multiple tabs, you'd better have a pretty modern system.
Memory use with extensions; Winner: Firefox 3.6!
You know what we learned about Chrome in these tests? That there's a price to pay for its walled garden model of security. It seems like each extension is individually contained in a tab-like shell, meaning that if your extension crashes, it won't bring down your whole browser. Even if it does, your tab contents and writing are preserved. That said, to have those extensions loaded, with or without tabs open, makes a pretty big difference in memory use. Extensions make a difference in Firefox, too, but much more incrementally:
The scores
As we first implemented in our last tests, we took the numeric score placement of each browser in each category and ranked them from 7, as best in category, to 1, as worst. We totaled those numbers up, and present them here as a total out of 35. If a browser had a 35, it would be best in every category: startup, page loading, code handling, and memory use. We didn't include the extension memory test, because it's not a fair fight between all the browsers.
Scores (out of 35 possible)
Google Chrome 4.0.302.3 (dev): 25
Google Chrome 4.0.249.78 (stable): 24
Firefox 3.6: 20
Firefox 3.5.4: 21
Opera 10.5 Pre-Alpha: 25
Opera 10.01: 15
Safari 4.0.4: 19
What did we learn? If you're using Opera 10.01, you have almost no reason not to upgrade. If you're using Chrome, the Stable channel is pretty much a wash with the development version, barring any new features that make their way first to the dev channel (which, granted, some certainly will). Firefox 3.6 seems more of an upgrade in features than core components, and Safari, well, doesn't it have a really neat launch page?
Like our results? Totally disagree? We want to hear about it in the comments.