Ever since we did a browser comparison test last year there have been a lot of emails and comments asking if we were going to update the article to reflect new releases. I thought about adding in the new browsers as they came out, but decided against it for one reason or another. Instead I thought it would be better to just do a fresh article, and include even more stats than last time.
The main reason that I thought this was worthy of its own article was because a lot has changed in the last year. Since March 2008 we’ve seen major milestone releases from each of big browser makers, and to top it off Google Chrome has come onto the scene. These browsers have also shifted focus from adding nifty new features to diving deep into the code trying to squeeze out every last ounce of performance.
Notes about testing:
- All of these tests were performed on the same Windows XP SP3 machine that is wired into a network to eliminate the effects of wireless disturbances.
- All browsers started with a clean profile and no add-ons/extensions were installed.
- All browser data, including caches, were cleared before each test was run.
- Only one browser was open at a time, and no other applications (other than standard XP services) were running.
- Internet Explorer 8 was used in the native rendering mode (“standards compliant mode”).
- No plug-ins (Flash, Java, etc…) were installed on the machine to ensure that slow performance wasn’t due to the loading of a plug-in.
So which one did we go with this year? Well, we did both. We ran each test, SunSpider and SlickSpeed, a total of three times each. Then we averaged the results together to get the pretty little graphs you see below. For both of them the goal was for the browser to complete the tests as fast as possible, and so a lower number is better.
- Safari 4: 603ms
- Google Chrome 3.0 Beta: 636ms
- Google Chrome 2.0: 720ms
- Firefox 3.5: 1278ms
- Opera 10 Beta: 2975ms
- Opera 9.64: 3931ms
- Internet Explorer 8: 5441ms
MooTools SlickSpeed Test:
- Opera 10 Beta: 330ms
- Safari 4: 355ms
- Opera 9.64: 375ms
- Google Chrome 3.0 Beta: 464ms
- Firefox 3.5: 580ms
- Google Chrome 2.0: 763ms
- Internet Explorer 8: 1901ms
–Website Rendering Tests–
We used two extremely popular sites for these tests: ESPN and the Wall Street Journal. Each site was loaded up three times in each browser, and then the results were averaged together. Obviously we were targeting which browser could load the websites the fastest, and so a lower number is better:
ESPN Load Time:
- Safari 4: 1.936 seconds
- Google Chrome 3.0 Beta: 2.194 seconds
- Firefox 3.5: 2.380 seconds
- Internet Explorer 8: 2.604 seconds
- Opera 10 Beta: 2.605 seconds
- Opera 9.64: 2.651 seconds
- Google Chrome 2.0: 2.873 seconds
Wall Street Journal Load Time:
- Google Chrome 3.0 Beta: 1.612 seconds
- Opera 10 Beta: 1.989 seconds
- Opera 9.64: 2.141 seconds
- Safari 4: 2.166 seconds
- Google Chrome 2.0: 2.552 seconds
- Firefox 3.5: 2.886 seconds
- Internet Explorer 8: 3.292 seconds
–Memory Usage Tests–
I’m sure this is what many of you were looking for. As geeks we like to have a lean browser that knows how to handle itself without us having to keep a watchful eye over it. That’s why we ran numerous different tests to see just how well a browser controls its memory usage when loading a decent number of sites, and also whether it’s able to release that memory once you’ve closed the tabs.
Here’s a rundown of the order in which we ran the tests to collect the stats:
- We started the browser, and took a memory usage reading.
- Loaded 10 predetermined sites in tabs, and took a memory usage reading after all the sites finished loading.
- Loaded 15 more predetermined sites in tabs (totaling 25 sites), and took a memory usage reading after all the sites finished loading.
- Let the browser sit for 10 minutes with the 25 tabs open, and then took a memory usage reading.
- Closed all the tabs except for Google.com, which was always the first site opened. Then we took a memory usage reading.
And here are the results. The best browser for each test is highlighted in green, and the worst is highlighted in red.
|Startup||10 Sites||25 Sites||25 Sites After 10 Minutes||Close Tabs|
|Google Chrome 2.0||29.2MB||152.8MB||279.9MB||172.4MB||56.9MB|
|Google Chrome 3.0 Beta||39.5MB||260.9MB||389.4MB||197.6MB||53.7MB|
|Internet Explorer 8||37.0MB||184.3MB||400.8MB||402.4MB||67.6MB|
|Opera 10 Beta||25.5MB||70.4MB||175.0MB||179.0MB||186.2MB|
–Acid 3 Tests–
Last year we also took a look at how the various browsers scored on the Acid 3 test. At the time a Safari nightly build was the closest to perfection by reaching a score of 86 out of 100. Today, however, is a different story. A few browsers can handle the test perfectly, some are very very close, and others (yeah, IE) have some work cut out for themselves.
Note: Click on any of the thumbnails for a full-size rendering.
- Safari 4 (100/100) – It gets a perfect score and renders everything correctly.
- Opera 10 Beta (100/100) – It gets a perfect score and renders everything correctly.
- Google Chrome 2.0 (100/100) – It gets a perfect score, but not all tests are executed successfully.
- Google Chrome 3.0 Beta (100/100) – It gets a perfect score, but not all tests are executed successfully.
- Firefox 3.5 (93/100) – It gets a near perfect score.
- Opera 9.6 (85/100) – This is the oldest release we tested, and it comes as no surprise that it doesn’t pass the test. It should be noted that the next milestone, version 10, does pass the test perfectly as seen above.
- Internet Explorer 8 (20/100) – While they still have a ways to go before they get a perfect, I do have to give them credit for at least making the image look halfway normal. You know what I’m talking about if you remember what IE7’s rendering of the Acid 3 test was like.
So which browser is the winner? I wouldn’t really say any of them outshine the others. The problem that we are going to face with performance tests from here on out is that the browsers will all come very close to each other in the standings… often within a fraction of a second from one another. As the browsers continue to get optimized we will see these times get even closer, and performance might become less of a concern which picking which one we want to use. So I’d say to pick the browser you feel the most comfortable with, because it’s getting hard to distinguish one browser from another when it comes to performance.
What’s your take on the stats? Anything stand out to you? Will you be switching browsers based upon anything you learned here?