browser benchmarks.png

Ever since we did a browser comparison test last year there have been a lot of emails and comments asking if we were going to update the article to reflect new releases. I thought about adding in the new browsers as they came out, but decided against it for one reason or another. Instead I thought it would be better to just do a fresh article, and include even more stats than last time.

The main reason that I thought this was worthy of its own article was because a lot has changed in the last year. Since March 2008 we’ve seen major milestone releases from each of big browser makers, and to top it off Google Chrome has come onto the scene. These browsers have also shifted focus from adding nifty new features to diving deep into the code trying to squeeze out every last ounce of performance.

We’ve got a lot in store for you today ranging from JavaScript speed tests to memory usage comparisons, and we’ll even throw in some Acid 3 coverage. Lets go ahead and dive right in.

Notes about testing:

  • All of these tests were performed on the same Windows XP SP3 machine that is wired into a network to eliminate the effects of wireless disturbances.
  • All browsers started with a clean profile and no add-ons/extensions were installed.
  • All browser data, including caches, were cleared before each test was run.
  • Only one browser was open at a time, and no other applications (other than standard XP services) were running.
  • Internet Explorer 8 was used in the native rendering mode (“standards compliant mode”).
  • No plug-ins (Flash, Java, etc…) were installed on the machine to ensure that slow performance wasn’t due to the loading of a plug-in.

–JavaScript Tests–

The main speed test that everyone seems to use for JavaScript is SunSpider. Last year we compared the browsers with the SunSpider test prior to writing our first comparison, and so we wanted to try something different. That’s when we turned to the MooTools SlickSpeed Test. It tests various operations against a lot of common JavaScript libraries including MooTools and JQuery.

So which one did we go with this year? Well, we did both. We ran each test, SunSpider and SlickSpeed, a total of three times each. Then we averaged the results together to get the pretty little graphs you see below. For both of them the goal was for the browser to complete the tests as fast as possible, and so a lower number is better.

Sunspider Test:

sunspider test.png

  1. Safari 4: 603ms
  2. Google Chrome 3.0 Beta: 636ms
  3. Google Chrome 2.0: 720ms
  4. Firefox 3.5: 1278ms
  5. Opera 10 Beta: 2975ms
  6. Opera 9.64: 3931ms
  7. Internet Explorer 8: 5441ms

MooTools SlickSpeed Test:

mootools test.png

  1. Opera 10 Beta: 330ms
  2. Safari 4: 355ms
  3. Opera 9.64: 375ms
  4. Google Chrome 3.0 Beta: 464ms
  5. Firefox 3.5: 580ms
  6. Google Chrome 2.0: 763ms
  7. Internet Explorer 8: 1901ms


–Website Rendering Tests–

I used the same method for testing website load times as I did last year. It’s a website called Numion Stopwatch that uses some fancy JavaScript to monitor when a page has finished loading, and then spits out the amount of time it took to complete.

We used two extremely popular sites for these tests: ESPN and the Wall Street Journal. Each site was loaded up three times in each browser, and then the results were averaged together. Obviously we were targeting which browser could load the websites the fastest, and so a lower number is better:

ESPN Load Time:

espn load test.png

  1. Safari 4: 1.936 seconds
  2. Google Chrome 3.0 Beta: 2.194 seconds
  3. Firefox 3.5: 2.380 seconds
  4. Internet Explorer 8: 2.604 seconds
  5. Opera 10 Beta: 2.605 seconds
  6. Opera 9.64: 2.651 seconds
  7. Google Chrome 2.0: 2.873 seconds

Wall Street Journal Load Time:

wsj load test.png

  1. Google Chrome 3.0 Beta: 1.612 seconds
  2. Opera 10 Beta: 1.989 seconds
  3. Opera 9.64: 2.141 seconds
  4. Safari 4: 2.166 seconds
  5. Google Chrome 2.0: 2.552 seconds
  6. Firefox 3.5: 2.886 seconds
  7. Internet Explorer 8: 3.292 seconds


–Memory Usage Tests–

I’m sure this is what many of you were looking for. As geeks we like to have a lean browser that knows how to handle itself without us having to keep a watchful eye over it. That’s why we ran numerous different tests to see just how well a browser controls its memory usage when loading a decent number of sites, and also whether it’s able to release that memory once you’ve closed the tabs.

Here’s a rundown of the order in which we ran the tests to collect the stats:

  1. We started the browser, and took a memory usage reading.
  2. Loaded 10 predetermined sites in tabs, and took a memory usage reading after all the sites finished loading.
  3. Loaded 15 more predetermined sites in tabs (totaling 25 sites), and took a memory usage reading after all the sites finished loading.
  4. Let the browser sit for 10 minutes with the 25 tabs open, and then took a memory usage reading.
  5. Closed all the tabs except for, which was always the first site opened. Then we took a memory usage reading.

And here are the results. The best browser for each test is highlighted in green, and the worst is highlighted in red.

Startup 10 Sites 25 Sites 25 Sites After 10 Minutes Close Tabs
Firefox 3.5 29.5MB 63.2MB 136.0MB 135.8MB 69.3MB
Google Chrome 2.0 29.2MB 152.8MB 279.9MB 172.4MB 56.9MB
Google Chrome 3.0 Beta 39.5MB 260.9MB 389.4MB 197.6MB 53.7MB
Internet Explorer 8 37.0MB 184.3MB 400.8MB 402.4MB 67.6MB
Opera 9.64 21.3MB 62.2MB 166.4MB 151.6MB 135.9MB
Opera 10 Beta 25.5MB 70.4MB 175.0MB 179.0MB 186.2MB
Safari 4 28.5MB 109.5MB 231.2MB 241.8MB 198.4MB


–Acid 3 Tests–

Last year we also took a look at how the various browsers scored on the Acid 3 test. At the time a Safari nightly build was the closest to perfection by reaching a score of 86 out of 100. Today, however, is a different story. A few browsers can handle the test perfectly, some are very very close, and others (yeah, IE) have some work cut out for themselves.

Note: Click on any of the thumbnails for a full-size rendering.

  1. Safari 4 (100/100) – It gets a perfect score and renders everything correctly.
    safari 4 acid 3.jpg
  2. Opera 10 Beta (100/100) – It gets a perfect score and renders everything correctly.
    opera 10 acid 3.jpg
  3. Google Chrome 2.0 (100/100) – It gets a perfect score, but not all tests are executed successfully.
    Google Chrome 20 Acide 3.jpg
  4. Google Chrome 3.0 Beta (100/100) – It gets a perfect score, but not all tests are executed successfully.
    google chrome 30 acid 3.jpg
  5. Firefox 3.5 (93/100) – It gets a near perfect score.
    Firefox 35 Acid 3.jpg
  6. Opera 9.6 (85/100) – This is the oldest release we tested, and it comes as no surprise that it doesn’t pass the test. It should be noted that the next milestone, version 10, does pass the test perfectly as seen above.
    opera 9 acid 3.jpg
  7. Internet Explorer 8 (20/100) – While they still have a ways to go before they get a perfect, I do have to give them credit for at least making the image look halfway normal. You know what I’m talking about if you remember what IE7’s rendering of the Acid 3 test was like.
    ie8 acid 3.jpg



So which browser is the winner? I wouldn’t really say any of them outshine the others. The problem that we are going to face with performance tests from here on out is that the browsers will all come very close to each other in the standings… often within a fraction of a second from one another. As the browsers continue to get optimized we will see these times get even closer, and performance might become less of a concern which picking which one we want to use. So I’d say to pick the browser you feel the most comfortable with, because it’s getting hard to distinguish one browser from another when it comes to performance.

What’s your take on the stats? Anything stand out to you? Will you be switching browsers based upon anything you learned here?