browser benchmarks.png

Ever since we did a browser comparison test last year there have been a lot of emails and comments asking if we were going to update the article to reflect new releases. I thought about adding in the new browsers as they came out, but decided against it for one reason or another. Instead I thought it would be better to just do a fresh article, and include even more stats than last time.

The main reason that I thought this was worthy of its own article was because a lot has changed in the last year. Since March 2008 we’ve seen major milestone releases from each of big browser makers, and to top it off Google Chrome has come onto the scene. These browsers have also shifted focus from adding nifty new features to diving deep into the code trying to squeeze out every last ounce of performance.

We’ve got a lot in store for you today ranging from JavaScript speed tests to memory usage comparisons, and we’ll even throw in some Acid 3 coverage. Lets go ahead and dive right in.

Notes about testing:

  • All of these tests were performed on the same Windows XP SP3 machine that is wired into a network to eliminate the effects of wireless disturbances.
  • All browsers started with a clean profile and no add-ons/extensions were installed.
  • All browser data, including caches, were cleared before each test was run.
  • Only one browser was open at a time, and no other applications (other than standard XP services) were running.
  • Internet Explorer 8 was used in the native rendering mode (“standards compliant mode”).
  • No plug-ins (Flash, Java, etc…) were installed on the machine to ensure that slow performance wasn’t due to the loading of a plug-in.

–JavaScript Tests–

The main speed test that everyone seems to use for JavaScript is SunSpider. Last year we compared the browsers with the SunSpider test prior to writing our first comparison, and so we wanted to try something different. That’s when we turned to the MooTools SlickSpeed Test. It tests various operations against a lot of common JavaScript libraries including MooTools and JQuery.

So which one did we go with this year? Well, we did both. We ran each test, SunSpider and SlickSpeed, a total of three times each. Then we averaged the results together to get the pretty little graphs you see below. For both of them the goal was for the browser to complete the tests as fast as possible, and so a lower number is better.

Sunspider Test:

sunspider test.png

  1. Safari 4: 603ms
  2. Google Chrome 3.0 Beta: 636ms
  3. Google Chrome 2.0: 720ms
  4. Firefox 3.5: 1278ms
  5. Opera 10 Beta: 2975ms
  6. Opera 9.64: 3931ms
  7. Internet Explorer 8: 5441ms

MooTools SlickSpeed Test:

mootools test.png

  1. Opera 10 Beta: 330ms
  2. Safari 4: 355ms
  3. Opera 9.64: 375ms
  4. Google Chrome 3.0 Beta: 464ms
  5. Firefox 3.5: 580ms
  6. Google Chrome 2.0: 763ms
  7. Internet Explorer 8: 1901ms

  

–Website Rendering Tests–

I used the same method for testing website load times as I did last year. It’s a website called Numion Stopwatch that uses some fancy JavaScript to monitor when a page has finished loading, and then spits out the amount of time it took to complete.

We used two extremely popular sites for these tests: ESPN and the Wall Street Journal. Each site was loaded up three times in each browser, and then the results were averaged together. Obviously we were targeting which browser could load the websites the fastest, and so a lower number is better:

ESPN Load Time:

espn load test.png

  1. Safari 4: 1.936 seconds
  2. Google Chrome 3.0 Beta: 2.194 seconds
  3. Firefox 3.5: 2.380 seconds
  4. Internet Explorer 8: 2.604 seconds
  5. Opera 10 Beta: 2.605 seconds
  6. Opera 9.64: 2.651 seconds
  7. Google Chrome 2.0: 2.873 seconds

Wall Street Journal Load Time:

wsj load test.png

  1. Google Chrome 3.0 Beta: 1.612 seconds
  2. Opera 10 Beta: 1.989 seconds
  3. Opera 9.64: 2.141 seconds
  4. Safari 4: 2.166 seconds
  5. Google Chrome 2.0: 2.552 seconds
  6. Firefox 3.5: 2.886 seconds
  7. Internet Explorer 8: 3.292 seconds

  

–Memory Usage Tests–

I’m sure this is what many of you were looking for. As geeks we like to have a lean browser that knows how to handle itself without us having to keep a watchful eye over it. That’s why we ran numerous different tests to see just how well a browser controls its memory usage when loading a decent number of sites, and also whether it’s able to release that memory once you’ve closed the tabs.

Here’s a rundown of the order in which we ran the tests to collect the stats:

  1. We started the browser, and took a memory usage reading.
  2. Loaded 10 predetermined sites in tabs, and took a memory usage reading after all the sites finished loading.
  3. Loaded 15 more predetermined sites in tabs (totaling 25 sites), and took a memory usage reading after all the sites finished loading.
  4. Let the browser sit for 10 minutes with the 25 tabs open, and then took a memory usage reading.
  5. Closed all the tabs except for Google.com, which was always the first site opened. Then we took a memory usage reading.

And here are the results. The best browser for each test is highlighted in green, and the worst is highlighted in red.

Startup10 Sites25 Sites25 Sites After 10 MinutesClose Tabs
Firefox 3.529.5MB63.2MB136.0MB135.8MB69.3MB
Google Chrome 2.029.2MB152.8MB279.9MB172.4MB56.9MB
Google Chrome 3.0 Beta39.5MB260.9MB389.4MB197.6MB53.7MB
Internet Explorer 837.0MB184.3MB400.8MB402.4MB67.6MB
Opera 9.6421.3MB62.2MB166.4MB151.6MB135.9MB
Opera 10 Beta25.5MB70.4MB175.0MB179.0MB186.2MB
Safari 428.5MB109.5MB231.2MB241.8MB198.4MB

  

–Acid 3 Tests–

Last year we also took a look at how the various browsers scored on the Acid 3 test. At the time a Safari nightly build was the closest to perfection by reaching a score of 86 out of 100. Today, however, is a different story. A few browsers can handle the test perfectly, some are very very close, and others (yeah, IE) have some work cut out for themselves.

Note: Click on any of the thumbnails for a full-size rendering.

  1. Safari 4 (100/100) – It gets a perfect score and renders everything correctly.
    safari 4 acid 3.jpg
  2. Opera 10 Beta (100/100) – It gets a perfect score and renders everything correctly.
    opera 10 acid 3.jpg
  3. Google Chrome 2.0 (100/100) – It gets a perfect score, but not all tests are executed successfully.
    Google Chrome 20 Acide 3.jpg
  4. Google Chrome 3.0 Beta (100/100) – It gets a perfect score, but not all tests are executed successfully.
    google chrome 30 acid 3.jpg
  5. Firefox 3.5 (93/100) – It gets a near perfect score.
    Firefox 35 Acid 3.jpg
  6. Opera 9.6 (85/100) – This is the oldest release we tested, and it comes as no surprise that it doesn’t pass the test. It should be noted that the next milestone, version 10, does pass the test perfectly as seen above.
    opera 9 acid 3.jpg
  7. Internet Explorer 8 (20/100) – While they still have a ways to go before they get a perfect, I do have to give them credit for at least making the image look halfway normal. You know what I’m talking about if you remember what IE7′s rendering of the Acid 3 test was like.
    ie8 acid 3.jpg

  

–Conclusion–

So which browser is the winner? I wouldn’t really say any of them outshine the others. The problem that we are going to face with performance tests from here on out is that the browsers will all come very close to each other in the standings… often within a fraction of a second from one another. As the browsers continue to get optimized we will see these times get even closer, and performance might become less of a concern which picking which one we want to use. So I’d say to pick the browser you feel the most comfortable with, because it’s getting hard to distinguish one browser from another when it comes to performance.

What’s your take on the stats? Anything stand out to you? Will you be switching browsers based upon anything you learned here?

There Are 50 Comments

  1. I wish FX really did perform like this when it comes to mem consumption and js, but even with a few extensions installed these numbers are way off, and who uses it without any extensions really?

    • I wish they would have included Firefox 2. I know its outdated but I still use it because it doesn’t have all the add ons and visual aspects fire fox 3 has which makes it use a lot more memory.

  2. Thank you for a very professional job. I enjoyed the post. For me, one things stand out and it’s that Microsoft can’t put its act together.

    Even though Microsoft has been in the field for a long time, being a pioneer in web browser, it’s lagging far behind all the rest. No wonder they are losing market share every month.

    I can see Apple is really pushing its browser as well as Mozilla, Google and Opera. I love it!

  3. Totally agree Omar. It seems it doesn’t really matter anymore which browser you pick (can just pick the one you feel most comfortable with), with one exception: Internet Explorer.

  4. My favorite has always been Firefox, but only for the add-ons. Mozilla should be doing similar tests for it’s most popular add-ons and posting the scores. (They should do it because they can get more accurate scores)

    Great article!

  5. Nice job but I have question. In memory usage tests which memory counter was used? Working Set (this is what in XP taskmgr shows and this is (symplyfying) the sum of private and shared with other processes physical memory), Private Working Set(this is what shows taskmgr in Vista) or Private Bytes(shows all commited by process memory not only not paged out by system)?

  6. I’m sure memory use can affect some systems (i.e netbooks), but with my laptop running 3GB and my desktop 8GB of RAM, I don’t really place that much emphasis on memory usage. Whats more important to me is speed and extensibility. Firefox 3.5 is fast enough for me, and has all the extensions my heart desires.

    • Must be nice to be able to afford any hardware one desires. Not all of us have that option, and, therefore, have to make due with older hardware.

  7. What? IE8 is left in the dust? Well at least it’s got Security, Privacy, and Ease of Use on its side:

    [cybernetnews.com]

    Maybe M$ should add “Last in Every Benchmark Test” and “Uses Over Twice the RAM of any other browser” categories to its Get the Facts (read: Propaganda) chart.

  8. Totally agree with Anonymous up there. Sure I would like to have Firefox with its extensions if it was what Change said, but after using Chrome makes Firefox feel slow motion.

  9. in my opinion, it’s all about what you need.
    i use firefox cause of it’s great basic functions, mainly the Tags and the add-ons.
    i use Firefox’s Prism for stuff like Gmail. hopefully it will get addons too, so things Like Google Gears, Greasemonkey, adblocks would work also.

    i rather wait a bit extra time for all of that.
    if you don’t need this stuff. i think something minimal and fast, like Chrome or Safari would do the trick. especially as a Prism replacement.

  10. I agree that its becoming very much down to personal preference and each browser is chosing its own way to get you to use it – Firefox are persisting making Extensions better and easier to find, Google are pushing the simplicity and developer tools, Opera are doing crazy new things like Unite and Safari sits there so Mac fanboys can stroke it.

    With CSS3 and a lot of advanced new JavaScript I just hope that people clear off IE sharpish so we can start using these great new features reliably.

  11. Anonymous wrote:
    I wish FX really did perform like this when it comes to mem consumption and js, but even with a few extensions installed these numbers are way off, and who uses it without any extensions really?

    I know more people who use Firefox without extensions than those who do. Many just download it because they hear about the browser from a friend that never explains the usefulness of extensions.

    Omar Upegui wrote:
    Even though Microsoft has been in the field for a long time, being a pioneer in web browser, it’s lagging far behind all the rest. No wonder they are losing market share every month.

    I would like to think that IE will eventually become a really good browser, but Microsoft has always had one primary focus: backwards compatibility. I complain about IE all the time, but I would hate to be in Microsoft’s shoes where they try to walk the fine line of compatibility versus standards. I guess it would have been good if they followed standards back when they were originally developing IE, but in may respects IE became the standard. It wasn’t until Firefox that everyone started to realize browsers needed to be more universally compatible with websites.

    pb2004 wrote:
    Nice job but I have question. In memory usage tests which memory counter was used? Working Set (this is what in XP taskmgr shows and this is (symplyfying) the sum of private and shared with other processes physical memory), Private Working Set(this is what shows taskmgr in Vista) or Private Bytes(shows all commited by process memory not only not paged out by system)?

    I used the working set memory (and we’ve actually written an article talking about the differences in memory measurement [cybernetnews.com] )

    Ben wrote:
    I’m sure memory use can affect some systems (i.e netbooks), but with my laptop running 3GB and my desktop 8GB of RAM, I don’t really place that much emphasis on memory usage.

    That’s what a lot of people are starting to say now as well. The only cause for concern that I have with that is we may start to get to forgiving with memory usage, and then developers won’t worry as much about it. And even if you have large amounts of memory there are always memory leaks that can plague your system. I once saw my Firefox 2 reach 1.6GB of memory usage.

    Jack of all Trades wrote:
    in my opinion, it’s all about what you need

    Exactly, and what I’ve seen becoming more and more common are users who are running more than one browser on their machine. Sometimes they separate out work in one browser, and personal stuff in another.

  12. How did you test Opera? Opera 10 has by default set Memory Cache to Automatic, meaning it will cache a lot and free up memory less on tab close. Technically speaking it releases cached closed tabs memory only when some other application asks for the memory and the system has no memory for it. I don’t know if this is good or bad, but I see it is a good thing, as when you revisit pages you visited within the same session it load lot faster. That’s why you have on Opera 10 Closed Tab memory test result 186MB memory. If you set the memory cache to a specific size instead of Automatic it will try to keep the cache in that amount … or if you have a system with a small amount of RAM (i.e. 512MB or less) stopping memory cache will decrease memory usage a lot.

    • I just straight-up tested the defaults. I didn’t do anything fancy and didn’t mess with any settings. I’m pretty sure Opera isn’t the only one that will release memory when other apps need it.

  13. In my opinion, the most important is the RAM usage on the long run.
    So I’m happy to see that Firefox 3.5 performs the best in that category.

  14. seems ie8 is headed nowhere

  15. I’m a diehard firefox user but lately I find myself using safari more and more.

    I say no to ie.

  16. :D :D

    Firefox is the best for me…

  17. IE8 is not that bad… dont trust this fanboi bullshit M$ bashing shit o_O

    • If you’re referring to my tests I wasn’t taking any sort of sides. Heck, IE8 is the browser I use the most since that’s what I use at work. So I would have liked to see it do awesome in these tests.

  18. The most important measure by far for me is the time a browser takes to start up. I switched from Firefox 2 to Chrome because when I started Firefox up I think I notice my lounge lights dimming. Inevitable consequence is you end up leaving it on all the time, gobbling resources. Chrome on the other hand is almost instant.

    I think the philosphy of Firefox is to keep cramming in features – its how they compete with the incumbent who sits largely preloaded in the OS, which is fine but they should be lazy-loaded.

  19. PS. Much as anyone who has ever made a webpage would love it to be true, IE is not headed nowhere – still the most common browser by far, more’s the pity.

  20. Hey Ryan, I’m curious how exactly you measure memory usage for Google Chrome. I’d assume you used Chrome’s about:memory, rather than trying to add every chrome.exe process. However, the about:memory page’s table lists MEMORY: Private, Shared, and Total(sum of private and shared) as well as VIRTUAL MEMORY: Private, and Mapped. I’m curious, which values did you add for your memory usage values (or did you use a different memory measurement method altogether)?

    • Ha, no. I didn’t use Chrome’s about:memory for the very reason you talked about. I did add up all the individual processes (yeah, it got tedious). I had to do the same thing for IE8 since it also isolates its processes.

  21. i would switch from firefox to safari if there was an ad blocker for windows safari users.

  22. I use FF for SEO & web development purposes but I find using 5 – 10 tabs with 5 or so plugins makes my PC really sluggish. Memory usage is about 500mb. All in all I use firefox for work & Chrome for personal browsing! Both work well for me!

  23. I like FireFox because of add-ons and web development features, not speed. If I need something fast I’ll use chrome.

  24. Being a Firefox 3.5 user I question the website load speed. I’ve heard that Safari (or is it Opera) doesn’t have add-ons. In FX I use Addblock Plus & Element Hiding Helper which without the ads should cause websites to load much faster.

    Perhaps we need another test using various common add-ons in all the browsers.

    • The browser you’re thinking of is Opera, and such a test would be irrelevant as Opera has features from AdBlock+ built in as standard, which is the reason they don’t do add-ons – as they believe integration is superior to extension. Most of the most popular FF add-ons are built-in to Opera.

  25. I have tried Safari, IE, Chrome, FF, and Opera, and I keep going back to FF. It gives me the right balance of speed, functionality, looks, and personalization (is that a real word?). The only thing I don’t like at this moment is the new logo, and I don’t even know why.

    • exactly, i love firefox for its simplicity and versatility and my university provides a 360mbps wlan connection(n channel) so i see virtually no difference in page loading time and speed.

  26. Good job Ryan, this is very informative especially to people like me who are web developer!

  27. C wrote:
    i would switch from firefox to safari if there was an ad blocker for windows safari users.

    There is: GlimmerBlocker on OSX and AdMuncher on Windows (they actually work for all browsers you have installed).

  28. Well Firefox is Firefox for a reason. Maybe some you guys rather use Safary; you must have a very good and personal reason for that. For research firefox has been and continue to be, hands down my favorite, and I know most savvy and webmasters would agree with me.

  29. i am a windows user but inlove with apple(can`t afford one) and i tried everything from ie to safari an firefox is in the #1 spot for me…speed…low mem consumption add`ons and many more.firefox is the best browser for my neads(excuse if my spelling isn`t right).

  30. i’m sticking with opera 10 beta 2 and iron browser (currently at v3.0.189 Beta). i’ve used safari (ugly ui/buggy on pc), ff (bloated), ie8 (bloated), and chrome (data miner/call home; use iron instead).

  31. Chrome is ok but I will stick with Firefox… Safary is not for me… I don’t like anything about safary… :mrgreen:

  32. Hi. Please, Chek it the SRWare Iron Browse

  33. perfect ive been looking for an article like this for a while because i had recently updated firefox iand i needed to know how fast it was preciate it! :p

  34. I don’t take sides, because to be honest preference depends on use, I would like to say Firefox is built more for the developer?
    Internet Explorer’s high percentage of users are in schools, family computers etc where the users just want to browse the world wide web where standard compliance isn’t a deal to them, may I also add about Internet Exploder 8, it’s taken a big step from previous versions, most common CSS issues have been fixed, although it still wants to be different in javascript…but hey it has drastically improved, and anyone who disagrees is abit naive. But Internet Explorer is still a pleasure to use for general browsing, it’s just a pain to develope for, but I think without internet explorer, the challenge of web development would dissapear for me.
    Google Chrome, Safari and Opera are wonderful browsers, compliant, fast and have great looking and easy to use interfaces, I hope they keep it up, but would I be wrong in saying that it seems to me that Chrome and Safari are copying eachother? Ah well ^^
    Great Post, some of the results were surprising and some dissapointing, but still a great 5 minutes worth of reading for me, thankyou :D

  35. Firefox aka Crazyfox fanboi this is nuts, I have used all those browsers but IE is the best, East to the West down to the Earth Microsft is the Best,

    When Firefox and Chrome crashes IE stand still like a Legend

    Internet Explore is still the most used web browser in the world and that’s what really matters to me.

    Are you hurt? your Crazyfox can never take IEs place. go to hell fanboi

    Mitchell Baker you can’t defeat Bill Gates lmao

  36. For those Windows PC’s have 1GB or less, the amount of time spent swapping to virtual memory is significant with a memory hungry browser. From a memory and speed perspective, Firefox seems to be the overall winner.

  37. nothing safari isnt so good it sucks opera 10.5 is the best then comes mozilla…..chrome is good for people with high config computers where memory consumption isnt a problem ~!~

  38. I’m nowhere near computer savvy, but so far on my $100 Toshiba NoRAM re-build, Opera seems to be the one that doesn’t put the computer into a digibetic coma as often.

  39. Lies, lies, lies.
    Firefox 3 & 4 leak memory by the buckets and never gets below 250 mb (clean install).

  40. Harry V Erlandz

    Hello everybody.
    I seem to be alone with my problem and my opinions.
    I have been waiting for a browser that does use the HD for tabs.
    In my kind of work the ONLY good solution would be to have just the local HD address in memory for ALL nonactive tabs, and thus only one full page in memory.
    Some people immediately start talkin about lost speed!
    That is simply silly. Why?
    I have need for a very high number of tabs which results in slow working speed of the computer and the software as a result of too low physical memory!!
    On the other hand reading one page from the HD seldom takes more than a fraction of a second; and with only one page at a time in memory the computer/software keep up maximal reaction times and speed.

    Has anybody heard of an existing or planned browser which could solve this apparent problem?

    I can see no real drawback in only having one page at a time in physical memory, and I assume that should not be impossible to solve for a good programmer (maybe in the Mozilla community?).

  41. K-MELEON [kmeleon.sourceforge.net] is indeed the fastest browser using the least memory and in fact much less than Firefox. The browser looks similar to Firefox so it is familiar and easy to use. It works perfectly even on the oldest computer. Try it. You won’t regret it.

  42. I have experienced all off this browsers and no doubt that Opera is my 1st choise!

Leave Your Comment


Message is the only required field.
Emails are not published.