One thing that you need to remember with these tests is that the results are all relative to each other. Each browser is running on the exact same machine so that the comparisons are accurate. What we heard the last time we did tests like this is “such and such browser performed a lot better/worse for me.” We appreciate hearing what your results are, but for the sake of accuracy they can’t really be compared to what we get.
- All of these tests are performed on the same Windows Vista SP1 machine, and is wired into a network to minimize the effects of wireless disturbances.
- To test both IE7 and IE8 Beta I ran all of the tests in IE 7, installed IE 8, and then reran all the tests again.
- All browsers started with a clean profile and no add-ons/extensions installed.
- Caches were cleared before each test was run.
- Only one browser was open at a time and no other applications (other than standard Vista services) were running.
- Internet Explorer 8 was always used in the native rendering mode (a.k.a. standards compliant mode).
Pretty much none of the browsers were able to complete all of the tests error-free, and so we’re focusing purely on the speed. We ran each test three times, totaled the runtime (measured in milliseconds) for all four libraries, and then averaged the results. In the parenthesis you’ll see the results of each test we ran before averaging them together (the overall smaller number is better):
- Safari 3.1: 447.33ms (407,536,399)
- Opera 9.5.9841 Beta: 502.00ms (523,456,527)
- Firefox 3 Beta 4: 909.00ms (921,904,902)
- Opera 9.26: 1036.33ms (992,1034,1083)
- Firefox 220.127.116.11: 1507.67ms (1523,1472,1528)
- Internet Explorer 7: 5944.33ms (5965,5998,5870)
- Internet Explorer 8 Beta: 6690ms (6245,7206,6619)
It looks like the new Safari 3.1 takes the crown on this set of tests!
–Page Load Times–
I was trying to figure out what the best method would be to measure page load times in all the browsers. Sure some of the browsers report how long it takes for a site to load, but we wanted a universal way that would work across all of the browsers. We figured sitting here with a stop watch just wouldn’t cut it.
We ran the page load test three times on two different sites so that we could really see what the results were like. We used the Official Google Blog and the Yahoo Search Blog for our benchmarks, and there are very good reasons that we chose those sites. Both of those serve up nearly the exact same content every time you load the site. If I chose a site such as ours we would run into the issue of different ads being served in the different browsers.
For each test the browser started with a cleared cache, and the three results were averaged together to get a single overall load time (measured in seconds). In the parenthesis you’ll see the results of each test we ran before averaging them together (the overall smaller number is better):
Note: I literally went and deleted each browser’s cache after each refresh just to remove any concern that a Control/Shift refresh was not deleting the site’s cache correctly.
The Google Blog:
- Opera 9.5.9841 Beta: 2.498s (2.129,2.606,2.760)
- Safari 3.1: 2.798s (2.619,2.963,2.811)
- Firefox 3 Beta 4: 3.009s (3.167,3.347,2.513)
- Opera 9.26: 3.360s (3.606,3.215,3.260)
- Internet Explorer 7: 4.235s (4.402,3.800,4.504)
- Firefox 18.104.22.168: 4.485s (4.852,4.258,4.346)
- Internet Explorer 8 Beta: 4.602s (4.409,4.238,5.158)
The Yahoo Search Blog:
- Safari 3.1: 1.411s (1.547,1.312,1.375)
- Opera 9.5.9841 Beta: 1.599s (1.578,1.625,1.593)
- Opera 9.26: 1.677s (1.547,1.625,1.860)
- Firefox 22.214.171.124: 1.771s (1.797,1.844,1.672)
- Firefox 3 Beta 4: 2.055s (2.430,2.143,1.591)
- Internet Explorer 7: 2.594s (2.563,2.219,3.000)
- Internet Explorer 8 Beta: 3.365s (2.875,3.750,3.470)
It looks like Safari 3.1 and Opera 9.5 both do really well in these tests, and I would consider it a tie between the two.
This is probably one of the areas that interests most of you. Memory usage has become a big concern these days as we’ve seen some browsers (*cough* Firefox *cough*) use up insane amounts of our computer’s resources. So we took each of the browsers seen in the previous tests, developed a list of sites to open in each, and went at it.
We’re not going to list out all of the sites that we decided to visit, but they are all sites that are extremely popular. Places like MySpace, YouTube, CNN, and others were all included as we chugged through our four different memory usage readings:
- Started the browser, and took a memory usage reading.
- Loaded 10 predetermined sites in tabs, and took a memory usage reading after all the sites finished loading.
- Loaded 15 more predetermined sites in tabs (totaling 25 sites), and took a memory usage reading after all the sites finished loading.
- Let the browser sit for 10 minutes with the 25 tabs open, and then took a memory usage reading.
And now for the results! The table below lists the different browsers and the result from each test mentioned above. The best browser from each test is highlighted in green, and the worst is highlighted in red.
|Startup||10 Sites||25 Sites||25 Sites After 10 Minutes|
|Firefox 3 Beta 4||21.3MB||68.9MB||118.2MB||124.7MB|
|Opera 9.5.9841 Beta||15.8MB||98.3MB||184.4MB||186.5MB|
|Internet Explorer 7||6.3MB||134.1MB||248.3MB||249.7MB|
|Internet Explorer 8 Beta||5.6MB||141.6MB||244.2MB||248.7MB|
For this round it’s clear that Firefox 3 Beta 4 walks away as the clear winner. Mozilla has obviously put some work into making Firefox 3 a more memory efficient browser than it previously was, and this is proof of that.
I can’t say that I was surprised that some version of Internet Explorer almost always did the worst, but I was quite taken back that Internet Explorer 8 shows little improvement over version 7. Apparently that is not the focus of Microsoft right now.
It took us about 6 hours to compile all of the results that you see above, and I would say that doing this on your own is not really for the faint of heart. Things like not being able to run IE7 and IE8 side-by-side is really a time killer, but we wanted to be sure to include both versions in our results. And surprisingly the only browser that crashed on us was Internet Explorer 8 Beta when trying to open the 25 tabs.
We’d love to hear any comments you may have regarding our results, but remember that what you’ll see on your computer will likely differ from what we see. The important thing to take out of the stats is how each of them relatively rank up against each other.