Now that Vista SP1 has been released there’s no surprise that performance tests and benchmarks are going to start flooding the Internet, and not surprisingly XP takes the crown in most of the cases. ZDNet has gotten the ball rolling (here and here) with their test results:
Looking at the data there’s only one conclusion that can be drawn – Windows XP SP2 is faster than Windows Vista SP1. End of story. Out of the fifteen tests carried out, XP SP2 beat Vista SP1 in eleven, Vista SP1 beat XP SP2 in two of the tests, and two of the tests resulted in a draw.
There’s one inherent problem that I’ve seen with several of the tests so far. Testers are putting the two operating systems up against each other using the exact same systems. No harm, right? Well, there’s a natural progression of technology that would obviously make you think that running XP on the same system as Vista will always result in XP being faster.
Obviously as time goes on the operating systems will take advantage of the system resources that are made available, and if they didn’t there would never be any need to upgrade to a new computer. I remember when I was running Windows 98 thinking that in 10 years we’ll be able to hit the power button and the computer’s bootup sequence would take just seconds because processors would be so fricken’ fast. Here we are about 10 years later, and the bootup times are about the same as they always have been simply because the software grows along with the hardware. Otherwise we might see Windows fit on a few floppy disks.
What I would like to see is someone running XP on a system that was released one year after XP was made available, and then comparing it to Vista running on a similarly priced new system. Or maybe running each of them on a computer that only meets the minimum or recommended requirements for each operating system would be even better. Then these tests would mean a little more to me.
To me these comparisons are like looking at the performance of a video game that was released back in 2001 and putting it up against a game that was released in 2007… using the same hardware. You would obviously expect to see the game from 2001 to do a lot better, but the one from 2007 will likely be better in the graphics compartment.
@Ryan:
Your image looks familiar. It looks like an Apple II-e with an attached fan and two floppy disks boxes. I don’t recognize the monitor. It doesn’t seem to be an Apple monitor.
Am I right? This was my first personal computer in 1983. I loved it!
Regards,
Omar.-
Well said!
Actually it is from a Microsoft ad:
[aresluna.org]
@Ryan:
Yep, it’s indeed an Apple II personal computer and the software shown in the ad is Multiplan. This electronic worksheet was introduced by Microsoft during the early eighties in an effort to debunk Dan Bricklin’s very successful VisiCalc.
I’m holding an original Multiplan in my hands right now. It came with a plastic cover, a beautiful binder manual, a Quick Reference Guide and two 5.25″ large black floppy disks (the System Diskette and the Boot Diskette). I can’t remember how much I paid for it, probably a fortune. Software and hardware was pretty expensive during those days.
The operating system was Apple DOS 3.3 and the Apple II used the MOS Technology’s famous micro processor 6502. The date on the manual is 1982.
Thanks for the memories, Ryan.
Regards,
Omar.-
If you are trying to compare apples-to-apples then I think you would have to use the exact same hardware to test both operating systems.
I would hope that as software engineers and architects get better at understanding complex systems they would be able to create a system that does more while using fewer resources.
I wonder if there are reviews that bench-marked XP to Windows 98? From what your saying Windows 98 should have been faster than XP on the same hardware.
Ryan,
I completely agree with you. You cannot compare XP with the configuration of system for Vista. Vista needs 1GB RAM, but XP will work fast even in 512 MB RAM. Not sure why people are behind Vista, they criticize it too much without having tested it properly. My friends used to say, dont go for Vista, but they never tried Vista even for a single time.
Fair points for sure! However, if you’re looking at which operating system to use now and XP gives you everything you need, it *is* important to know the difference in performance for the same PC as well. These comparisons just make clear what kind of performance loss you need to accept for having the extra features Vista offers, in my opinion.
You may be right Ryan from a different point of view but sometimes it is important to know that how much performance you will get in each OS from a given hardware specs to run on. Also fundamentals of benchmarking will not allow you to compare performance of two software on two different machines! Then what is the point of benchmarking the performance?
I don’t really get your reasoning against these tests.
What is flawed, though, is some of the benchmarks. Who, in his right mind, could even think in using Windows’ own ZIP thingie for benchmarking, when it’s well known that its implementation is terrible? Use a proper compression tool, well developed, tested and multithreaded, like WinRAR, not a poor excuse. In my opinion, that’s the only benchmark in which Vista looks bad. Yes, maybe some difference percentages in other tests are high, but if you look at the real numbers, the difference is not that great (particularly, I’m referring to the multi file copy to an external HDD)
I think the key phrase in your analogy was “better in the graphics compartment”. From my experience with Vista, that is the only real improvement (and I use it every day on a laptop and a desktop).
Something I came across, which isn’t exactly an official comparison but demonstrates the point, is this forum post:
[devhood.mit.edu]
People saying that performance will definitely be poor on the 400MHz system if you’re running XP, and that the guy should stick with 98 unless he upgrades the hardware. Operating systems are obviously designed to use the resources that are currently available on the market, and not for machines that are 4 or 5 years old.
I have a ton of friends who have told me that, and when I ask why they don’t like it they always reference things that they’ve read. It’s not always best to follow the leader.
That’s a good point, but you almost always have to expect that a new operating system will utilize more resources than the previous versions. If it didn’t then what would be the point of getting new hardware? Tests like that don’t take into account the additional things Vista is doing in the background (indexing files for search, gather SuperFetch data, etc…).
Vista is doing a lot more in the background than XP ever did to make sure applications start quickly, and that searching for files is instantaneous. Obviously these features are going to utilize the resources that are available to them, but since they take extra processing power and memory usage other things are going to suffer. Obviously stuff like this is going to find its way into every new operating system, but it makes it harder to do equal comparisons of the operating systems without putting each on the recommended hardware that they were intended for.
Why do we need to expect that? Is OSX utilizing more resources with every new version? They manage to do more with less, why can’t Microsoft do that? I think people are accepting performance loss too easily.
As for the features you mentioned: of course you need better hardware for a feature like SuperFetch (more RAM), but it shouldn’t make your existing PC slower compared to using XP – SuperFetch could just be turned off or tweaked if you don’t have enough RAM for instance. As for indexing files for search – I’m using AvaFind & Copernic Desktop Search on XP and it doesn’t slow my computer at all. They seem to be even faster than the searches I’ve done in Vista..
What I’m trying to say is, Vista should not be slower than XP on the same machine – it should be automatically adapted to it and even be faster than XP when comparing the same features on both OSs.
I’ve read plenty of stories of people upgrading to Leopard and it being slow though. Apple also doubled the RAM requirements from Tiger to Leopard (it’s 512MB now, same as Vista), and tripled hard drive space. So obviously Leopard does require more oomph.
I’ve read stories like that too about the last version, but I also read plenty of stories about people installing an upgrade on 6-year old hardware without too much of a problem. Try that with Vista
(I am in no way a Mac fan, don’t even own one but know plenty of people who do)
I don’t get the reason behind the main argument. So, if I do exactly the same things on a new OS, it should be as fast and convenient ONLY if I also upgrade my computer? The problem is, I would still do exactly the same things but with a perpetually empty stomach.
To continue with the analogies: think hopping over to Vista as selling your old car and buying a new, gas-guzzling luxury car that you still only use to fetch beer from the local grocery store. While still obeying speed limits. And the burglar alarm keeps you up at night.
Installing Leopard on a six-year-old Mac? Not even *remotely* plausible. Leopard is itself a resource hog compared to previous versions. It’s a great OS, don’t get me wrong, but it does require more resources than previous versions. People don’t hold the Mac OS to the same standard they do with Vista.
I’m actually running Vista on a now three-year-old system. It runs just as fast as XP did (and I’m running it with Aero, too). I’m sure if I timed things, the Vista system would be slower in some things. But to what degree? Does it take one or two seconds longer to load a file? I’m not really going to notice that. Some things, though, like loading Photoshop and Indesign, the Vista system is actually faster. Go figure. I’m perfectly happy to sacrifice a second or two here and there for all the added features of Vista, like the media center, search, and security features.
I get what you’re saying, but I think a more accurate car analogy would be to take an older car that is nearly falling apart from every angle. Then sprucing it up by redoing the interior and giving it a paint job only to find out that it falls apart the next week since the transmission blows out. The car may look nice and powerful, but if you don’t upgrade the “hardware” it will perform just as you would expect it would with what lies underneath the hood.
That would be the SuperFetch kicking in which makes use of the extra RAM that would normally sit idle. It caches the applications you frequently use in memory so that they start incredibly fast.
I recently bought an HP G60 notebook with Vista SP1. During the first week Vista suffered not one, but TWO catastrophic crashes requiring use of the restore DVDs. In contrast, I loaded a dual boot with Mandriva Linux 2009.0 – and IT works beautifully. Not a single glitch. Now my primary OS is Mandriva Linux with a very simple install of Vista (just in case someone gives me a file that I can’t work with in Linux – which is rare). Debating how quickly Vista crashes doesn’t seem to address the serious stability issues when compared to Linux or Mac.