Please note I did not do these tests, they were done by http://www.anandtech.com

Networking Benchmarks

Just to get a quick idea of what these new features can do, we ran our usual networking benchmark suite on a pair of ASUS P5B-Deluxe motherboards using both the on-board PCI and PCIe connected gigabit network controllers (Marvell 88E8056 and 88E8001 respectively). We used two tests for this, a multiple small file transfer test consisting of about 450 files totaling 600MB, and a large file transfer test consisting of a pair of ISO images totaling 3GB. In order to keep the load on our D-Link Gamer Lounge router consistent, we also had the WMV-HD version of Terminator 3 streaming in the background. We tested Windows XP SP2 for a base score, followed by Vista with Compound TCP switched on and off.

Unfortunately for Vista, neither test is particularly favorable. Compared to XP when Compound TCP is disabled, Vista is anywhere between 25% and 50% slower than XP in terms of the total time required for these tests. Without the ability to separate Vista's networking stack from the drivers for our NICs, it's impossible to tell if this slowdown is the fault of the networking stack being worse for this situation, or if the Vista drivers for this line of Marvell NICs are not quite as tuned, so as a comparison to XP this test is inconclusive. Either way, for this particular setup Vista ends up being slower at file transfers than XP.

The one bright spot however is that when enabled, Compound TCP is clearly having some effect even on our low-latency network. The 5% or so boost in Vista's low scores won't bring it back above XP, but it clearly proves that Compound TCP does have a real-world effect on performance. This will be something we will be able to talk more about as Microsoft perfects this algorithm set for the release of Longhorn Server later this year. Hopefully, the relatively slow performance we're seeing right now is being caused by drivers rather than by Vista's "new and improved" networking stack. We will definitely keep an eye on networking performance over the coming months, as most of the people we've talked with expected Vista's networking performance to be faster than XP's.

CPU Performance and Threading in Vista

While we found memory requirements in Vista to be greater than XP, CPU requirements weren't as much of a big deal. There are parts of Vista that will obviously benefit from faster CPUs, but if you have anything in the Core 2 or Athlon 64 X2 class you should be just fine. The argument for dual and quad core processors remains relatively unchanged with Vista. For multitaskers and many CPU-intensive workloads, a dual core CPU makes perfect sense. Vista doesn't make the argument for dual or quad core any more compelling than XP in our opinion; the additional background tasks that run in Vista that weren't present in XP don't really eat up that much CPU time to begin with, so dedicating an entire core to them isn't necessary.

The new applications in Vista don't appear to be any more threaded than XP, despite Vista being heralded as the beginning of a highly threaded future. Microsoft Word remains single threaded, although Excel can now take advantage of multiple cores when performing calculations. Windows Movie Maker seems to be optimized for two cores, while importing and attaching files in Photo Gallery is surprisingly single threaded. Dual core still makes a big difference in the overall experience, while quad core still isn't necessary but remains useful for a handful of situations.

With high definition video playback and encoding being two very big drivers of CPU performance and number of cores, Vista will be the OS under which new highly threaded applications really start to appear but there's no reason to feel like four cores are necessary to run Vista today. An interesting bit of trivia is that on a Core 2 Duo E6300, simply opening a new Explorer window in Vista will eat up about 19% of your total CPU time while the window opens and animates; turning Aero Glass off doesn't change the CPU usage either. Maybe four cores are necessary...

Vista Search for Fast Drives Only?

In our opinion the two biggest reasons to migrate from XP to Vista are its Search and SuperFetch technologies, as they both dramatically impact productivity. When Mac OS X introduced system-wide indexed search functionality, we wondered if disk performance would dramatically impact how responsive the search was. More specifically, would notebooks running OS X have significantly slower search times than desktops with faster 3.5" drives? Under OS X, while we noticed a difference between desktop and notebook drives, it wasn't large enough to render the feature crippled on a slower drive. Thankfully, the same can be said about Windows Vista.

We noticed absolutely no difference in how long it took indexed search results to appear whether we used a 150GB 10,000RPM Western Digital Raptor, our 500GB WD test drive, or even a 5-year old 100GB drive - the results were always near-instantaneous. In fact, a much larger impact on search performance was how much memory the system had. The less system memory you have the more disk I/O there's going to be due to swapping data in/out of the pagefile, and that I/O ends up reducing search performance tremendously. We noticed a much bigger search performance improvement going from 512MB to 1GB of memory than going from a 5-year old drive to a modern day, high-end 10,000 RPM Raptor.

While performing searches didn't show any difference between various hard drives, there is a noticeable performance difference between drives when it comes to how long it takes to index your drive if you should have to rebuild your index. The chart below shows the amount of time it took to rebuild Vista's search index on the three drives we've been using in this review:

Obviously the larger the amount of data to index, the greater the impact drive speed will make on it, but this should give you a bit of a reference point. Of course all of the normal benefits of moving to a faster drive still apply (faster application launches, documents open quicker, games load faster, etc...), but the point we're trying to make here is that if you've got a reasonably fast drive already, don't feel like you have to replace it in order to keep up with Vista.

Overall Vista Performance: XP vs. Vista vs. Vista x64

Benchmarking in Vista is an extremely difficult task, thanks to features like SuperFetch designed to improve performance at the sacrifice of repeatability. We're in love with SuperFetch as a feature, but it does make our jobs a bit more difficult as we have to be very mindful of what has been cached into main memory and how that impacts the performance of our benchmarks. Testing difficulties under Vista are compounded by the fact that our major test suites don't work under the new OS. SYSMark and PC WorldBench simply won't run and thus we have to rely on individual application tests and home-brew scripts to measure performance.

Our goal for this article was to find out if Windows Vista, on modern day hardware, continues to be slower than Windows XP as it has been in previous beta and RC versions. The secondary goal is to look at x64 performance to see if it too has changed behavior, to the point where you no longer need to worry about losing 32-bit performance if you move to the 64-bit version of Vista. We ran a handful of application tests to compare the three OSes (XP, Vista and Vista x64), and later on we'll focus on gaming tests as well to showcase graphics performance.

PCMark '05 Performance

Despite looking quite synthetic, PCMark has grown into a nice lightweight alternative to the big application suite benchmarks like SYSMark. It was also one of the first benchmarks to get official Vista support, although it remains a 32-bit only test.

Performance under PCMark is fairly close between Vista and XP, with XP generally holding on to a 1% - 4% performance advantage in most of the categories. The only exception to the rule is the graphics score of PCMark which has XP at around 35% ahead of Vista, but we'll address graphics performance shortcomings of Vista later in the article.

The difference between the 32-bit and 64-bit versions of Vista are hardly pronounced, with the two being evenly matched across the board. Once again, remember that PCMark is a 32-bit benchmark so it is running in Windows on Windows (WOW64) mode. That it can remain competitive is good news, but whether that's because WOW64 works well in all cases or it is merely optimized for PCMark05 remains to be seen.

3DMark '06 Performance

We chose to include 3DMark '06 performance here instead of with our game tests to look at the gaming performance of Vista on a system level rather than a GPU level:

Overall XP is about 6% faster than Vista, although the 3DMark06 CPU score is nearly 13% higher on XP than it is on Vista. As a 32-bit benchmark, it's once again good to see no negative performance difference between the Vista x86 and x64 versions.

General Application Performance

3dsmax Performance

Our 3dsmax 8 tests are particularly interesting under Vista because they showcase the poor OpenGL performance of current Vista GPU drivers in something that isn't a 3D game:

If we switch to Direct3D for 3dsmax, the performance delta narrows considerably to the point where XP only has a 2% performance advantage over Vista.

Encoding Performance

Encoding Performance using DivX and WME9 are both slightly slower on Vista than under XP, but the difference is negligible.

Application Launch Performance

Similar to our earlier ReadyBoost testing, we timed how long it took to open Adobe Photoshop CS3 and 14 images under the three OSes,

Finally we have a benchmark with Vista is faster than XP, and for some reason Vista x64 is even faster than that.

Office 2007 Performance

Our final two Vista vs. XP tests are from Intel and they measure performance in Microsoft Word 2007 and Excel 2007.

Document comparison performance in Word is about 7% faster under XP than under Vista, while numerical calculations under Excel are about equal under both OSes.