How to not review the Nvidia Ion

There's been a lot of buzz lately surround the Nvidia Ion Platform.

For those of you who don't know, the Nvidia Ion platform is an Intel Atom CPU combined with the Nvidia Geforce 9400 core logic chipset. The Intel Atom is a regular, but very low-power, CPU. The Geforce 9400 core logic chipset is everything that a northbridge and a southbridge typically do, a memory controller, audio, LAN, storage (Serial SATA) controller, and of course, the best integrated graphics every made. The point of putting such a nice GPU with such as weak CPU is to run 3d simulations (games) better, offload (upto 1080p) video processing to the GPU, or offload general purpose calculations to the GPU (CUDA, openCL.)

Now, the intel Atom isn't brand new. Models came out with it last year roughly around August. Combined with the old Intel GMA 945 GSE chipset, it was not a stellar performer. It had a FSB of only 533 mhz, and the memory controller on the intel chipset would only provide 533 mhz memory.

But performance was not the point of the Atom, as the intel atom was low power (a maximum 2.5 watts for the netbook model @ 1.6 ghz) and low price ($43 for that model.) Performance showed that the atom was roughly as fast as 2.0 ghz pentium 4 CPU. That fact is important. In the days of CPU's with model numbers like T7100 and E2220, it is hard to have any clue how a processor compares to another processor. However, considering that LegitReviews found the Atom 230 (1.6 ghz, 533 mhz FSB, just like the netbook N270) to get 477 CPU Marks in 3DMark06, and a sample score from the Online Result Database shows a 2.0 ghz Pentium-4 to get 524 CPU Marks, it's obvious that they are pretty similar.

So from now on when deciding what games to test a single-core atom CPU with, we should look for games that require nothing more than a 2.0 Ghz Pentium 4. Since cranking up the graphics detail puts more stress on the CPU, and the minimum requirements are for minimal graphics settings, logic would dictate that we would run the games at minimum detail if they require a 1.6 ghz to 2.0 ghz pentium-4. If a game requires something greather than a 2.0 ghz pentium-4, we shouldn't waste our time testing it. If a game requires something between 2.0 ghz and 4.0 ghz in processor speed, it is worth testing it with the dual-core atom N330, and the results will likely depend heavily on how well multi-threaded the game is.

Guess what professional reviewers did? First XbitLabs tested the Atom N230 (1.6 ghz) with three games; Quake 4 (minimum CPU: 2.0 ghz P4), Trackmania: Nations Forever (minimum CPU: 1.6 ghz P4), and World of Warcraft (minimum CPU: 1.3 ghz P4 or Athlon XP 1500+).

Considering these requirements, it would make since definitely to run Quake 4 on low detail. Trackmania and WoW should be tested out definitely on low, and probably on medium detail.

What did they do? They tested out all the games on medium! The results were pathetic, but they hardly gave it a chance. Remember that 30 FPS or so is necessary for a game to be playable.

In fact, the supposedly 5 to 10x faster Geforce 9400 wasn't much faster in trackmania than the Intel chipset. This highly suggests that the game was CPU limited at that detail level.

Next I came across Hot Hardware's Nvidia Ion review. Much like XbitLabs, they prepared a very professional review, with meticulous detail as to how they tested the system. I am glad they tested out the Atom 330, which proved to be far superior to the 230 at certain games. However, they tortured the poor Atom CPU even worse. They ran Left 4 Dead, which has a minimum CPU of a 3.0 ghz Pentium 4, and Quake Wars, which has a 2.8 ghz Pentium 4 requirement! Again, the Atom 1.6 ghz Pentium-4 CPU is only equivalent to roughly a 2.0 ghz Pentium 4. Only the Atom 330 deserved to run these games. However, they then did something even worse. They tested them "with gaming quality settings set to medium or high, depending on the feature!"

Here are the lackluster results:

I'm honestly impressed with the Intel Atom's ability to run games that greatly outclass it, but the average reader has no clue how fast the atom is supposed to be relative to the games requirements.

I believe testing these games was unfair to the Atom N270 and the Atom N230 CPUs, as well as the Ion platform in general. It made them look like they were incapable of gaming, when at the appropriate detail level on the appropriate games, it could have performed well.

I'm guessing the reason this mistake was made was that reviewers aren't used to testing integrated graphics chipsets like this. Typically, they test them with the fastest possible CPUs, where the minimum CPU requirements of a game are never an issue. When testing the Ion Platform, they are testing the Geforce 9400 with either a very weak CPU (the Atom N230 or N270) or a very specialized CPU (the Atom N330.)

What games could Hot Hardware have tested the atom with? Let's look at the top sellers on steam. Civ4 complete edition is up there, and Civ4 itself only requires a 1.2 ghz P4, Athlon, or equivalent.

So please, next time you test out either a low-end CPU or a low-end GPU, make sure you test it under conditions it is made for. By default, test it out under games with low or minimum detail, and include games that it matches the minimum requirements for.