Introduction to testing

We all know that real-world testing of CPU and GPU performance is always better than synthetic tests. It’s no surprise that in an evaluation of a gaming product such as a desktop CPU or GPU, every review website will have performance data from the latest and most popular games. Unfortunately, multithreaded games aren’t here yet. So, when it comes to the evaluation of higher-end workstation-class systems such as dual processor systems or 64-bit CPUs, it’s much tougher to build a correct set of benchmarks. Synthetic benchmarks make it easier for reviewers, but it’s unrealistic to gauge system performance using a suite featuring Photoshop 7 when everyone in the real-world is running Photoshop CS. It’s here where you’ll find the greatest amount of differences between various reviewers -- each person has their own idea of what a real-world test consists of.

Of course, that’s the beauty of the Internet and the free-access to review websites – you can always read reviews from multiple sources and even today, FiringSquad has two completely independent writers covering the same product (Chris’s review is coming later today).

Personally, I’m not the type of computer user who spends much time regularly importing Outlook PST files. However, since I’m a scientific computing and multimedia guy, that’s where I’ve focused my efforts. Due to time limitations however, this preview article won’t have all the benchmarks ready. I’ll have all of these numbers available for our upcoming sequel to this 2003 article.

Synthetic Tests

We use SiSoft Sandra Professional for our synthetic tests. These tests are probably best representative of system performance when you’re developing customized software. That is to say, while these tests won’t be useful to the average end user, they are probably the most predictive for all those aerospace and semiconductor companies who need all the compute power they can get their hands on.

Scientific Computing

My scientific computing tests involve two applications: MATLAB and LS-DYNA.

MATLAB Release 14

MATLAB is your basic multipurpose scientific computing application. Every engineer and his brother have used Matlab at one point or another. It's a very flexible application used in high school to teach basic Newtonian physics and was used in industry to design the Joint Strike Fighter. It used to be said that Matlab was single-threaded because for most tasks, a lot of computation time is spent processing the script, something that isn't parallel at all. Parsing scripts isn't a very glamorous aspect of scientific computing, but it's very important to real-world use. Think of the car that does 0-60 in 4 seconds but requires you to refill the gas tank every 10 miles. There's no doubt that the car is fast, but no one would really use it. Well, starting with Release 14 of Matlab, multithreaded support is included through the use of the Intel Math Kernel Library. Although this library is optimized for Intel processors, it works with AMD CPUs too. AMD’s own optimized math performance library exists for Linux, where most of the applications lie, but they’ll have a Win32 version compatible with Matlab in the future.

To measure real-world performance, I'll be using a lengthy script from our lab which I used in the 2003 article, still codenamed as N72. Since it is about two years later, I can tell you more about the script. N72 really stands for Number 72. The specifics remain proprietary but data from an MRI of the heart is read into memory and the 3D geometry of the heart is processed out and then the positions of myocytes are determined. The whole thing requires the system to have a GB of system RAM. This script represents a real-world example of reading in raw data and then processing it to get the meaningful data. Obviously different fields of science have different tasks and different tasks will perform differently on different architectures.

It's obviously not designed to be a comprehensive Matlab performance benchmark, but it's a real-world test that has a realistic balance of true computation and script parsing performance. This is something that I feel is missing from ScienceMark. ScienceMark is better at representing the back-end calculation rather than the actual performance measures that affect user-input and response.

LS-DYNA Release 970

LS-DYNA is a general purpose transient finite element solver capable of simulating complex real-world problems. That’s the party line at least. Essentially, it’s software that lets you simulate all sorts of things. Automakers use it to developer safer cars by simulating crash tests, the military uses it to simulate weapons explosions, and scientists can use it to study biomechanics. I will bench the CPUs using two classic tests, a 3-vehicle collision and a single front-collision. The 3-vehicle collision takes more than 24 hours to complete – we do not have these numbers ready for this round of articles.