Test Suite
Our test suite is as follows:
Ask any owner with a digital SLR what they look for in a computer and it'll be RAW processing performance. I don't mean "raw performance" in the marketing sense, but development of the unprocessed RAW files from the imaging sensor. That is, while the typical consumer digital camera saves images in JPEG, photography enthusiasts and professionals prefer to save RAW images. These "smaller-than-TIFF" files losslessly record exactly what the sensor sees and allow for greater flexibility in developing the film. Capture One D-SLR is the professional's choice for RAW image development and is a multithreaded application that takes advantage of multiple CPUs.
We run Capture One D-SLR on a set of five Canon EOS D30 images and three Canon EOS D60 images. The images are converted using JPEG High Quality settings in the sRGB colorspace.
Sometimes called the "Big-CCD-in-software," NeatImage is a revolutionary noise reduction application which removes noise while maintaining a surprising amount of detail. The algorithm is fine tuned and robust enough that it's possible to clean up images from webcam with good results to cleaning up images with flagship D-SLRs. If don't use the term revolutionary often, but this is truly revolutionary. We'll likely be including this benchmark in the future
Our test involves running NeatImage on 2 Canon EOS-10D JPEGs running a complete noise reduction algorithm. This is a single-threaded application.
Look around on the 'net and chances are you'll see people using PSBench for their digital imaging benchmarks. While PSBench has its merits, when is the last time you saw a PhotoShop user use the "Lens Flare" filter? Ok, how about the last time it wasn't used as a spoof at SomethingAwful? Lens flares in Photoshop are like the flashing "under construction" sign of the 'net. Likewise, how frequently do you need to rotate large 100MB images by 0.9 degrees? While that's certainly used more frequently than the lens flare, it's still not very common.
You see, the problem with PSBench is that it doesn't accurately reflect the manner in which Photoshop is used in the real world. Graphics artists make their magic by drawing - not using filters. Based upon the direction Photoshop has taken over the last few years, we chose to test Photoshop performance from the perspective of the digital photographer. In order to do this, we used the Fred Miranda digital imaging scripts as our real-world test. These Photoshop actions represent the types of Photoshop activities a real digital photographer would use on a daily basis.
Specifically we tested using Custom Sharpen Pro 2.0, Medium Strength on a D60 image and Stair Interpolation Pro 2.0 which converts a 6 megapixel D60 image to a 13x19 @ 300dpi image (22 megapixels). Professional photographers will resample every image before it is printed out, so even small differences in performance will have an enormous benefit in the long run. Normally when you print out any image, the printer driver is doing the upsampling to the 1440 or 2880 dpi of the printer. The rationale for pre-upsampling your images is that products like Photoshop or Qimage use smarter interpolation strategies which make your pictures look sharper.
Although Capture One D-SLR is probably the most advanced RAW development software, Adobe Camera Raw is also very popular and provides a good camera-independent test. We evaluated the time it took to open up a D60 image at the native 6 mpixels at 8-bit color and the time it took to internally upsample to a 15 megapixel image at 16-bit color through Camera RAW. These images were processed in Adobe RGB colorspace.
This is how we do it in the OC
Digg
Del.icio.us
Reddit
SU
furl
