|Solid State Drive (SSD) Benchmark Performance Testing|
|Articles - Featured Guides|
|Written by Olin Coles|
|Monday, 22 December 2008|
Page 12 of 12
SSD Testing Conclusion
EDITOR'S NOTE 27-August-2010: This section has been updated to provide relevant testing advice for modern SSDs using current benchmark tools.
As we've explained in our SSD Benchmark Tests: SATA IDE vs AHCI Mode guide, Solid State Drive performance revolves around two dynamics: bandwidth speed (MB/s) and operational performance (IOPS). These two metrics work together, but one is more important than the other. Consider this analogy: operational IOPS performance determines how much cargo a ship can transport in one voyage, and the bandwidth speed is to fast the ship moves. By understanding this and applying it to SSD storage, there is a clear importance set on each variable depending on the task at hand.
For casual users, especially those with laptop or desktop computers that have been upgraded to use an SSD, the naturally quick response time is enough to automatically improve the user experience. Bandwidth speed is important, but only to the extent that operational performance meets the minimum needs of the system. If an SSD has a very high bandwidth speed but a low operational performance, it will take longer to load applications and boot the computer into Windows than if the SSD offered a higher IOPS performance
If you've read through the details of this article, you've probably reached this section wondering which benchmark software is good for testing SSDs. When this article was originally written in late 2008, there were very few options available. As of August 2010 the SSD market has dramatically expanded, yet only a few more useful SSD test tools exist.
DOS-based System Speed Test v4.78 by Vladimir Afanasiev is an excellent precision tool for measuring SSD response time, and results have always been extremely accurate and consistent. System Speed Test also offers sequential read benchmarks, but these results are not suited for testing SSDs because a proper device driver is not present. Response times reported by HD-Tach, HD-Tune, or similar software are often rounded to the nearest tenth of a millisecond. System Speed Test reports exact figures to the nearest thousandth.
Bandwidth speed tests on Solid State Drive technology is best done with linear tools, in my opinion. Linear test tools reveal buffer limitations and NAND IC separation. There are two software suites which provide linear testing that I am aware of: Lavalys EVEREST offers an excellent 'Disk Benchmark' component as part of the test suite. Equipped with both Linear Read and Linear Write bandwidth tests with selectable block size options, I feel that this benchmark tool offers the best bandwidth spped snap-shot for SSD performance. SiSoftware Sandra offers linear bandwidth tests, although I personally feel the options and ergonomics of this software leave a lot to be desired.
As we discovered a few sections back, our benchmark tests showed favoritism towards the Intel SATA controller on one SSD, while it seems that performance between ICH10 and JMB322 are nearly equal using another Solid State Drive for performance testing. Using linear tests, both EVEREST and Sandra gave the same results per drive and controller. So even with linear testing removing doubt from NAND wear-level algorithm inconsistencies, you'll still have to work around the motherboard controller and it's limitations. The Microsoft Windows Operating System is another consideration, as only Windows 7 presently offers TRIM garbage collection support.
Sequential test tools such as ATTO Disk Benchmark, HD-Tach, HD-Tune, Passmark PerformaceTest , CrystalDiskMark, and AS-SSD are all usable benchmarks, but occasionally report performance inconsistencies because of buffered spot sampling and NAND condition. Additionally, CrystalDiskMark and AS-SSD often report much lower sequential read and write bandwidth speeds compared to HD-Tach and HD-Tune, while ATTO Disk Benchmark relies on file size chucks to report bandwidth. The sequential bandwidth speeds reported by CrystalDiskMark and AS-SSD are so low they become questionable, while Passmark PerformaceTest, HD-Tach, and HD-Tune reveal very little information about buffer saturation and are prone to NAND condition impacting performance results.
Of the sequential tools, ATTO Disk Benchmark is most preferred because it illustrates bandwidth speed results at varying file size transfers. From my testing for this article and in other projects, along with the results I've seen from the software tools used, I can conclude that ATTO Disk Benchmark has proven itself consistent in recording SSD bandwidth results and doesn't seem to have a preference for faster SSD DRAM cache mechanisms found in some SSDs. The variety of file size chunks that it tests does give a broad picture of bandwidth performance at each level. Although ATTO is not perfect and still uses spot-testing, it's less imperfect for SSD testing than many of the other alternatives.
Other test tools discard the need for measuring raw bandwidth speeds, and attempt a 'real world' approach towards comparing product performance. PCMark05 and PCMark Vantage are two products that fit this description, and work questionably well for comparing SSD performance. Our own test results section in the article demonstrated that SSD's cannot be benchmarked accurately with PCMark05 or Vantage, and further examination of their White Paper document (pg 27) indicates that the computer system's video card, memory, processor, and Operating System all play a factor in the final score.
One of the most preferred methods of illustrating SSD performance levels is IOPS operational performance measured at a deep queue depth. Unfortunately there are so many different ways to produce these results it becomes difficult to determine which tools are most valuable. Most review websites have adopted Iometer as their preferred test tool, because it offers the greatest queue depth and can be configured for sector offset. Since the SandForce SF-1200 SSD Processor was introduced, Benchmark Reviews has used Iometer to express IOPS performance using this configuration file.
AS-SSD and CrystalDiskMark (3.0 and later) all offer IOPS performance at deep queue depth, and express their results in MB/s rather then the Input/Output Per Second result. These are also very helpful at illustrating SSD performance, and useful in comparison. While HD-Tune does offer IOPS performance, this tool is limited to a single queue depth and is meaningless to SSDs.
In conclusion, Solid State Drives are excellent products with plenty of performance gains to offer even the most casual computer user. Even though the SSD industry has grown, benchmark tools are still limited in their ability to convert performance results into a useful number. Ultimately, I warn readers to regard SSD reviews with a high degree of caution, and lean towards articles that compare results against a wide variety of well-know HDD products to use as a baseline comparison. Other useful articles on this topic include SSD Benchmark Tests: SATA IDE vs AHCI Mode, and Marvell SATA-6G SSD Performance vs Intel ICH10. Benchmark Reviews offers dozens of SSD comparisons in our Featured Reviews: Storage section, and we hope you'll give them a read.