Today I completed the initial performance testing of my EMC XtremIO PoC system. I wanted to take a shot at it myself before the EMC SMEs come in to tune and optimise the configuration. In a single word, “Wow!” This is the first time I have witnessed 400,000 IOPS in any kind of enterprise lab. I look forward to seeing what additional tricks the experts can make my “X-bricks” perform.
Business Requirement for XtremIO
I can imagine people reading this and asking, “Why? It is so expensive!”. Well, the organisation I work for uses monolithic storage (EMC Symmetrix VMAX) which has been sized for capacity, and after 2 years of use we are feeling the impact of performance degradation as we consume the total capacity of the solution. My business requirement is to create a small but powerful “High Performance” cluster of compute, network and storage that will provide low latency, high I/O resources for my business critical applications that are currently suffering. This XtremeIO PoC is an attempt to meet that business requirement; I am also seriously considering hyper-converged infrastructure and server-side flash-cache acceleration as well.
Iometer Test Configuration
- 3 x HS22 blades with 2 x 4C Intel Xeon X5570 2.9GHz CPU, 96GB RAM and QLogic HBAs per blade running ESXi 5.5 Update 2 (Boot from DAS)
- IBM BladeCenter Chassis with Brocade Switch modules connected to XtremIO chassis with 6 x 8Gb FC
- IBM BladeCenter Cisco 1GE Switch Modules connected to Core switch network
- EMC XtremIO X-bricks version 2.4.1 with EMC XtremIO Storage Management Application version 2.4.1
- 8 x 1TB Volumes (Encryption enabled) mounted as VMFS-5 Datastores with VMware NMP set to “Round Robin”
- 8 x Iometer Dynamos running on Windows Server 2008 R2 with 3 x 40GB vDisks connected to Paravirtual vSCSI Adapters (1:0, 2:0, 3:0)
- 1 x Iometer Manager running on Windows Server 2008 R2
- Test 512b, 4K and All-in-one Access Specifications (Two variants: 100% Read 0% Random, 0% Read 100% Random)
Iometer Test Results
Additional Resources
- EMC XtremIO – Provisioning a LUN
- EMC XtremIO – Dedupe: Replace a VMAX?
- Pluralsight EMC XtremIO Implementation Training by Jason Nash
- XtremIO: The Out of the Box Experience by Jason Nash
- XtremIO Gotcha by Andrew Dauncey
- EMC XtremIO management console walkthrough by Brian Suhr
- XtremIO (xmcli) – Creating Initiator Groups and Mapping LUNs by David Ring
- On disruptive storage upgrades… by Chad Sakac
- EMC XtremIO – Setting Disk.SchedNumReqOutstanding On vSphere 5.5 (PowerCLI) by David Ring
vcdx133.com 
They’ve got a great trick when you upgrade the firmware – they wipe you data 🙂
Awesome but… latency seems so high to me!?
I would expect sub ms actually especially with reads since there is no ack to return!?
Curious to see the same tests with the new firmware v3
the same thought… IO results are impressive, but latency should be <1ms in such state-of-the art AFA box…
Yes latencies seem very high for an AFA, maybe it’s because you’re pushing it to the limit. You’d usually be looking to see what it can deliver under 1ms, It would be really interested to see more of a real world test using 8K, 16K & 24K, 100% random, @ 60/40 R/W ratio.
Agreed, in the coming weeks I will post the results for a BCA with SQL Server 2012.
Once an array is at its 100% performance limit, latency will go higher. Once over 100%, latency is determined by outstanding I/Os, not by array capabilities. It is not an issue.
I’m more interested in the artificially high deduplication rate. Is IOMeter writing redundant data? If so, the performance numbers are going to be way off from real world as those writes aren’t making it to disk It would be like doing an IO meter write test to /dev/null. Impressive, but useless.
You should consider a similar benchmark with unique or less reducible data. I know vdbench can simulate that, but I’m not sure about the current version of IOMeter. I believe the 2008 version wrote all 0s.
Isn’t this the box you have to wipe your data out for the last couple firmware upgrades? I think a million iOps in a benchmark are relatively useless (you really need a million iOps) vs mission critical uptime during failures and firmware upgrades. Show us the impact to latency during a code upgrade.
IOMeter is indeed writing redundant data. It’s ‘pseudo-random’ wasn’t that good – perhaps it was fixed in the latest version (1.1.0) but earlier versions certainly weren’t good and produced dedup’able data.
The latency is indeed high. I’d like to suggest to use the VSI 6.3 plugin, which should help in configuring the ESX with XtremIO best practices, which should effect performance, inc. latency.
Would also be interesting to re-run with XtremIO 3.0.