Stick a Needle in Your IOPS

What are IOPS anyway?  Simply put, they are Input/Output Operations Per Second, and they are meaningless.

(How many IOPS do you think I can get this thing up to, anyway?)

Wait, what?  How could I possibly be calling IOPS meaningless?  I’m a STORAGE person!  How could I say this? I think that maybe it was one of those metrics that had meaning, at one point.  Maybe.  Now?  Not so much.  Here’s why.

Let’s say I go down to the Storage Depot and I’m looking at all the shiny storage controllers on the shelves.  I point at the prettiest one, and I ask the helpful person with the name tag how many IOPS I can get out of it.

This is almost never a straight up, easy answer.  There are many, many factors to take into consideration, and the Storage Depot person doesn’t have a quick answer for me.  Let’s just take a look at a few.

• What’s going on this thing?  That’s always a good starting point to hopefully being to understand what applications are capable of.
• How much space does it need?
• Read/Write profile.  Random or sequential reads?  Random or sequential Writes?  How much of each?
• What types of disks are in the storage array?  How fast are they, and how big are they?
• Latency sensitivity.  Is whatever is going on this system sensitive to latency?  Better yet, what is the business impact if the latency increases?

I got a little frustrated with the helpful person at the storage depot when they couldn’t give me an answer, so I asked them for a storage array with 100,000 IOPS.  They came back with a bunch of different options, and I got even more frustrated.

Now, let’s go with some assumptions here.  We’re going to assume that I can get 75 IOPS out of a 7,200 RPM SATA drive, 150 IOPS out of a 10,000 RPM SAS drive, and 20,000 IOPS out of a SSD.

The first option they gave me had some storage controllers, and 1334 data disks.  These were SATA drives.  Wow, that would take up a lot of space, even assuming I can cram 48 disks in 4U!  The disks only would take up almost 3 racks in my data center.  Plus powering it and cooling it?  Yikes!  If they were 2TB drives, that’s over 2PB!

Now, the second configuration they gave me had 667 data disks.  These were SAS drives, and they were the little guys.  I can get 24 disks in 2U.  That’s a little over a rack in my data center, and a little bit more manageable.  If they were 900 GB drives, that’s a bit under 600 TB.  Much more manageable.  Don’t they cost more because they spin faster though?

The third configuration, well, this person just handed me some controllers and a handful of drives.  Took up almost no space, and would be really cheap to run.  Maybe I’ll throw it under my desk.  I would only need 5 SSDs, and if they were 800 GB each, that’s not even 4 TB.  Slight problem though, I don’t have enough room for my stuff.

So which one am I going to buy?  Well that depends on what I want to do with it.  If I have a HUGE backup environment, maybe I’ll get the first one.  Perhaps if I was running everything in VMware, I’d buy the second one.  Maybe if I had a small database that had to go really really fast, I’d buy the third.

The number of IOPS, in this case 100,000 can be reached a number of ways.  I ran to Storage Depot on my lunch break, how on earth am I going to get out of here with something that can do 100,000 IOPS in the next half hour?  I can’t easily, which is a good thing.  No one should walk into storage depot and pick out a random array just by the number of IOPS.  I just reviewed three completely different configurations that would all provide the required 100,000 IOPS.  Now tell me how that metric is meaningful.

Let’s go back to the above points, and think this through a little bit more.

So what are we putting on this thing?  For simplicity’s sake, let’s say we’re going to put a shared storage environment on it.  We have backups, virtual machines, and a high performance database or two.  We also need some flexibility inside this array for when we need to do maintenance.  I’ve got VMware on this thing, which I never have to take down, so why should I need a storage outage for upgrades?  What about capacity?  How much space do I need for each?

Without going into too many more boring details, because I could go on for pages and pages about this, I’d probably buy a combination of the three.  Once I have a handle on my workloads, how much room they take up, and what I expect them to be doing, I can reasonably determine how much of each disk type and how many controllers I should get.

Okay, I’ll budge a bit.  It isn’t that IOPS are completely meaningless, but they are only one small part of an overall set of metrics (like the L word, but that probably deserves a dedicated post), and probably is not the most important one.  While everyone races to hit a million, a billion, or a TRILLION IOPS, I’ll be over here architecting my solutions by thinking a bit more about the real workload and less about IOPS.