The Mighty USP V. RAID1 (4+4) concat or stripe?
Stripe!
I could end the post here but for those who might be interested in why, I will continue......
Hitachi internals warning. This post goes a little under the hood of a USP, so if you don't do Hitachi storage then you probably won't care about this post. If you are blessed enough to work with Hitachi storage, there's still a chance you won't care or may already know this.
Although what I'm about to explain is neither rocket science nor a secret, there are a lot of people who misunderstand it, and the occasional person who stands up and preaches the wrong argument. I've considered blogging on this in the past but not gotten around to it, but when it cropped up again today on the HP storage forums I thought enough was enough and time to dispel the myth, hopefully once and for all.....
The Myth
A lot of people believe that a USP/USPV/XP12000/XP24000 (hereafter referred to simply as the Mighty USPV) Array Group configured as RAID1 (4+4) is a concatenation of two RAID1 (2+2) Parity Groups. It is not!
(At least not in my own, and what I believe to be, the accepted definition of a concatenation)
First things first
1. RAID1 (2+2) is a way of saying two pairs of mirrored disks. You have four disks; Disk1, Disk2, Disk3 and Disk4. You take Disk1 and Disk2 and configure them in a mirror. You then do the same with Disk3 and Disk4. This gives you two mirrored pairs. See diagram
2. A quick search on Google gives the following as a definition of concatenation - "to connect or link in a series or chain". And, so far as I am aware, almost everybody in the storage world understands a concatenation to be a configuration where you consume all of the space on one disk, or set of disks, before moving on to consume space on the remainder, in series not parallel.
3. During the process of creating a RAID1 (4+4) on the Mighty USPV, you create two RAID1 (2+2) configurations and then "concatenate" them. The tools use the word "concatenate".
So, with the above in mind, surely a concatenation of two RAID1 (2+2) Parity Groups will act like ....well.....a concatenation. That is, you consume the capacity of one of your RAID1 (2+2) sets and then when that is full, move on to the second - right?
Wrong!
Theory
Lets take a look under the hood......
Track size on the Mighty USPV for OPEN-V volumes is 256K. However, when destaging to disk, the Mighty USPV actually writes two tracks per spindle before switching IO to the next spindle in the RAID set., This then gives us an effective chunk size, per spindle, of 512K.
Sooooooooooo...... in a RAID1 (2+2) the stripe size is 1024K. We know this because the Mighty USPV writes 2 tracks per data spindle, and there are 2 data spindles in the RAID set.
The formula for working out stripe size for all RAID configurations on the Mighty USPV is -
(256 * x) * y
Where x is always 2 (because it always writes 2 tracks at a time ) and y is the number of data spindles in the RAID set.
Dispelling the myth
RAID1 (4+4) is not a concatenation. At least not in what we have established is the generally accepting understanding of a concatenation. It is a single stripe across 4 mirrored pairs of disks with a stripe size of 2048K. Each full stripe write to disk touches all 4 (actually 8 because of mirroring) disks. See diagram
The formula this time is (256 x 2) x 4 = 2048
4 because its 4+4
ZZZZZZZzzzzzzzz
Why am I bothering to mention this?
Well, because each time somebody gets it wrong, it makes the Mighty USPV look a little less mighty than it really is. So let's give the box the respect it deserves and not take anything away from it. In theory, a RAID1 (4+4) provides twice the performance of RAID1 (2+2).
1, 10, 1+0, 0+1, 01 ???????
Now while we're on the topic, let me also explain that when the Mighty USPV does RAID1 it is actually doing RAID10. Let there be no mistakes in thinking that it does RAID0+1 or anything nasty like that. It does not, and RAID 10 and RAID01 are crucially different.
So what is RAID 10?
Exactly what it says on the tin - RAID1 and RAID0
To break it down -
RAID 1 = Mirroring
RAID 0 = Striping
So RAID10 is a combination of mirroring and striping, in that order. First you create your mirrored pairs (that's the RAID1), and then you stripe across them (that's the RAID0). 1 + 0. Simple.
This is more resilient than 0+1, where you first create your stripes and then mirror them. If your 4+4 was RAID0+1 you would create a stripe using your first 4 disks, and then create a second stripe on your remaining 4 disks. Once those two stripe sets were created you would configure one as a mirror of the other.
However, in this situation the failure of a single disk in either stripe will break the mirror! Not good! So obviously not done on the Mighty USPV.
As expected, the Mighty USPV gives you the best option available.
Nigel