1TB SATA, raid group size, 16TB limits etc

[No More Linux!]

is the raid group recommended size still 16 with the new 1TB sata drives?  basically we are pretty much limited to one shelf per aggregate--not sure how this is good thing.

is the 16tb limit going away anytime soon?  it is becoming increasingly too restrictive as these larger drive sizes are introduced.   or am i just supposed to create one shelf aggregates of 1tb drives from now on?

[Blake Golliher]

You can (and should) span multiple shelves in an aggregate.  You can
turn on 'options raid.raid_dp.raidsize.override' which allows you to
grow past 16 disks for raid_dp, but not more then 19 disks 1T drives
in a single array (the size limit kicks in).  In 7.3RC1 you can move
to 21 disks in a single aggregate.  We do that here a lot and (knock
on wood) haven't had any serious failures.

GX let's you get around the 16T aggregate (12T usable space.. boo)
limit by overlaying several volumes.  But not all applications call
for gx (yet) so ymmv on that.

-Blake

On Thu, Jun 5, 2008 at 12:09 PM, No More Linux! <no.more.linux@...> wrote:

>
> is the raid group recommended size still 16 with the new 1TB sata drives?
> basically we are pretty much limited to one shelf per aggregate--not sure
> how this is good thing.
>
> is the 16tb limit going away anytime soon?  it is becoming increasingly too
> restrictive as these larger drive sizes are introduced.   or am i just
> supposed to create one shelf aggregates of 1tb drives from now on?
>
>

[tmac-3]

Let ontap figure where to put things.

Besides,. the recommended size is 14 for SATA and 16 for FC per raid group.
Obviously, if you want to MAX out an aggregate at 16TB, you have to
take into consideration
you cannot do it in a single RAID group (well, I think you can, but it
is not recommended!!).
Therefore, your raid groups may look like 8+8 (maybe even 9+9), but
you probably do not
want 14+2 or 14+4.

Either way, you are eating the parity drives. Might as well make them
balanced groups.

Oh, one more thing. Let ONTAP pick which disks to use. It will
intelligently pick across shelves and controllers.

--tmac

On Thu, Jun 5, 2008 at 3:09 PM, No More Linux! <no.more.linux@...> wrote:

>
> is the raid group recommended size still 16 with the new 1TB sata drives?
> basically we are pretty much limited to one shelf per aggregate--not sure
> how this is good thing.
>
> is the 16tb limit going away anytime soon?  it is becoming increasingly too
> restrictive as these larger drive sizes are introduced.   or am i just
> supposed to create one shelf aggregates of 1tb drives from now on?
>
>

--
--tmac

RedHat Certified Engineer #804006984323821 (RHEL4)
RedHat Certified Engineer #805007643429572 (RHEL5)

Principal Consultant

[No More Linux!]

the point was that we needed 10TB of space on a new filer and a single 1TB drive shelf provides this but pretty much violates the redundancy and performance benefits of spreading load across multiple drives/shelves/controllers in an aggregate.

so to even make it somewhat more appropriate we would have to buy 2 shelves so the aggregate load would be more adequately spread but we still run up across the 16TB aggregate limit.

i guess the issue is with 1TB drives the whole 16TB aggregate limit is just plain silly and ridiculous.

maybe we should just go with the 500GB sata shelves and spread the load across more drives.   im not sure i want to confine my aggregates to just 8 to 12 drives max.

On Thu, Jun 5, 2008 at 12:55 PM, tmac <tmacmd@...> wrote:

Let ontap figure where to put things.

Besides,. the recommended size is 14 for SATA and 16 for FC per raid group.
Obviously, if you want to MAX out an aggregate at 16TB, you have to
take into consideration
you cannot do it in a single RAID group (well, I think you can, but it
is not recommended!!).
Therefore, your raid groups may look like 8+8 (maybe even 9+9), but
you probably do not
want 14+2 or 14+4.

Either way, you are eating the parity drives. Might as well make them
balanced groups.

Oh, one more thing. Let ONTAP pick which disks to use. It will
intelligently pick across shelves and controllers.

--tmac

On Thu, Jun 5, 2008 at 3:09 PM, No More Linux! <no.more.linux@...> wrote:
>
> is the raid group recommended size still 16 with the new 1TB sata drives?
> basically we are pretty much limited to one shelf per aggregate--not sure
> how this is good thing.
>
> is the 16tb limit going away anytime soon?  it is becoming increasingly too
> restrictive as these larger drive sizes are introduced.   or am i just
> supposed to create one shelf aggregates of 1tb drives from now on?
>
>

--
--tmac

RedHat Certified Engineer #804006984323821 (RHEL4)
RedHat Certified Engineer #805007643429572 (RHEL5)

Principal Consultant

[tmac-3]

Use multi-pathing.

1 full shelf, 4 FC controllers
Check the storage guide for configuration, but essentially, you connect one fc to the IN and another to the OUT. Do this for both A/B loops.

Now with 1 shelf, you have 4 paths to each disk and the filer should split between all 4 giving pretty good perfoamce from just 1 shelf.

Of couse, you can add more shelves as well.

--tmac
Sent from my Verizon Wireless BlackBerry

-----Original Message-----
From: "No More Linux!" <no.more.linux@...>

Date: Thu, 5 Jun 2008 14:47:13
To:tmac <tmacmd@...>
Cc:"toasters@..." <toasters@...>
Subject: Re: 1TB SATA, raid group size, 16TB limits etc


the point was that we needed 10TB of space on a new filer and a single 1TB drive shelf provides this but pretty much violates the redundancy and performance benefits of spreading load across multiple drives/shelves/controllers in an aggregate.
 
so to even make it somewhat more appropriate we would have to buy 2 shelves so the aggregate load would be more adequately spread but we still run up across the 16TB aggregate limit.

i guess the issue is with 1TB drives the whole 16TB aggregate limit is just plain silly and ridiculous.
 
maybe we should just go with the 500GB sata shelves and spread the load across more drives.   im not sure i want to confine my aggregates to just 8 to 12 drives max.




On Thu, Jun 5, 2008 at 12:55 PM, tmac <tmacmd@... <mailto:tmacmd@...> > wrote:
 Let ontap figure where to put things.
 
 Besides,. the recommended size is 14 for SATA and 16 for FC per raid group.
 Obviously, if you want to MAX out an aggregate at 16TB, you have to
 take into consideration
 you cannot do it in a single RAID group (well, I think you can, but it
 is not recommended!!).
 Therefore, your raid groups may look like 8+8 (maybe even 9+9), but
 you probably do not
 want 14+2 or 14+4.
 
 Either way, you are eating the parity drives. Might as well make them
 balanced groups.
 
 Oh, one more thing. Let ONTAP pick which disks to use. It will
 intelligently pick across shelves and controllers.
 
 --tmac
 



 On Thu, Jun 5, 2008 at 3:09 PM, No More Linux! <no.more.linux@... <mailto:no.more.linux@...> > wrote:
 >
 > is the raid group recommended size still 16 with the new 1TB sata drives?
 > basically we are pretty much limited to one shelf per aggregate--not sure
 > how this is good thing.
 >
 > is the 16tb limit going away anytime soon?  it is becoming increasingly too
 > restrictive as these larger drive sizes are introduced.   or am i just
 > supposed to create one shelf aggregates of 1tb drives from now on?
 >
 >
 
 
 
 --
 --tmac
 
 RedHat Certified Engineer #804006984323821 (RHEL4)
 RedHat Certified Engineer #805007643429572 (RHEL5)
 
 Principal Consultant
 
 

[Glenn Walker]

Some parts of this message have been removed. Learn more about Nabble's security policy.

We have stuck with the 500GB drives for that reason so far – however, over the next few years I’m sure there will be no choice.  By then, the rules will likely have changed (as per Blake’s email).

 

While the default is 12D+2P on ATA, we’ve been consistently using 14D+2P to make it match our FC config and allow us to have a relatively standard approach to calculating available storage when we go to purchase stuff.  Good luck so far, but a bit hesitant to push that luck on 1TB drives.

 

Just make sure you stick to Raid-DP – we don’t have anything in our environment that isn’t J

 

Glenn

 

---

From: owner-toasters@... [mailto:owner-toasters@...] On Behalf Of No More Linux!
Sent: Thursday, June 05, 2008 5:47 PM
To: tmac
Cc: toasters@...
Subject: Re: 1TB SATA, raid group size, 16TB limits etc

 

the point was that we needed 10TB of space on a new filer and a single 1TB drive shelf provides this but pretty much violates the redundancy and performance benefits of spreading load across multiple drives/shelves/controllers in an aggregate.

so to even make it somewhat more appropriate we would have to buy 2 shelves so the aggregate load would be more adequately spread but we still run up across the 16TB aggregate limit.

i guess the issue is with 1TB drives the whole 16TB aggregate limit is just plain silly and ridiculous.

maybe we should just go with the 500GB sata shelves and spread the load across more drives.   im not sure i want to confine my aggregates to just 8 to 12 drives max.

On Thu, Jun 5, 2008 at 12:55 PM, tmac <tmacmd@...> wrote:

Let ontap figure where to put things.

Besides,. the recommended size is 14 for SATA and 16 for FC per raid group.
Obviously, if you want to MAX out an aggregate at 16TB, you have to
take into consideration
you cannot do it in a single RAID group (well, I think you can, but it
is not recommended!!).
Therefore, your raid groups may look like 8+8 (maybe even 9+9), but
you probably do not
want 14+2 or 14+4.

Either way, you are eating the parity drives. Might as well make them
balanced groups.

Oh, one more thing. Let ONTAP pick which disks to use. It will
intelligently pick across shelves and controllers.

--tmac

On Thu, Jun 5, 2008 at 3:09 PM, No More Linux! <no.more.linux@...> wrote:
>
> is the raid group recommended size still 16 with the new 1TB sata drives?
> basically we are pretty much limited to one shelf per aggregate--not sure
> how this is good thing.
>
> is the 16tb limit going away anytime soon?  it is becoming increasingly too
> restrictive as these larger drive sizes are introduced.   or am i just
> supposed to create one shelf aggregates of 1tb drives from now on?
>
>

--
--tmac

RedHat Certified Engineer #804006984323821 (RHEL4)
RedHat Certified Engineer #805007643429572 (RHEL5)

Principal Consultant