HP Proliant DL360 G7

[Anh Quach]

Hello,

Has anyone out there been able to qualify the Proliant DL360 G7 for your Solaris/OI/Nexenta environments? Any pros/cons/gotchas (vs. previous generation HP servers) would be greatly appreciated.

Thanks in advance!

-Anh

[Edmund White]

This depends upon what you want to do. I've used G6 and G7 ProLiants
extensively in ZFS deployments (Nexenta, mostly). I'm assuming you'd be
using an external JBOD enclosure?

All works well. I disable the onboard Smart Array P410 RAID controller and
replace it with an LSI SAS HBA. If using internal disks, I'll use the
9211-8i. If external, the 9205-8e. Or sometimes, both.

I think the DL380 G7 is a better choice for PCIe flexibility, though. The
DL360 is pretty limited in expansion space.
--
Edmund White

Post by Anh Quach
Hello,
Has anyone out there been able to qualify the Proliant DL360 G7 for your
Solaris/OI/Nexenta environments? Any pros/cons/gotchas (vs. previous
generation HP servers) would be greatly appreciated.
Thanks in advance!
-Anh
_______________________________________________
zfs-discuss mailing list
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

[Anh Quach]

Yes, planning on attaching multiple DataOn JBODs.

When you say you've been replacing the onboard Smart Array controller (in external disk setups), do you also mean that your root pool is configured on the JBOD(s), essentially completely bypassing any of the built-in drive bays?

Thanks, Edmund!

-Anh

Post by Edmund White
This depends upon what you want to do. I've used G6 and G7 ProLiants
extensively in ZFS deployments (Nexenta, mostly). I'm assuming you'd be
using an external JBOD enclosure?
All works well. I disable the onboard Smart Array P410 RAID controller and
replace it with an LSI SAS HBA. If using internal disks, I'll use the
9211-8i. If external, the 9205-8e. Or sometimes, both.
I think the DL380 G7 is a better choice for PCIe flexibility, though. The
DL360 is pretty limited in expansion space.
--
Edmund White

Post by Anh Quach
Hello,
Has anyone out there been able to qualify the Proliant DL360 G7 for your
Solaris/OI/Nexenta environments? Any pros/cons/gotchas (vs. previous
generation HP servers) would be greatly appreciated.
Thanks in advance!
-Anh
_______________________________________________
zfs-discuss mailing list
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

[Edmund White]

No, I'll install using the internal drive bays mapped to an LSI
controller. Using the internal disks bays is also handy for ZIL/L2ARC
devices.
--
Ed

Post by Anh Quach
Yes, planning on attaching multiple DataOn JBODs.
When you say you've been replacing the onboard Smart Array controller (in
external disk setups), do you also mean that your root pool is configured
on the JBOD(s), essentially completely bypassing any of the built-in
drive bays?
Thanks, Edmund!
-Anh

Post by Edmund White
This depends upon what you want to do. I've used G6 and G7 ProLiants
extensively in ZFS deployments (Nexenta, mostly). I'm assuming you'd be
using an external JBOD enclosure?
All works well. I disable the onboard Smart Array P410 RAID controller and
replace it with an LSI SAS HBA. If using internal disks, I'll use the
9211-8i. If external, the 9205-8e. Or sometimes, both.
I think the DL380 G7 is a better choice for PCIe flexibility, though. The
DL360 is pretty limited in expansion space.
--
Edmund White

Post by Anh Quach
Hello,
Has anyone out there been able to qualify the Proliant DL360 G7 for your
Solaris/OI/Nexenta environments? Any pros/cons/gotchas (vs. previous
generation HP servers) would be greatly appreciated.
Thanks in advance!
-Anh
_______________________________________________
zfs-discuss mailing list
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

[Richard Elling]

Post by Edmund White
This depends upon what you want to do. I've used G6 and G7 ProLiants
extensively in ZFS deployments (Nexenta, mostly). I'm assuming you'd be
using an external JBOD enclosure?

When I was at Nexenta, we qualed the DL380 G7, D2600, and D2700.
These are some of the better boxes on the market.

Post by Edmund White
All works well. I disable the onboard Smart Array P410 RAID controller and
replace it with an LSI SAS HBA. If using internal disks, I'll use the
9211-8i. If external, the 9205-8e. Or sometimes, both.

FYI, HP also sells an 8-port IT-style HBA (SC-08Ge), but it is hard to locate
with their configurators. There might be a more modern equivalent cleverly
hidden somewhere difficult to find.
-- richard

--
ZFS Performance and Training
***@RichardElling.com
[phone]

[Anh Quach]

Thanks Richard and Edmund for the input.

Looking at the DL380 now...

-Anh

Post by Richard Elling

Post by Edmund White
This depends upon what you want to do. I've used G6 and G7 ProLiants
extensively in ZFS deployments (Nexenta, mostly). I'm assuming you'd be
using an external JBOD enclosure?

When I was at Nexenta, we qualed the DL380 G7, D2600, and D2700.
These are some of the better boxes on the market.

Post by Edmund White
All works well. I disable the onboard Smart Array P410 RAID controller and
replace it with an LSI SAS HBA. If using internal disks, I'll use the
9211-8i. If external, the 9205-8e. Or sometimes, both.

FYI, HP also sells an 8-port IT-style HBA (SC-08Ge), but it is hard to locate
with their configurators. There might be a more modern equivalent cleverly
hidden somewhere difficult to find.
-- richard
--
ZFS Performance and Training
[phone]
_______________________________________________
zfs-discuss mailing list
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

[mark]

FYI, HP also sells an 8-port IT-style HBA (SC-08Ge), but it is hard to locate 
with their configurators. There might be a more modern equivalent cleverly
hidden somewhere difficult to find.
 -- richard

Richard,

Do you know if the HBAs in HP controllers be swapped out with any well
characterized (by nexenta) HBAs like the 9211-8e or do they require a specific
'controller HBA' like the SC-08Ge? IE, does it void the warranty if you open up
the controller and stick a third party card in there? Did you ever try to
'bypass' the controllers at all and just plug into an expander? I prefer HP
hardware also but the controller is getting in the way.

Ill be asking HP the same questions in the next few weeks with any luck but your
opinion and experiences are on another level compared to HPs pre-sales
department... not that theyre bad but in this realm youre the man :)

Thanks,
Mark

[Sašo Kiselkov]

Post by mark

Post by Richard Elling
FYI, HP also sells an 8-port IT-style HBA (SC-08Ge), but it is hard to locate
with their configurators. There might be a more modern equivalent cleverly
hidden somewhere difficult to find.
-- richard

Richard,
Do you know if the HBAs in HP controllers be swapped out with any well
characterized (by nexenta) HBAs like the 9211-8e or do they require a specific
'controller HBA' like the SC-08Ge? IE, does it void the warranty if you open up
the controller and stick a third party card in there? Did you ever try to
'bypass' the controllers at all and just plug into an expander? I prefer HP
hardware also but the controller is getting in the way.
Ill be asking HP the same questions in the next few weeks with any luck but your
opinion and experiences are on another level compared to HPs pre-sales
department... not that theyre bad but in this realm youre the man :)

I know you didn't ask me, but I can tell you my experience: it depends
on what you mean by "warranty". If you mean as in "warranty on sales of
goods" (as mandated by law), then no, sticking a different HBA in your
servers does not void your warranty (unless this is expressly labeled on
the product - manufacturers typically also put protective labels on
screws then).

When it comes to support services, though, such as phone support and
firmware updates, then yes, using a third-party HBA can make these
difficult and/or impossible. HP storage enclosure and drive firmware,
for example, can only be flashed through an HP-branded SmartArray card.

Depending on what software you are running on the machines it can make
no difference at all, or a lot of difference. For instance, if you're
running proprietary storage controller software on the server (think
something like NexentaStor, but from the HW vendor), then your custom
HBA might simply be flat out unsupported and the only response you'll
get from the vendor support team is "stick the card we shipped it with
back in". OTOH if you're running something not HW vendor-specific (like
the aforementioned NexentaStor, or any other Illumos variant), and the
HW vendor at least gives lip service to supporting your platform (always
tell the support folk you're running "Solaris"), then chances are that
your support contract will be just as valid as before. I've had drives
fail on Dell machines and each time support was happy when I just told
them "drive dead, running Solaris, here's the log output, send a new one
please".

Cheers,
--
Saso

[Mark -]

Good call Saso. Sigh... I guess I wait to hear from HP on supported IT
mode HBAs in their D2000s or other jbods.

Post by Richard Elling

Post by mark

Post by Richard Elling
FYI, HP also sells an 8-port IT-style HBA (SC-08Ge), but it is hard to

locate

Post by mark

Post by Richard Elling
with their configurators. There might be a more modern equivalent

cleverly

Post by mark

Post by Richard Elling
hidden somewhere difficult to find.
-- richard

Richard,
Do you know if the HBAs in HP controllers be swapped out with any well
characterized (by nexenta) HBAs like the 9211-8e or do they require a

specific

Post by mark
'controller HBA' like the SC-08Ge? IE, does it void the warranty if you

open up

Post by mark
the controller and stick a third party card in there? Did you ever try

to

Post by mark
'bypass' the controllers at all and just plug into an expander? I

prefer HP

Post by mark
hardware also but the controller is getting in the way.
Ill be asking HP the same questions in the next few weeks with any luck

but your

Post by mark
opinion and experiences are on another level compared to HPs pre-sales
department... not that theyre bad but in this realm youre the man :)

I know you didn't ask me, but I can tell you my experience: it depends
on what you mean by "warranty". If you mean as in "warranty on sales of
goods" (as mandated by law), then no, sticking a different HBA in your
servers does not void your warranty (unless this is expressly labeled on
the product - manufacturers typically also put protective labels on
screws then).
When it comes to support services, though, such as phone support and
firmware updates, then yes, using a third-party HBA can make these
difficult and/or impossible. HP storage enclosure and drive firmware,
for example, can only be flashed through an HP-branded SmartArray card.
Depending on what software you are running on the machines it can make
no difference at all, or a lot of difference. For instance, if you're
running proprietary storage controller software on the server (think
something like NexentaStor, but from the HW vendor), then your custom
HBA might simply be flat out unsupported and the only response you'll
get from the vendor support team is "stick the card we shipped it with
back in". OTOH if you're running something not HW vendor-specific (like
the aforementioned NexentaStor, or any other Illumos variant), and the
HW vendor at least gives lip service to supporting your platform (always
tell the support folk you're running "Solaris"), then chances are that
your support contract will be just as valid as before. I've had drives
fail on Dell machines and each time support was happy when I just told
them "drive dead, running Solaris, here's the log output, send a new one
please".
Cheers,
--
Saso

[Edmund White]

The D2600 and D2700 enclosures are fully supported as Nexenta JBODs.<http://serverfault.com/a/461977/13325> I run them in multiple production environments<http://flic.kr/p/dJWXBd>.<http://flic.kr/p/dJWXBd>
I *could* use an HP-branded LSI controller (SC08Ge<http://h18004.www1.hp.com/products/quickspecs/13082_na/13082_na.HTML>), but I prefer the higher performance of the LSI 9211 and 9205e HBA's.

I recently posted on Server Fault with the Nexenta console<http://serverfault.com/a/461977/13325> representation of the HP D2700 JBOD. It's already integrated with NexentaStor.

--
Edmund White
***@mac.com

From: Mark - <***@gmail.com<mailto:***@gmail.com>>
Date: Tuesday, January 8, 2013 12:09 PM
To: Sa¹o Kiselkov <***@gmail.com<mailto:***@gmail.com>>
Cc: "zfs-***@opensolaris.org<mailto:zfs-***@opensolaris.org>" <zfs-***@opensolaris.org<mailto:zfs-***@opensolaris.org>>
Subject: Re: [zfs-discuss] HP Proliant DL360 G7

Good call Saso. Sigh... I guess I wait to hear from HP on supported IT mode HBAs in their D2000s or other jbods.

Post by mark

Post by Richard Elling
FYI, HP also sells an 8-port IT-style HBA (SC-08Ge), but it is hard to locate
with their configurators. There might be a more modern equivalent cleverly
hidden somewhere difficult to find.
-- richard

Richard,
Do you know if the HBAs in HP controllers be swapped out with any well
characterized (by nexenta) HBAs like the 9211-8e or do they require a specific
'controller HBA' like the SC-08Ge? IE, does it void the warranty if you open up
the controller and stick a third party card in there? Did you ever try to
'bypass' the controllers at all and just plug into an expander? I prefer HP
hardware also but the controller is getting in the way.
Ill be asking HP the same questions in the next few weeks with any luck but your
opinion and experiences are on another level compared to HPs pre-sales
department... not that theyre bad but in this realm youre the man :)

I know you didn't ask me, but I can tell you my experience: it depends
on what you mean by "warranty". If you mean as in "warranty on sales of
goods" (as mandated by law), then no, sticking a different HBA in your
servers does not void your warranty (unless this is expressly labeled on
the product - manufacturers typically also put protective labels on
screws then).

When it comes to support services, though, such as phone support and
firmware updates, then yes, using a third-party HBA can make these
difficult and/or impossible. HP storage enclosure and drive firmware,
for example, can only be flashed through an HP-branded SmartArray card.

Depending on what software you are running on the machines it can make
no difference at all, or a lot of difference. For instance, if you're
running proprietary storage controller software on the server (think
something like NexentaStor, but from the HW vendor), then your custom
HBA might simply be flat out unsupported and the only response you'll
get from the vendor support team is "stick the card we shipped it with
back in". OTOH if you're running something not HW vendor-specific (like
the aforementioned NexentaStor, or any other Illumos variant), and the
HW vendor at least gives lip service to supporting your platform (always
tell the support folk you're running "Solaris"), then chances are that
your support contract will be just as valid as before. I've had drives
fail on Dell machines and each time support was happy when I just told
them "drive dead, running Solaris, here's the log output, send a new one
please".

Cheers,
--
Saso

[Richard Elling]

This post might be inappropriate. Click to display it.

[Tim Fletcher]

Post by Edmund White
The D2600 and D2700 enclosures are fully supported as Nexenta JBODs.
[1] I run them in multiple production environments [2]. [2]
I *could* use an HP-branded LSI controller (SC08Ge [3]), but I prefer
the higher performance of the LSI 9211 and 9205e HBA's.

The HP H221 is the newer SAS2008 based HBA that replaces the SC08Ge,
it's definitely a pure HBA as I have one but I don't have any external
disk shelves to test with currently.

http://h18004.www1.hp.com/products/quickspecs/14222_div/14222_div.html

[Edmund White]

This has been covered here, but it works. I typically use LSI-9211 and
9205 controller in G6 and G7 ProLiant systems for ZFS use. I usually
bypass the onboard controller. I don't use the HP expander card since the
LSI cards provide enough ports. That expander card is good for ZFS, though.

See: http://serverfault.com/a/398579/13325 and http://flic.kr/s/aHsjtyfUBB
--
Edmund White

Post by mark

Post by Richard Elling
FYI, HP also sells an 8-port IT-style HBA (SC-08Ge), but it is hard to
locate
with their configurators. There might be a more modern equivalent
cleverly
hidden somewhere difficult to find.
-- richard

Richard,
Do you know if the HBAs in HP controllers be swapped out with any well
characterized (by nexenta) HBAs like the 9211-8e or do they require a specific
'controller HBA' like the SC-08Ge? IE, does it void the warranty if you open up
the controller and stick a third party card in there? Did you ever try to
'bypass' the controllers at all and just plug into an expander? I prefer HP
hardware also but the controller is getting in the way.
Ill be asking HP the same questions in the next few weeks with any luck but your
opinion and experiences are on another level compared to HPs pre-sales
department... not that theyre bad but in this realm youre the man :)
Thanks,
Mark
_______________________________________________
zfs-discuss mailing list
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

[Anh Quach]

It's been about a year now and I figured I'd send out a little update for anyone curious.

For density and cost considerations, we ended up going with a custom built Supermicro "double sided storage" chassis, which can hold 72 drives in 4U. We initially deployed 10 raidz2 vdevs for about 40TB usable - the remaining 12 drive bays were used for cache, zil, spares. We have been very happy with this set up and are about to double the capacity with the JBOD version of the same Supermicro chassis.

I went with 3 LSI 9211-8i controllers to match the Supermicro's 3 backplanes.

In the testing phase, we evaluated Solaris 11 and Nexenta - ultimately we went with Nexenta due to the set of tools pre-written, although performance was slightly better on Solaris.

-Anh

Post by Anh Quach
Thanks Richard and Edmund for the input.
Looking at the DL380 now...
-Anh

Post by Richard Elling

Post by Edmund White
This depends upon what you want to do. I've used G6 and G7 ProLiants
extensively in ZFS deployments (Nexenta, mostly). I'm assuming you'd be
using an external JBOD enclosure?

When I was at Nexenta, we qualed the DL380 G7, D2600, and D2700.
These are some of the better boxes on the market.

Post by Edmund White
All works well. I disable the onboard Smart Array P410 RAID controller and
replace it with an LSI SAS HBA. If using internal disks, I'll use the
9211-8i. If external, the 9205-8e. Or sometimes, both.

FYI, HP also sells an 8-port IT-style HBA (SC-08Ge), but it is hard to locate
with their configurators. There might be a more modern equivalent cleverly
hidden somewhere difficult to find.
-- richard
--
ZFS Performance and Training
[phone]
_______________________________________________
zfs-discuss mailing list
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss