Discussion:
HP Proliant DL360 G7
(too old to reply)
Anh Quach
2012-07-02 21:29:44 UTC
Permalink
Hello,

Has anyone out there been able to qualify the Proliant DL360 G7 for your Solaris/OI/Nexenta environments? Any pros/cons/gotchas (vs. previous generation HP servers) would be greatly appreciated.

Thanks in advance!

-Anh
Edmund White
2012-07-02 21:40:57 UTC
Permalink
This depends upon what you want to do. I've used G6 and G7 ProLiants
extensively in ZFS deployments (Nexenta, mostly). I'm assuming you'd be
using an external JBOD enclosure?

All works well. I disable the onboard Smart Array P410 RAID controller and
replace it with an LSI SAS HBA. If using internal disks, I'll use the
9211-8i. If external, the 9205-8e. Or sometimes, both.

I think the DL380 G7 is a better choice for PCIe flexibility, though. The
DL360 is pretty limited in expansion space.
--
Edmund White
Post by Anh Quach
Hello,
Has anyone out there been able to qualify the Proliant DL360 G7 for your
Solaris/OI/Nexenta environments? Any pros/cons/gotchas (vs. previous
generation HP servers) would be greatly appreciated.
Thanks in advance!
-Anh
_______________________________________________
zfs-discuss mailing list
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Anh Quach
2012-07-02 22:07:30 UTC
Permalink
Yes, planning on attaching multiple DataOn JBODs.

When you say you've been replacing the onboard Smart Array controller (in external disk setups), do you also mean that your root pool is configured on the JBOD(s), essentially completely bypassing any of the built-in drive bays?

Thanks, Edmund!

-Anh
Post by Edmund White
This depends upon what you want to do. I've used G6 and G7 ProLiants
extensively in ZFS deployments (Nexenta, mostly). I'm assuming you'd be
using an external JBOD enclosure?
All works well. I disable the onboard Smart Array P410 RAID controller and
replace it with an LSI SAS HBA. If using internal disks, I'll use the
9211-8i. If external, the 9205-8e. Or sometimes, both.
I think the DL380 G7 is a better choice for PCIe flexibility, though. The
DL360 is pretty limited in expansion space.
--
Edmund White
Post by Anh Quach
Hello,
Has anyone out there been able to qualify the Proliant DL360 G7 for your
Solaris/OI/Nexenta environments? Any pros/cons/gotchas (vs. previous
generation HP servers) would be greatly appreciated.
Thanks in advance!
-Anh
_______________________________________________
zfs-discuss mailing list
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Edmund White
2012-07-02 22:17:33 UTC
Permalink
No, I'll install using the internal drive bays mapped to an LSI
controller. Using the internal disks bays is also handy for ZIL/L2ARC
devices.
--
Ed
Post by Anh Quach
Yes, planning on attaching multiple DataOn JBODs.
When you say you've been replacing the onboard Smart Array controller (in
external disk setups), do you also mean that your root pool is configured
on the JBOD(s), essentially completely bypassing any of the built-in
drive bays?
Thanks, Edmund!
-Anh
Post by Edmund White
This depends upon what you want to do. I've used G6 and G7 ProLiants
extensively in ZFS deployments (Nexenta, mostly). I'm assuming you'd be
using an external JBOD enclosure?
All works well. I disable the onboard Smart Array P410 RAID controller and
replace it with an LSI SAS HBA. If using internal disks, I'll use the
9211-8i. If external, the 9205-8e. Or sometimes, both.
I think the DL380 G7 is a better choice for PCIe flexibility, though. The
DL360 is pretty limited in expansion space.
--
Edmund White
Post by Anh Quach
Hello,
Has anyone out there been able to qualify the Proliant DL360 G7 for your
Solaris/OI/Nexenta environments? Any pros/cons/gotchas (vs. previous
generation HP servers) would be greatly appreciated.
Thanks in advance!
-Anh
_______________________________________________
zfs-discuss mailing list
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Richard Elling
2012-07-03 02:57:10 UTC
Permalink
Post by Edmund White
This depends upon what you want to do. I've used G6 and G7 ProLiants
extensively in ZFS deployments (Nexenta, mostly). I'm assuming you'd be
using an external JBOD enclosure?
When I was at Nexenta, we qualed the DL380 G7, D2600, and D2700.
These are some of the better boxes on the market.
Post by Edmund White
All works well. I disable the onboard Smart Array P410 RAID controller and
replace it with an LSI SAS HBA. If using internal disks, I'll use the
9211-8i. If external, the 9205-8e. Or sometimes, both.
FYI, HP also sells an 8-port IT-style HBA (SC-08Ge), but it is hard to locate
with their configurators. There might be a more modern equivalent cleverly
hidden somewhere difficult to find.
-- richard

--
ZFS Performance and Training
***@RichardElling.com
+1-760-896-4422
Anh Quach
2012-07-03 20:49:56 UTC
Permalink
Thanks Richard and Edmund for the input.

Looking at the DL380 now...

-Anh
Post by Richard Elling
Post by Edmund White
This depends upon what you want to do. I've used G6 and G7 ProLiants
extensively in ZFS deployments (Nexenta, mostly). I'm assuming you'd be
using an external JBOD enclosure?
When I was at Nexenta, we qualed the DL380 G7, D2600, and D2700.
These are some of the better boxes on the market.
Post by Edmund White
All works well. I disable the onboard Smart Array P410 RAID controller and
replace it with an LSI SAS HBA. If using internal disks, I'll use the
9211-8i. If external, the 9205-8e. Or sometimes, both.
FYI, HP also sells an 8-port IT-style HBA (SC-08Ge), but it is hard to locate
with their configurators. There might be a more modern equivalent cleverly
hidden somewhere difficult to find.
-- richard
--
ZFS Performance and Training
+1-760-896-4422
_______________________________________________
zfs-discuss mailing list
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
mark
2013-01-08 15:27:24 UTC
Permalink
FYI, HP also sells an 8-port IT-style HBA (SC-08Ge), but it is hard to locate 
with their configurators. There might be a more modern equivalent cleverly
hidden somewhere difficult to find.
 -- richard
Richard,

Do you know if the HBAs in HP controllers be swapped out with any well
characterized (by nexenta) HBAs like the 9211-8e or do they require a specific
'controller HBA' like the SC-08Ge? IE, does it void the warranty if you open up
the controller and stick a third party card in there? Did you ever try to
'bypass' the controllers at all and just plug into an expander? I prefer HP
hardware also but the controller is getting in the way.

Ill be asking HP the same questions in the next few weeks with any luck but your
opinion and experiences are on another level compared to HPs pre-sales
department... not that theyre bad but in this realm youre the man :)

Thanks,
Mark
Sašo Kiselkov
2013-01-08 16:40:00 UTC
Permalink
Post by mark
Post by Richard Elling
FYI, HP also sells an 8-port IT-style HBA (SC-08Ge), but it is hard to locate
with their configurators. There might be a more modern equivalent cleverly
hidden somewhere difficult to find.
-- richard
Richard,
Do you know if the HBAs in HP controllers be swapped out with any well
characterized (by nexenta) HBAs like the 9211-8e or do they require a specific
'controller HBA' like the SC-08Ge? IE, does it void the warranty if you open up
the controller and stick a third party card in there? Did you ever try to
'bypass' the controllers at all and just plug into an expander? I prefer HP
hardware also but the controller is getting in the way.
Ill be asking HP the same questions in the next few weeks with any luck but your
opinion and experiences are on another level compared to HPs pre-sales
department... not that theyre bad but in this realm youre the man :)
I know you didn't ask me, but I can tell you my experience: it depends
on what you mean by "warranty". If you mean as in "warranty on sales of
goods" (as mandated by law), then no, sticking a different HBA in your
servers does not void your warranty (unless this is expressly labeled on
the product - manufacturers typically also put protective labels on
screws then).

When it comes to support services, though, such as phone support and
firmware updates, then yes, using a third-party HBA can make these
difficult and/or impossible. HP storage enclosure and drive firmware,
for example, can only be flashed through an HP-branded SmartArray card.

Depending on what software you are running on the machines it can make
no difference at all, or a lot of difference. For instance, if you're
running proprietary storage controller software on the server (think
something like NexentaStor, but from the HW vendor), then your custom
HBA might simply be flat out unsupported and the only response you'll
get from the vendor support team is "stick the card we shipped it with
back in". OTOH if you're running something not HW vendor-specific (like
the aforementioned NexentaStor, or any other Illumos variant), and the
HW vendor at least gives lip service to supporting your platform (always
tell the support folk you're running "Solaris"), then chances are that
your support contract will be just as valid as before. I've had drives
fail on Dell machines and each time support was happy when I just told
them "drive dead, running Solaris, here's the log output, send a new one
please".

Cheers,
--
Saso
Mark -
2013-01-08 18:09:56 UTC
Permalink
Good call Saso. Sigh... I guess I wait to hear from HP on supported IT
mode HBAs in their D2000s or other jbods.
Post by Richard Elling
Post by mark
Post by Richard Elling
FYI, HP also sells an 8-port IT-style HBA (SC-08Ge), but it is hard to
locate
Post by mark
Post by Richard Elling
with their configurators. There might be a more modern equivalent
cleverly
Post by mark
Post by Richard Elling
hidden somewhere difficult to find.
-- richard
Richard,
Do you know if the HBAs in HP controllers be swapped out with any well
characterized (by nexenta) HBAs like the 9211-8e or do they require a
specific
Post by mark
'controller HBA' like the SC-08Ge? IE, does it void the warranty if you
open up
Post by mark
the controller and stick a third party card in there? Did you ever try
to
Post by mark
'bypass' the controllers at all and just plug into an expander? I
prefer HP
Post by mark
hardware also but the controller is getting in the way.
Ill be asking HP the same questions in the next few weeks with any luck
but your
Post by mark
opinion and experiences are on another level compared to HPs pre-sales
department... not that theyre bad but in this realm youre the man :)
I know you didn't ask me, but I can tell you my experience: it depends
on what you mean by "warranty". If you mean as in "warranty on sales of
goods" (as mandated by law), then no, sticking a different HBA in your
servers does not void your warranty (unless this is expressly labeled on
the product - manufacturers typically also put protective labels on
screws then).
When it comes to support services, though, such as phone support and
firmware updates, then yes, using a third-party HBA can make these
difficult and/or impossible. HP storage enclosure and drive firmware,
for example, can only be flashed through an HP-branded SmartArray card.
Depending on what software you are running on the machines it can make
no difference at all, or a lot of difference. For instance, if you're
running proprietary storage controller software on the server (think
something like NexentaStor, but from the HW vendor), then your custom
HBA might simply be flat out unsupported and the only response you'll
get from the vendor support team is "stick the card we shipped it with
back in". OTOH if you're running something not HW vendor-specific (like
the aforementioned NexentaStor, or any other Illumos variant), and the
HW vendor at least gives lip service to supporting your platform (always
tell the support folk you're running "Solaris"), then chances are that
your support contract will be just as valid as before. I've had drives
fail on Dell machines and each time support was happy when I just told
them "drive dead, running Solaris, here's the log output, send a new one
please".
Cheers,
--
Saso
Edmund White
2013-01-08 18:30:33 UTC
Permalink
The D2600 and D2700 enclosures are fully supported as Nexenta JBODs.<http://serverfault.com/a/461977/13325> I run them in multiple production environments<http://flic.kr/p/dJWXBd>.<http://flic.kr/p/dJWXBd>
I *could* use an HP-branded LSI controller (SC08Ge<http://h18004.www1.hp.com/products/quickspecs/13082_na/13082_na.HTML>), but I prefer the higher performance of the LSI 9211 and 9205e HBA's.

I recently posted on Server Fault with the Nexenta console<http://serverfault.com/a/461977/13325> representation of the HP D2700 JBOD. It's already integrated with NexentaStor.

--
Edmund White
***@mac.com

From: Mark - <***@gmail.com<mailto:***@gmail.com>>
Date: Tuesday, January 8, 2013 12:09 PM
To: Sa¹o Kiselkov <***@gmail.com<mailto:***@gmail.com>>
Cc: "zfs-***@opensolaris.org<mailto:zfs-***@opensolaris.org>" <zfs-***@opensolaris.org<mailto:zfs-***@opensolaris.org>>
Subject: Re: [zfs-discuss] HP Proliant DL360 G7

Good call Saso. Sigh... I guess I wait to hear from HP on supported IT mode HBAs in their D2000s or other jbods.
Post by mark
Post by Richard Elling
FYI, HP also sells an 8-port IT-style HBA (SC-08Ge), but it is hard to locate
with their configurators. There might be a more modern equivalent cleverly
hidden somewhere difficult to find.
-- richard
Richard,
Do you know if the HBAs in HP controllers be swapped out with any well
characterized (by nexenta) HBAs like the 9211-8e or do they require a specific
'controller HBA' like the SC-08Ge? IE, does it void the warranty if you open up
the controller and stick a third party card in there? Did you ever try to
'bypass' the controllers at all and just plug into an expander? I prefer HP
hardware also but the controller is getting in the way.
Ill be asking HP the same questions in the next few weeks with any luck but your
opinion and experiences are on another level compared to HPs pre-sales
department... not that theyre bad but in this realm youre the man :)
I know you didn't ask me, but I can tell you my experience: it depends
on what you mean by "warranty". If you mean as in "warranty on sales of
goods" (as mandated by law), then no, sticking a different HBA in your
servers does not void your warranty (unless this is expressly labeled on
the product - manufacturers typically also put protective labels on
screws then).

When it comes to support services, though, such as phone support and
firmware updates, then yes, using a third-party HBA can make these
difficult and/or impossible. HP storage enclosure and drive firmware,
for example, can only be flashed through an HP-branded SmartArray card.

Depending on what software you are running on the machines it can make
no difference at all, or a lot of difference. For instance, if you're
running proprietary storage controller software on the server (think
something like NexentaStor, but from the HW vendor), then your custom
HBA might simply be flat out unsupported and the only response you'll
get from the vendor support team is "stick the card we shipped it with
back in". OTOH if you're running something not HW vendor-specific (like
the aforementioned NexentaStor, or any other Illumos variant), and the
HW vendor at least gives lip service to supporting your platform (always
tell the support folk you're running "Solaris"), then chances are that
your support contract will be just as valid as before. I've had drives
fail on Dell machines and each time support was happy when I just told
them "drive dead, running Solaris, here's the log output, send a new one
please".

Cheers,
--
Saso
Richard Elling
2013-01-09 05:56:45 UTC
Permalink
The D2600 and D2700 enclosures are fully supported as Nexenta JBODs. I run them in multiple production environments.
Yes, I worked on the field qualifications for these… very nice JBODs :-)
I *could* use an HP-branded LSI controller (SC08Ge), but I prefer the higher performance of the LSI 9211 and 9205e HBA's.
Many of the big-box vendors have to deal with Windows as the target OS. Until Server 2012,
the use of JBODs with lots of disks was challenging for Windows. Hence, they offer few
options for the folks who want JBOD control.
-- richard
I recently posted on Server Fault with the Nexenta console representation of the HP D2700 JBOD. It's already integrated with NexentaStor.
--
Edmund White
Date: Tuesday, January 8, 2013 12:09 PM
Subject: Re: [zfs-discuss] HP Proliant DL360 G7
Good call Saso. Sigh... I guess I wait to hear from HP on supported IT mode HBAs in their D2000s or other jbods.
Post by mark
Post by Richard Elling
FYI, HP also sells an 8-port IT-style HBA (SC-08Ge), but it is hard to locate
with their configurators. There might be a more modern equivalent cleverly
hidden somewhere difficult to find.
-- richard
Richard,
Do you know if the HBAs in HP controllers be swapped out with any well
characterized (by nexenta) HBAs like the 9211-8e or do they require a specific
'controller HBA' like the SC-08Ge? IE, does it void the warranty if you open up
the controller and stick a third party card in there? Did you ever try to
'bypass' the controllers at all and just plug into an expander? I prefer HP
hardware also but the controller is getting in the way.
Ill be asking HP the same questions in the next few weeks with any luck but your
opinion and experiences are on another level compared to HPs pre-sales
department... not that theyre bad but in this realm youre the man :)
I know you didn't ask me, but I can tell you my experience: it depends
on what you mean by "warranty". If you mean as in "warranty on sales of
goods" (as mandated by law), then no, sticking a different HBA in your
servers does not void your warranty (unless this is expressly labeled on
the product - manufacturers typically also put protective labels on
screws then).
When it comes to support services, though, such as phone support and
firmware updates, then yes, using a third-party HBA can make these
difficult and/or impossible. HP storage enclosure and drive firmware,
for example, can only be flashed through an HP-branded SmartArray card.
Depending on what software you are running on the machines it can make
no difference at all, or a lot of difference. For instance, if you're
running proprietary storage controller software on the server (think
something like NexentaStor, but from the HW vendor), then your custom
HBA might simply be flat out unsupported and the only response you'll
get from the vendor support team is "stick the card we shipped it with
back in". OTOH if you're running something not HW vendor-specific (like
the aforementioned NexentaStor, or any other Illumos variant), and the
HW vendor at least gives lip service to supporting your platform (always
tell the support folk you're running "Solaris"), then chances are that
your support contract will be just as valid as before. I've had drives
fail on Dell machines and each time support was happy when I just told
them "drive dead, running Solaris, here's the log output, send a new one
please".
Cheers,
--
Saso
_______________________________________________
zfs-discuss mailing list
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
--
ZFS and performance consulting
http://www.RichardElling.com
Tim Fletcher
2013-01-09 12:40:17 UTC
Permalink
Post by Edmund White
The D2600 and D2700 enclosures are fully supported as Nexenta JBODs.
[1] I run them in multiple production environments [2]. [2]
I *could* use an HP-branded LSI controller (SC08Ge [3]), but I prefer
the higher performance of the LSI 9211 and 9205e HBA's.
The HP H221 is the newer SAS2008 based HBA that replaces the SC08Ge,
it's definitely a pure HBA as I have one but I don't have any external
disk shelves to test with currently.

http://h18004.www1.hp.com/products/quickspecs/14222_div/14222_div.html
--
Tim Fletcher
<***@night-shade.org.uk>
Edmund White
2013-01-08 16:41:22 UTC
Permalink
This has been covered here, but it works. I typically use LSI-9211 and
9205 controller in G6 and G7 ProLiant systems for ZFS use. I usually
bypass the onboard controller. I don't use the HP expander card since the
LSI cards provide enough ports. That expander card is good for ZFS, though.

See: http://serverfault.com/a/398579/13325 and http://flic.kr/s/aHsjtyfUBB
--
Edmund White
Post by mark
Post by Richard Elling
FYI, HP also sells an 8-port IT-style HBA (SC-08Ge), but it is hard to
locate
with their configurators. There might be a more modern equivalent
cleverly
hidden somewhere difficult to find.
-- richard
Richard,
Do you know if the HBAs in HP controllers be swapped out with any well
characterized (by nexenta) HBAs like the 9211-8e or do they require a specific
'controller HBA' like the SC-08Ge? IE, does it void the warranty if you open up
the controller and stick a third party card in there? Did you ever try to
'bypass' the controllers at all and just plug into an expander? I prefer HP
hardware also but the controller is getting in the way.
Ill be asking HP the same questions in the next few weeks with any luck but your
opinion and experiences are on another level compared to HPs pre-sales
department... not that theyre bad but in this realm youre the man :)
Thanks,
Mark
_______________________________________________
zfs-discuss mailing list
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Anh Quach
2013-01-31 01:44:09 UTC
Permalink
It's been about a year now and I figured I'd send out a little update for anyone curious.

For density and cost considerations, we ended up going with a custom built Supermicro "double sided storage" chassis, which can hold 72 drives in 4U. We initially deployed 10 raidz2 vdevs for about 40TB usable - the remaining 12 drive bays were used for cache, zil, spares. We have been very happy with this set up and are about to double the capacity with the JBOD version of the same Supermicro chassis.

I went with 3 LSI 9211-8i controllers to match the Supermicro's 3 backplanes.

In the testing phase, we evaluated Solaris 11 and Nexenta - ultimately we went with Nexenta due to the set of tools pre-written, although performance was slightly better on Solaris.

-Anh
Post by Anh Quach
Thanks Richard and Edmund for the input.
Looking at the DL380 now...
-Anh
Post by Richard Elling
Post by Edmund White
This depends upon what you want to do. I've used G6 and G7 ProLiants
extensively in ZFS deployments (Nexenta, mostly). I'm assuming you'd be
using an external JBOD enclosure?
When I was at Nexenta, we qualed the DL380 G7, D2600, and D2700.
These are some of the better boxes on the market.
Post by Edmund White
All works well. I disable the onboard Smart Array P410 RAID controller and
replace it with an LSI SAS HBA. If using internal disks, I'll use the
9211-8i. If external, the 9205-8e. Or sometimes, both.
FYI, HP also sells an 8-port IT-style HBA (SC-08Ge), but it is hard to locate
with their configurators. There might be a more modern equivalent cleverly
hidden somewhere difficult to find.
-- richard
--
ZFS Performance and Training
+1-760-896-4422
_______________________________________________
zfs-discuss mailing list
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Loading...