Discussion:
Thumper (X4500), and CF SSD for L2ARC = ?
Jim Klimov
2011-10-14 12:39:07 UTC
Permalink
Hello, I was asked if the CF port in Thumpers can be accessed by the OS?
In particular, would it be a good idea to use a modern 600x CF card
(some reliable one intended for professional photography) as an L2ARC
device using this port?

Thanks,
//Jim
Darren J Moffat
2011-10-14 12:49:40 UTC
Permalink
Post by Jim Klimov
Hello, I was asked if the CF port in Thumpers can be accessed by the OS?
In particular, would it be a good idea to use a modern 600x CF card
(some reliable one intended for professional photography) as an L2ARC
device using this port?
I don't know about the Thumpers internal CF slot.

I can say I have tried using a fast (at the time, this was about 3 years
ago) CF card via a CF to IDE adaptor before and it turned out to be a
really bad idea because the spinning rust disk (which was SATA) was
actually faster to access. Same went for USB to CF adaptors at the time
too.
--
Darren J Moffat
Erik Trimble
2011-10-14 18:25:22 UTC
Permalink
Post by Darren J Moffat
Post by Jim Klimov
Hello, I was asked if the CF port in Thumpers can be accessed by the OS?
In particular, would it be a good idea to use a modern 600x CF card
(some reliable one intended for professional photography) as an L2ARC
device using this port?
I don't know about the Thumpers internal CF slot.
I can say I have tried using a fast (at the time, this was about 3
years ago) CF card via a CF to IDE adaptor before and it turned out to
be a really bad idea because the spinning rust disk (which was SATA)
was actually faster to access. Same went for USB to CF adaptors at
the time too.
Last I'd checked, the CF port was fully functional.

However, I'd not use it as L2ARC (and, certainly not ZIL). CF is not
good in terms of either random write or read - professional-grade CF
cards are optimized for STREAMING write - that is, the ability to write
a big-ass JPG or BMP or TIFF as quickly as possible. The CF controller
isn't good on lots of little read/write ops.

In Casper's case, the CF->IDE adapter makes this even worse, since IDE
is spectacularly bad at IOPS.

I can't remember - does the X4500 have any extra SATA ports free on the
motherboard? And, does it have any extra HD power connectors?

http://www.amazon.com/dp/B002MWDRD6/ref=asc_df_B002MWDRD61280186?smid=A2YLYLTN75J8LR&tag=shopzilla_mp_1382-20&linkCode=asn&creative=395105&creativeASIN=B002MWDRD6

Is a great way to add a 2.5" drive slot, but it's just a physical slot
adapter - you need to attach a standard SATA cable and HD power
connector to it.


If that's not an option, find yourself a cheap PCI-E adapter with eSATA
ports on it, then use an external HD enclosure with eSATA for a small SSD.


As a last resort, remove one of the 3.5" SATA drives, and put in an SSD
in a 2.5"->3.5" converter enclosure.

Remember, you can generally get by fine with a lower-end SSD as L2ARC,
so a 60GB SSD should be $100 or less.

-Erik
Gregory Shaw
2011-10-14 19:57:08 UTC
Permalink
You might want to keep in mind that the X4500 was a ~2006 box, and had only PCI-X slots.

Or, at least, that's what the 3 Iv'e got have. I think the X4540 had PCIe, but I never got one of those. :-(

I haven't seen any cache accelerator PCI-X cards.

However, what I've done on the X4500 systems in the lab is to use two drives on the system disk bus for the cache and log devices (each).

With the 175 release of Solaris 11, I have literally seen a scrub running at 960mb/sec, and around 400mb/sec for 10ge NFS.
Post by Erik Trimble
Post by Darren J Moffat
Post by Jim Klimov
Hello, I was asked if the CF port in Thumpers can be accessed by the OS?
In particular, would it be a good idea to use a modern 600x CF card
(some reliable one intended for professional photography) as an L2ARC
device using this port?
I don't know about the Thumpers internal CF slot.
I can say I have tried using a fast (at the time, this was about 3 years ago) CF card via a CF to IDE adaptor before and it turned out to be a really bad idea because the spinning rust disk (which was SATA) was actually faster to access. Same went for USB to CF adaptors at the time too.
Last I'd checked, the CF port was fully functional.
However, I'd not use it as L2ARC (and, certainly not ZIL). CF is not good in terms of either random write or read - professional-grade CF cards are optimized for STREAMING write - that is, the ability to write a big-ass JPG or BMP or TIFF as quickly as possible. The CF controller isn't good on lots of little read/write ops.
In Casper's case, the CF->IDE adapter makes this even worse, since IDE is spectacularly bad at IOPS.
I can't remember - does the X4500 have any extra SATA ports free on the motherboard? And, does it have any extra HD power connectors?
http://www.amazon.com/dp/B002MWDRD6/ref=asc_df_B002MWDRD61280186?smid=A2YLYLTN75J8LR&tag=shopzilla_mp_1382-20&linkCode=asn&creative=395105&creativeASIN=B002MWDRD6
Is a great way to add a 2.5" drive slot, but it's just a physical slot adapter - you need to attach a standard SATA cable and HD power connector to it.
If that's not an option, find yourself a cheap PCI-E adapter with eSATA ports on it, then use an external HD enclosure with eSATA for a small SSD.
As a last resort, remove one of the 3.5" SATA drives, and put in an SSD in a 2.5"->3.5" converter enclosure.
Remember, you can generally get by fine with a lower-end SSD as L2ARC, so a 60GB SSD should be $100 or less.
-Erik
_______________________________________________
zfs-discuss mailing list
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
-----
Gregory Shaw, Enterprise IT Architect
Phone: (303) 246-5411
Oracle Global IT Service Design Group
500 Eldorado Blvd, UBRM02-157 ***@oracle.com (work)
Broomfield, CO 80021 ***@fmsoft.com (home)
Hoping the problem magically goes away by ignoring it is the "microsoft approach to programming" and should never be allowed. (Linus Torvalds)
Jim Klimov
2011-10-14 20:54:29 UTC
Permalink
Post by Gregory Shaw
You might want to keep in mind that the X4500 was a ~2006 box, and had only PCI-X slots.
Or, at least, that's what the 3 Iv'e got have. I think the X4540 had PCIe, but I never got one of those. :-(
I haven't seen any cache accelerator PCI-X cards.
However, what I've done on the X4500 systems in the lab is to use two drives on the system disk bus for the cache and log devices (each).
So you have 44 data drives, 2 os drives and 2 zil/cache devices?
And what do you use for zil/cache? SSDs? Specific ones?
Post by Gregory Shaw
With the 175 release of Solaris 11, I have literally seen a scrub running at 960mb/sec, and around 400mb/sec for 10ge NFS.
Hmm, and where can you get that release in the open? ;)
Gregory Shaw
2011-10-14 21:35:47 UTC
Permalink
Here's the zpool layout. You don't really have a choice on the boot
volume -- the system supports only two drives on the same chain. The
remaining drives are as shown:

pool: internal
state: ONLINE
scan: scrub repaired 0 in 0h0m with 0 errors on Sat Oct 8 21:15:42 2011
config:

NAME STATE READ WRITE CKSUM
internal ONLINE 0 0 0
raidz1-0 ONLINE 0 0 0
c3t0d0 ONLINE 0 0 0
c4t0d0 ONLINE 0 0 0
c6t0d0 ONLINE 0 0 0
c8t0d0 ONLINE 0 0 0
c9t0d0 ONLINE 0 0 0
raidz1-1 ONLINE 0 0 0
c3t1d0 ONLINE 0 0 0
c4t1d0 ONLINE 0 0 0
c6t1d0 ONLINE 0 0 0
c8t1d0 ONLINE 0 0 0
c9t1d0 ONLINE 0 0 0
raidz1-2 ONLINE 0 0 0
c3t2d0 ONLINE 0 0 0
c4t2d0 ONLINE 0 0 0
c6t2d0 ONLINE 0 0 0
c8t2d0 ONLINE 0 0 0
c9t2d0 ONLINE 0 0 0
raidz1-3 ONLINE 0 0 0
c3t3d0 ONLINE 0 0 0
c4t3d0 ONLINE 0 0 0
c6t3d0 ONLINE 0 0 0
c8t3d0 ONLINE 0 0 0
c9t3d0 ONLINE 0 0 0
raidz1-4 ONLINE 0 0 0
c3t4d0 ONLINE 0 0 0
c4t4d0 ONLINE 0 0 0
c6t4d0 ONLINE 0 0 0
c8t4d0 ONLINE 0 0 0
c9t4d0 ONLINE 0 0 0
raidz1-5 ONLINE 0 0 0
c3t5d0 ONLINE 0 0 0
c4t5d0 ONLINE 0 0 0
c6t5d0 ONLINE 0 0 0
c8t5d0 ONLINE 0 0 0
c9t5d0 ONLINE 0 0 0
raidz1-6 ONLINE 0 0 0
c3t6d0 ONLINE 0 0 0
c4t6d0 ONLINE 0 0 0
c6t6d0 ONLINE 0 0 0
c8t6d0 ONLINE 0 0 0
c9t6d0 ONLINE 0 0 0
raidz1-7 ONLINE 0 0 0
c3t7d0 ONLINE 0 0 0
c4t7d0 ONLINE 0 0 0
c6t7d0 ONLINE 0 0 0
c8t7d0 ONLINE 0 0 0
c9t7d0 ONLINE 0 0 0
logs
c7t2d0 ONLINE 0 0 0
c7t6d0 ONLINE 0 0 0
spares
c7t1d0 AVAIL
c7t5d0 AVAIL
c7t3d0 AVAIL

pool: rpool
state: ONLINE
scan: scrub repaired 0 in 0h6m with 0 errors on Sat Oct 8 21:21:54 2011
config:

NAME STATE READ WRITE CKSUM
rpool ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
c7t0d0s0 ONLINE 0 0 0
c7t4d0s0 ONLINE 0 0 0

errors: No known data errors

It appears I mis-spoke on the caches. I thought I'd used two drives
for cache, but apparently not.

And Solaris 11 is supposed to be out Real Soon Now. :-)
Post by Jim Klimov
Post by Gregory Shaw
You might want to keep in mind that the X4500 was a ~2006 box, and had only PCI-X slots.
Or, at least, that's what the 3 Iv'e got have. I think the X4540 had
PCIe, but I never got one of those. :-(
I haven't seen any cache accelerator PCI-X cards.
However, what I've done on the X4500 systems in the lab is to use two
drives on the system disk bus for the cache and log devices (each).
So you have 44 data drives, 2 os drives and 2 zil/cache devices?
And what do you use for zil/cache? SSDs? Specific ones?
Post by Gregory Shaw
With the 175 release of Solaris 11, I have literally seen a scrub
running at 960mb/sec, and around 400mb/sec for 10ge NFS.
Hmm, and where can you get that release in the open? ;)
Jim Klimov
2011-10-14 17:50:26 UTC
Permalink
Try and get your hands on a Sun F20 Card which has 4 x 25GB SSD
Modules on a PCI card.
Thanks, but I believe currently that's out of budget, but a 90MB/s
CF module may be acceptable for the small business customer.
I wondered if that is known to work or not...
Also if you can get an X4540 system controller you can then use 64GB
of RAM aka L1ARC.
I am not sure I can get that (not until they become refurbished and cheap),
but still: are the X4540 mainboards compatible with X4500 chassis?
Is it possible to upgrade one into another? ;)
Jordan
Post by Jim Klimov
Hello, I was asked if the CF port in Thumpers can be accessed by the OS?
In particular, would it be a good idea to use a modern 600x CF card (some
reliable one intended for professional photography) as an L2ARC device using
this port?
Thanks,
//Jim
Andrew Gabriel
2011-10-14 18:30:19 UTC
Permalink
Post by Jim Klimov
Thanks, but I believe currently that's out of budget, but a 90MB/s
CF module may be acceptable for the small business customer.
I wondered if that is known to work or not...
I've had a compact flash IDE drive not work in a white-box system. In
that case it was a ufs root disk, but any attempt to put a serious load
on it, and it corrupted data all over the place. So if you're going to
try one, make sure you hammer it very hard in a test environment before
you commit anything important to it.
--
Andrew Gabriel
Loading...