Computer Storage

I mainly use the harddisks in the PCs I run for storage of Hidef movies and TV series.

Current storage situation
MachineBruto*RedundancyNetto*Disks
livingroom PC2.7TBsingle disk2.5TB256GB Crucial M4 SSD, 480GB Crucial M500, 1x 2TB Seagate
NasPC92TB60GB Boot
14x4T RAIDZ2
12x3T RAIDZ2
76.0TB60GB Intel 330 series, 14x4TB Seagate, 12x3TB Seagate
NasPC252.0TB120GB Boot
13x4T RAIDZ2
44.0TB120GB Intel 330, 13x4TB Seagate
LinuxPC20TBSSD 30G for boot
SSD 60G share
10x2TB RAIDZ2 ZFS
16TB128GB Crucial C300 SSD, 9x2TB Seagate, 1x2TB WD green
ServerPC5TBSSD 120G for boot
SSD 5x 1TB RAIDZ1 ZFS
4TB120GB Crucial M500 SSD, 5x1.05TB Crucial SSD
Atom Low power PC256GB1x30GB boot
226G
226GB256GB Crucial M4 SSD
Total:172TB.142TB.

I store all data to ZFS on Linux. All previous LVM raid 0 volumes have been moved to ZFS pools with RAIDZ2.
RAIDZ1 is like RAID5 but after disk failure recovery is only needed for the data, not the entire disk so it is much quicker than plain RAID5. ZFS keeps checksums on all data to ensure data integrity at all times. For RAIDZ1 there is the space of 1 spare disk, for RAIDZ2 (like RAID6) the space of 2 spare disks is needed and used for CRC data and other ZFS overhead data.
Currently there are 10x2T as spare disks. In nas1 there are 12x3T in RAIDZ2 plus 14x4T in RAIDZ2. In nas2 I have 13x4T in RAIDZ2. In the test setup with 6x1.5T in RAIDZ2. The new always on xeonD server has 5x 1TB SSD in RAIDZ1. I keep separate pools in order to be able to swap them out to different PC's. I have removed the 10x2T (because 2 disks were failing) and may move 8 of them to the test setup. I plan to replace the 12x3T with 12x larger disk (like 12x 6T or 12x 8T) in the future.
The main NAS PC has 8 onboard sata interfaces and 24 extra sata interfaces through a sas card, there are 25 regular bays and I have stacked 1 seperate drive bay inside the case. Currently 26 bays are in use. The free PCI-E 4x slot is now used for a 10Gbe Intel X520-DA1 network card with SFP+.
The second NAS PC has 14 onboard sata interfaces, 1x for boot 13x for the data disks. It has a 10Gbe Intel X520-DA2 network card with SFP+.
The XeonD has 6 sata interfaces. 1x boot plus 5x data. It has onboard 10Gbe RJ45.

27x Seagate 4.0TB : ST4000DM000 (5900 rpm) (sata3, 64MB)
12x Seagate 3.0TB : ST3000DM001 (7200 rpm) (sata3, 64MB)
9x Seagate 2.0TB : ST2000DL003 (5900 rpm) (sata3, 64MB) (2x broken)
1x Seagate 2.0TB : ST2000DM001 (7200 rpm)
2x Seagate 2.0TB : ST32000542AS (5900 rpm)
2x WesternDigital 2.0TB : WD20EARS Caviar Green 2TB, 64MB (1x broken)
4x Samsung 1.5TB : HD154UI Green (5400 rpm)
4x Seagate 1.5TB : ST31500341AS
6x Samsung 1TB : HD103UJ
5x Samsung 750G : HD753LJ
4x Samsung 500G : HD501LJ
8x SSD : 60GB + 120GB Intel 330 , 128GB Crucial C300, 2x 256GB Crucial M4, 120GB + 480GB Crucial M500, 60GB Corsair.
7x SSD : 6x 1TB Crucial MX300, 500GB Crucial BX100


This table shows Linux kernel versions and ZFS/SPL versions used currently :
ZFS version (git zfs) kernelSPLZFS
riwilinux4.10.80.7.0-rc4_4_gac483610.7.0-rc4_56_g419c80e
riwinas4.4.140.6.5.10-10.6.5.10-1
riwinas24.4.380.6.5.9-10.6.5.9-1
riwixeon4.4.380.7.0-rc4_4_gac483610.7.0-rc4_54_g88c3012


This table shows the storage usage per pool:
drivetotalusedfree%graph bar
/dev/sda2217G110G107G51%
riwilinux:/pool114361G176G14185G 1%
riwinas:/pool325751G18925G6826G73%
riwinas:/pool443091G24883G18207G58%
riwinas2:/pool540317G34747G5570G86%
riwixeon:/pool03947G1801G2146G46%
total127684G80642G47041G63%


This table shows the health of each individual disk as well as some statistics.
devicevolumediskmodelserialtemphourspower on countreported_uncorrectcurrent_pending_sectoroffline_uncorrectablereallocated_sector_count
riwilinux:/dev/sda-noneC300-CTFDDAC128MAG00000000104602FF3866none61048700002048
riwilinux:/dev/sdbzfs-pool1ST2000DL003-9VT1665YD1932H33 C1811312698000
riwilinux:/dev/sdczfs-pool1ST2000DL003-9VT1665YD2ERHJ31 C429012801000
riwilinux:/dev/sddzfs-pool1ST2000DM001-1ER164Z4Z07GMY32 C34757230000
riwilinux:/dev/sdezfs-pool1ST32000542AS5XW0XVZB30 C713013360000
riwilinux:/dev/sdfzfs-pool1ST2000DL003-9VT1665YD15AE332 C180201270191201200
riwilinux:/dev/sdgzfs-pool1ST2000DL003-9VT1665YD19A0W32 C18491127033000
riwilinux:/dev/sdhzfs-pool1WDC WD20EARS-00S8B1WD-WCAVY366101534 C7529136733000
riwilinux:/dev/sdizfs-pool1ST32000542AS5XW0VA1230 C735713350000
riwilinux:/dev/sdjzfs-pool1ST2000DL003-9VT1665YD37KEE33 C753218390000
riwilinux:/dev/sdkzfs-pool1ST2000DL003-9VT1665YD2FKLZ32 C433612800000
devicevolumediskmodelserialtemphourspower on countreported_uncorrectcurrent_pending_sectoroffline_uncorrectablereallocated_sector_count
riwinas:/dev/sda-noneINTEL SSDSC2CT060A3CVMP215605Q1060AGNnone970312690000
riwinas:/dev/sdaazfs-pool3ST3000DM001-1CH166Z1F1QAT831 C592412070000
riwinas:/dev/sdbzfs-pool4ST4000DM000-1F2168Z301GP5Q26 C10992690000
riwinas:/dev/sdczfs-pool4ST4000DM000-1F2168Z301GE5S25 C10992690000
riwinas:/dev/sddzfs-pool4ST4000DM000-1F2168Z304WYTA25 C92550000
riwinas:/dev/sdezfs-pool4ST4000DM000-1F2168Z303Y4ZP25 C92520000
riwinas:/dev/sdfzfs-pool4ST4000DM000-1F2168Z300K08K30 C509310100000
riwinas:/dev/sdgzfs-pool3ST3000DM001-1CH166Z1F1NLX931 C591912060000
riwinas:/dev/sdhzfs-pool3ST3000DM001-1CH166Z1F1PR6B31 C592812050000
riwinas:/dev/sdizfs-pool3ST3000DM001-1CH166Z1F1P72Z31 C592812120000
riwinas:/dev/sdjzfs-pool4ST4000DM000-1F2168Z300MC1Z29 C508810100000
riwinas:/dev/sdkzfs-pool4ST4000DM000-1F2168Z300MB1C29 C508510100000
riwinas:/dev/sdlzfs-pool4ST4000DM000-1F2168Z300MB1L29 C509010100000
riwinas:/dev/sdmzfs-pool4ST4000DM000-1F2168Z300K0PQ28 C509010100000
riwinas:/dev/sdnzfs-pool3ST3000DM001-1ER166Z501H1Q729 C9952550000
riwinas:/dev/sdozfs-pool3ST3000DM001-1ER166Z501KWLN29 C9952510000
riwinas:/dev/sdpzfs-pool4ST4000DM000-1F2168Z30156CH28 C39438060000
riwinas:/dev/sdqzfs-pool4ST4000DM000-1F2168Z30147QZ29 C39418060000
riwinas:/dev/sdrzfs-pool4ST4000DM000-1F2168W300BMH029 C47899460000
riwinas:/dev/sdszfs-pool4ST4000DM000-1F2168W300G8KQ28 C47919460000
riwinas:/dev/sdtzfs-pool3ST3000DM001-1CH166W1F39TEB30 C41188390000
riwinas:/dev/sduzfs-pool3ST3000DM001-1CH166W1F39TEE30 C41278380000
riwinas:/dev/sdvzfs-pool3ST3000DM001-1CH166W1F30SJX31 C50809970000
riwinas:/dev/sdwzfs-pool4ST4000DM000-1F2168Z300K11N29 C508110100000
riwinas:/dev/sdxzfs-pool3ST3000DM001-1CH166Z1F31RN530 C503410030000
riwinas:/dev/sdyzfs-pool3ST3000DM001-1CH166Z1F1PTZ730 C591712030000
riwinas:/dev/sdzzfs-pool3ST3000DM001-1CH166Z1F1MLHG30 C592612040000
devicevolumediskmodelserialtemphourspower on countreported_uncorrectcurrent_pending_sectoroffline_uncorrectablereallocated_sector_count
riwinas2:/dev/sdazfs-pool5ST4000DM000-1F2168Z301GRFN35 C13102150000
riwinas2:/dev/sdb-noneINTEL SSDSC2CT120A3CVMP2152038G120BGN35 C189112640000
riwinas2:/dev/sdczfs-pool5ST4000DM000-1F2168Z301GDD336 C13092150000
riwinas2:/dev/sddzfs-pool5ST4000DM000-1F2168Z301GNNK36 C13092150000
riwinas2:/dev/sdezfs-pool5ST4000DM000-1F2168Z3013XE436 C17142850000
riwinas2:/dev/sdfzfs-pool5ST4000DM000-1F2168Z30146PF36 C17572870000
riwinas2:/dev/sdgzfs-pool5ST4000DM000-1F2168Z300V3G137 C17952840000
riwinas2:/dev/sdhzfs-pool5ST4000DM000-1F2168Z3013W9T36 C17592840000
riwinas2:/dev/sdizfs-pool5ST4000DM000-1F2168Z300V3KX37 C17052850000
riwinas2:/dev/sdjzfs-pool5ST4000DM000-1F2168Z3013WLW37 C17872850000
riwinas2:/dev/sdkzfs-pool5ST4000DM000-1F2168Z3013VLC36 C17592840000
riwinas2:/dev/sdlzfs-pool5ST4000DM000-1F2168Z300V34237 C17932850000
riwinas2:/dev/sdmzfs-pool5ST4000DM000-1F2168Z3013VNL34 C17562830000
riwinas2:/dev/sdnzfs-pool5ST4000DM000-1F2168Z3013YJL36 C17552850000
devicevolumediskmodelserialtemphourspower on countreported_uncorrectcurrent_pending_sectoroffline_uncorrectablereallocated_sector_count
riwixeon:/dev/sda-noneCrucial_CT120M500SSD11308092BD91E31 C12645530000
riwixeon:/dev/sdbzfs-pool0Crucial_CT1050MX300SSD1163813FCCF2434 C522950000
riwixeon:/dev/sdczfs-pool0Crucial_CT1050MX300SSD1163813FCD7FE32 C523040000
riwixeon:/dev/sddzfs-pool0Crucial_CT1050MX300SSD11631137A420531 C523150000
riwixeon:/dev/sdezfs-pool0Crucial_CT1050MX300SSD11631137A1D3832 C451540000
riwixeon:/dev/sdfzfs-pool0Crucial_CT1050MX300SSD1163813FCCD4D31 C451540000
devicevolumediskmodelserialtemphourspower on countreported_uncorrectcurrent_pending_sectoroffline_uncorrectablereallocated_sector_count
atom64:/dev/sda-noneM4-CT256M4SSD20000000013040927E66Dnone36864300000
atom64:/dev/sdb-noneCT500BX100SSD11449F000999E33 C20260000