Good stuff, NightShade!
Good stuff, NightShade!
LOL, well to answer a few redneck questions, the tractor analogy is semi correct but I am pulling a trailer many times the size an average one would be able to handle with the ability to expand that trailer by leaps and bounds. Actually throw the tractor out and think train, I have 8 engines and each one has the power of two standard ones, 48 fuel tanks and 7 cars behind it. I can double the cars to 14 (without any extra hardware other) and the capacity of each car as well. With a new case, SAS expander or another SAS controller I can add more cars of larger sizes. I can expand to 192 fuel tanks and more as well as more powerful engines too.
One of the reasons why I did raidZ3 vs raidZ2 is if and when a disk fails I have no clue how long I will have to wait for an RMA or procure a replacement. If the funds are low I may have to wait a little bit and would hate to have data loss. Second off ZFS works off of vDev's or virtual devices. Basically one vDev in raidZ3 is the seven drives I have, adding a second vDev will double the read and write speed as well as IOPS, I have the space to do so and will be a way to expand my storage pool, when 4TB drives like the ones I have are dirt cheap I can pick up another set of 7 and throw them in. The bad thing is that once two vDev's are a part of the pool EITHER vDev failing will cause a loss of the pool which makes having a stable structure in place that can take a failure without data loss more important farther down the road IMHO.
https://forums.freenas.org/index.ph...ning-vdev-zpool-zil-and-l2arc-for-noobs.7775/ This has a ton of info about what I am saying. Plus When drives get even cheaper I intend to swap out all the 4TB drives for some 8TB drives and grow the pool some more. My intention is to use this box for 8 to 10 years and it should be able to do it as well as serve without transcoding a 4K stream that is uncompressed and likely transcode a moderately compressed 4K stream.
Another thing is that during a resilver process after replacing a drive there is a greater chance of another drive failure. Not to mention and unrecoverable read error during resilvering http://www.zdnet.com/article/why-raid-5-stops-working-in-2009/ Right now your raid5 is at risk for a couple reasons, it's hardware raid and if the card ghosts you will have issues getting data back unless you find a controler that is identical to what is used now and even then it's a crap shoot. And during a resilver you have a HUGE chance of encountering an unrecoverable read error (read the link above).
I went with FreeNAS for a couple reasons, I am a geek and off the shelf just doesn't do it for me. Plus I run multiple things on the same server with the intent to setup a private gameserver on it as well. That is where FreeBSD (the base OS of FreeNAS) and the jail system comes in. Each jail is in essence it's own private operating space, like a virtual machine without all the extra overhead. FreeNAS in itself doesn't require a lot of CPU power unless you are having a lot of stuff accessing via CIFS (windows share) but when you throw Plex in play and start transcoding on the fly it loves the extra cpu power. I can max out 8 cores and 16 threads by starting two transcode streams at the same time for about 60 seconds and can actually transcode reliably four streams at any given point. My step daughter has access as well as my mother in law and wife, so say we all start a stream that needs to transcode they will all be able to work without much of a slowdown. And I end up transcoding if I watch a movie on my phone while I am sitting in say...Dallas waiting on a connecting flight that is running late. The video will stream from home.
I also use OpenVPN as a client and a server, I can connect to my home from anywhere and the FreeNAS jail handles it with no problems. Add Syncthing and a Murmur (Mumble) server and I can connect with family and friends and have a conference call while playing a game or whatever. The FreeNAS UI is great for seeing how much data is in certain directories gives me updates about information that may be happening like smart errors and runs smart tests on my schedule and with the ZFS scrubs the filesystem heals itself so I don't lose data from bitrot.
ZFS with hardware raid DOES NOT PLAY NICE, I have a SuperMicro X8DT6-F which includes a LSI SAS 2008 Controller that is flashed into IT(Initiator Target) mode rather than working as a raid controller. Without using a SAS expander I can connect 14 drives which is part of the reason why I went with a 7 drive vDev. FreeNAS minimum amount of ram is 8GB as ZFS does love ram, it uses it for an Ahead Read Cache and it also does do a little write caching though not much. Minimums are here https://forums.freenas.org/index.php?threads/hardware-recommendations-read-this-first.23069/ I have 48GB of ECC ram and it's tri-channel so even though it is DDR3 it's fast... And technically it's hexa-channel since each CPU is tri-channel and they interface at something like 5.86 GT/s. So it's almost as fast as that dual channel ddr4 rig you have been eyeing at a fraction of the cost memory wise and with a CPU upgrade to a dual X5690(around 400 each) it would be even faster. I spend 35 on two E5640 CPU's BTW and they are hyperthreaded so 8 physical cores and 16 as far as the OS is concerned. Ram was about 6.00 per 4GB stick.
As far as using the Netgear NAS, that would probably be a no, it's likely a cheap raid5 controller with a PSU and a hotswap bay and a low power board to serve the data via network. Do a large file copy to it via windows and you likely will not even reach the CIFS limit speed on gigabit of around 130MB/s. If you are wanting something like this though I am more than happy to help set something up or whatever. I ever end up out that way and I can lug mine along to check out if you like.
I have movies and music along with some tv show seasons. Personal pictures and documents. Game installs and game as well as other software ISO's. But this is mainly for my home use though I may store some webserver backups on it as well at some points. Personally I think it was money well spent after having a seagate drive give up on me and blow 2TB worth of media down the toilet. Plus having the drive speed is great. Now I just have to either get a switch that supports aggregation or upgrade to 10Gig networking.
As I said in the original post I built this on the cheap, buying a little more modern hardware will get around the same power from one cpu and use a little less electricity. Or you could use an Atom based board and get in the 60 watt range but lose the ability to transcode. It does have a pcie X8 slot so you could still add a SAS controller into it and add more drives.
One of the reasons why I did raidZ3 vs raidZ2 is if and when a disk fails I have no clue how long I will have to wait for an RMA or procure a replacement. If the funds are low I may have to wait a little bit and would hate to have data loss. Second off ZFS works off of vDev's or virtual devices. Basically one vDev in raidZ3 is the seven drives I have, adding a second vDev will double the read and write speed as well as IOPS, I have the space to do so and will be a way to expand my storage pool, when 4TB drives like the ones I have are dirt cheap I can pick up another set of 7 and throw them in. The bad thing is that once two vDev's are a part of the pool EITHER vDev failing will cause a loss of the pool which makes having a stable structure in place that can take a failure without data loss more important farther down the road IMHO.
https://forums.freenas.org/index.ph...ning-vdev-zpool-zil-and-l2arc-for-noobs.7775/ This has a ton of info about what I am saying. Plus When drives get even cheaper I intend to swap out all the 4TB drives for some 8TB drives and grow the pool some more. My intention is to use this box for 8 to 10 years and it should be able to do it as well as serve without transcoding a 4K stream that is uncompressed and likely transcode a moderately compressed 4K stream.
Another thing is that during a resilver process after replacing a drive there is a greater chance of another drive failure. Not to mention and unrecoverable read error during resilvering http://www.zdnet.com/article/why-raid-5-stops-working-in-2009/ Right now your raid5 is at risk for a couple reasons, it's hardware raid and if the card ghosts you will have issues getting data back unless you find a controler that is identical to what is used now and even then it's a crap shoot. And during a resilver you have a HUGE chance of encountering an unrecoverable read error (read the link above).
I went with FreeNAS for a couple reasons, I am a geek and off the shelf just doesn't do it for me. Plus I run multiple things on the same server with the intent to setup a private gameserver on it as well. That is where FreeBSD (the base OS of FreeNAS) and the jail system comes in. Each jail is in essence it's own private operating space, like a virtual machine without all the extra overhead. FreeNAS in itself doesn't require a lot of CPU power unless you are having a lot of stuff accessing via CIFS (windows share) but when you throw Plex in play and start transcoding on the fly it loves the extra cpu power. I can max out 8 cores and 16 threads by starting two transcode streams at the same time for about 60 seconds and can actually transcode reliably four streams at any given point. My step daughter has access as well as my mother in law and wife, so say we all start a stream that needs to transcode they will all be able to work without much of a slowdown. And I end up transcoding if I watch a movie on my phone while I am sitting in say...Dallas waiting on a connecting flight that is running late. The video will stream from home.
I also use OpenVPN as a client and a server, I can connect to my home from anywhere and the FreeNAS jail handles it with no problems. Add Syncthing and a Murmur (Mumble) server and I can connect with family and friends and have a conference call while playing a game or whatever. The FreeNAS UI is great for seeing how much data is in certain directories gives me updates about information that may be happening like smart errors and runs smart tests on my schedule and with the ZFS scrubs the filesystem heals itself so I don't lose data from bitrot.
ZFS with hardware raid DOES NOT PLAY NICE, I have a SuperMicro X8DT6-F which includes a LSI SAS 2008 Controller that is flashed into IT(Initiator Target) mode rather than working as a raid controller. Without using a SAS expander I can connect 14 drives which is part of the reason why I went with a 7 drive vDev. FreeNAS minimum amount of ram is 8GB as ZFS does love ram, it uses it for an Ahead Read Cache and it also does do a little write caching though not much. Minimums are here https://forums.freenas.org/index.php?threads/hardware-recommendations-read-this-first.23069/ I have 48GB of ECC ram and it's tri-channel so even though it is DDR3 it's fast... And technically it's hexa-channel since each CPU is tri-channel and they interface at something like 5.86 GT/s. So it's almost as fast as that dual channel ddr4 rig you have been eyeing at a fraction of the cost memory wise and with a CPU upgrade to a dual X5690(around 400 each) it would be even faster. I spend 35 on two E5640 CPU's BTW and they are hyperthreaded so 8 physical cores and 16 as far as the OS is concerned. Ram was about 6.00 per 4GB stick.
As far as using the Netgear NAS, that would probably be a no, it's likely a cheap raid5 controller with a PSU and a hotswap bay and a low power board to serve the data via network. Do a large file copy to it via windows and you likely will not even reach the CIFS limit speed on gigabit of around 130MB/s. If you are wanting something like this though I am more than happy to help set something up or whatever. I ever end up out that way and I can lug mine along to check out if you like.
I have movies and music along with some tv show seasons. Personal pictures and documents. Game installs and game as well as other software ISO's. But this is mainly for my home use though I may store some webserver backups on it as well at some points. Personally I think it was money well spent after having a seagate drive give up on me and blow 2TB worth of media down the toilet. Plus having the drive speed is great. Now I just have to either get a switch that supports aggregation or upgrade to 10Gig networking.
As I said in the original post I built this on the cheap, buying a little more modern hardware will get around the same power from one cpu and use a little less electricity. Or you could use an Atom based board and get in the 60 watt range but lose the ability to transcode. It does have a pcie X8 slot so you could still add a SAS controller into it and add more drives.
I skipped down.....it didn't help.
ROFLMAO, I have said before that I am a geek right?
Lots of research in this nightshade, great job of putting the pieces together. I've been out of the I/T world for a while and would have to scramble to get back up to speed, but I understand what you did. I'm a big fan of open source stuff.
Gotta love FREE NAS.
For those of you who may be interested in what makes it all work:
A general parts list:
SuperMicro X8DT6-F
Intel Xeon E5640 X2
12 sticks of DDR3 ECC ram
Rosewill Photon 850 PowerSupply
Rosewill RSV-L4500 - 4U Rackmount Server case
7 HGST Deskstar NAS drives 4TB each
2 CoolerMaster Hyper212 EVO coolers
A general parts list:
SuperMicro X8DT6-F
Intel Xeon E5640 X2
12 sticks of DDR3 ECC ram
Rosewill Photon 850 PowerSupply
Rosewill RSV-L4500 - 4U Rackmount Server case
7 HGST Deskstar NAS drives 4TB each
2 CoolerMaster Hyper212 EVO coolers
Very nice setup. One of these days I'ma do a rackmount server system in my house. When I get a house with some room. lol
Awesome....
(Tapatalk)- on the road.
(Tapatalk)- on the road.
Thanks for the compliments.
I was originally going to build a high airflow case out of 3/4 plywood and build a drive cage with 1.5 inch perforated angle using rubber grommets as isolators. Sadly since I knew I was going to have to get my shoulder fixed I picked up the rosewill case. Let me tell you something else, it's nearly impossible to find a case large enough to fit and extended atx board with at least 14 hdd's among other things that is cheap. The 4U rackserver case was the only thing that fit the bill, it will likely never live in a rack though but it works well and the handles make it easier to move around. It is sitting on a table now and I may eventually get a mount so I can horizontally mount it on a wall.
Similar threads
