Who needs 10 gigabit?

at some point in your home lab career, you will see posts of people doing things like fiber in their home, or upgrading their infrastructure to do much faster LAN speeds than whatever comes on their motherboard. it may seem crazy that anyone would need that kind of speed in their home, but you probably want to read this post before judging.

I recently upgraded to 10 gigabit speeds in my home. there is a long thought process behind that, but I’m going to throw some numbers at you to help you understand how I got to this decision.

I run a RAIDZ1 ZFS pool with 4 x 4TB seagate ironwolf drives. these drives are rated at a 180MBps sustained read speed. since I have 4 of them and 1 is used for parity, I get 3x read speeds using this type of raid. this means my pool has sustained reads of 540MBps. now lets looks at standard hardware.

my mobo is a regular desktop unit I sourced from ebay. it has an onboard gigabit network controller, like most mobos made for the intel 10th gen cpu’s. to convert the gigabit speed into MBps for an apples to apples comparison, we need to multiply by 8 since there are 8 bits in a byte. that means my previously mentioned pool is reading at approx 4.32 gigabits per second (gbps). so immediately anytime i want to move any information off of my server, my bottleneck is my network adapter by a factor of 4.

this is what was keeping my up at night. it turns out, the easiest way to get to 10G (or 10 gigabit) is to either a) get a new mobo with an onboard 10G NIC [network interface card] or b) buy a separate 10G NIC. for compatibility purposes, I did both. also, when I am talking about 10G networking, I do not mean fiber.

after doing some research, I will tell you for the average home user, fiber is not the way to go. not because it isnt awesome, but because most people have the most experience with ethernet cabling. also, you probably have a ton of ethernet already in your house. in a perfect world that ethernet would be CAT7 or higher, but its most likely CAT5e since that is the most common cable in use since the early 2000’s. what you may not know is that old CAT5e is fast enough to do 10G speeds in short enough runs, like the kind in your house. in other words, you’re already most of the way there for 10G speeds. you just need some endpoints and switches.

my server was what I was most worried about since it runs more long term software, which means an older kernel, which means less compatibility. i swapped its mobo for one with an onboard 10G NIC. in my desktop which runs fedora 39, I bought a $30 card off of ebay to plug into the PCIe slot. now both of my machines can receive and send at 10G speeds.

to finish the job, I needed a switch which was capable of making 10G speeds work. there are very few I like since I wanted a switch which was ALL 10G-base-T ports (fancy networking terms for all RJ45 ethernet ports @ 10G speeds). I found a few great ones which do the job, but I got lucky and scored an Netgear ProSafe Plus XS708E for $150 on ebay. this switch has 8 ports and fits in my rack perfectly.

after installing it and connecting all my CAT5e, I was able to get 10G speeds from my router to my switch, then to my server as well as my desktop. testing this with a real-world large file transfer (12 gigabytes) I saw speeds 3-4x faster than I did on the old connections. by placing my rack where all my home ethernet terminates, I now effectively turned my entire house into a 10G LAN.

something to note, while this is awesome for my home, it wont help me at all on the greater internet. FIOS gives me gigabit symmetrical to the door, but thats as fast as I can go. my 10G speeds are way faster than I can get as a regular joe living in a standard home. but the good news is, once I get the data into my rack, I can move it around my place so quickly it feels almost instant for anything less than 1 gigabyte. best.upgrade.ever.