Ever since the mini-ITX bug bit me this spring, I’ve been intrigued by the little Linux clusters people have been building with them. I absolutely despise loud and bulky home servers (not to mention their energy consumption) and thought this was a great idea to get a lot of computers into a small footprint with little to no noise. My favorite one was Glen Gardner’s mini-cluster as described on mini-itx.com. In my daytime profession I’m constantly dealing with the automation of lots of Linux systems and running distributed apps, so I thought it’d be cool to have my own little setup at home to wrench on. One doesn’t get much experience with load balancing or fault tolerance with a couple of systems. I could’ve just installed a bunch of VMs on a CentOS box (which I’ve already done) or use EC2 instances, but I like getting my hands dirty and felt like building something.
In my setup I wanted it to be more of a general purpose lab, optimize for a greater number of individual systems and not necessarily raw computing power. I wanted local storage for each system so I could someday play with things like HBase/Hadoop. Most of all I wanted it to be silent and consume only a modest amount of energy since it’s likely to be idle a lot. CPU fans may be okay, but so far I’m sticking to fan-less Atom boards.
I discovered picoPSUs, which are DC-DC power supplies, very small and eliminate the rats nest of ATX cabling. Being a ham operator I already had the token bigass 12V Samlex linear power supply laying around so I thought I had power solved. I also already had a stack of SATA hard drives, so this sparked pulling the trigger and buying the rest of the gear. At first I bought a few Shuttle XPCs from Weird Stuff Warehouse for $50 to salvage the motherboards, but turns out they were dead and/or very broken. I also discovered my linear PSU had shorted out and was horribly inefficient (50% ?!) anyways. By now I was sort of committed to the project with half the gear needed. I wound up going with Intel Atom D2500 motherboards and a new 192 W power adapter.
The first idea for mounting was to use threaded bolts between wooden plates to stack motherboards, but this wound up being a horrible idea. If the holes were off the slightest bit, they caused the rods to bind through the motherboard mounting holes. Plus, I still wanted hard drives. I sketched up a design for a little blade enclosure, so each system of motherboard and hard drive(s) could be mounted on a metal blade to be slid into a rack. I decided this was too much work and abandoned the idea.
…
Fast forward to a couple of weeks ago when I was randomly googling and discovered the “Helmer” linux cluster. Somebody had taken an IKEA HELMER cabinet, cut it up, and stuffed it full of micro-ATX boards. Looking at the dimensions of the drawers I realized they were almost exactly the dimensions of the blades I wanted to build. Then I found another design, and another. It also has the added benefit of being sized to slide underneath a desk to get out of the way. IKEA had solved the chassis problem I had and that weekend I went out to buy one to get hacking.
My design differs from others in that I made doors out of expanded metal to put on the front+back of the case, and went with system “blades” consisting of a mini-ITX board and dual 3.5″ hard drives. My requirements for fanless and quiet dictated that I couldn’t use a bunch of ATX power supplies and needed the airflow for passive cooling. Plus I didn’t want cables spilling out all over the place, both for aesthetic purposes and the fact that my cat loves playing with dangling SATA cables. In the end it really does look like a cabinet you’d find in a datacenter.
The cabinet was designed to hold six drawers (4″ tall) and thus six systems. I wanted to try doubling the density, putting 10-12 systems in the cabinet. This involves putting aluminum angle stock in between the existing rails, providing for 2″ tall systems. For six systems I’ll likely cut down the metal from the drawers and use them for blades, for the remainder I’m using sheets of acrylic. I have no idea at this point how hot things will get and how much the plastic will sag or even melt. I figure this can easily be solved with aluminum cross-members.
Right now I don’t have a real way to turn the boards on/off without shorting the power connector or pulling the plug. I configured the BIOS of each one to always power-on after power loss, and discovered wake-on-LAN works. Each motherboard is also configured to PXE boot first, so if I want to fiddle with a system (e.g. kickstart, memtest) without dragging out a monitor and keyboard I can drop in a PXE config on my tftp server or just let it boot locally.
Overall the project went much smoother than I expected. The HELMER assembled with only a screwdriver and I wasn’t left with bloody fingers from sharp edges like I would a normal 42U cabinet in a datacenter. The only power tool I needed was a drill, I was able to cut all of the expanded metal and aluminum stock with shears and tin snips. (silent construction + apartment = win!) It’s not a cheap project, the cost of the retrofit was basically the same cost as the HELMER and going the picoPSU route was more expensive than an army of ATX PSUs.
So far I only have three blades and an ethernet switch installed and heat doesn’t seem to be an issue yet. It’s virtually silent, although I’ve noticed if the hard drives seek just the right way it causes the door to rattle a bit. Each blade draws 30 W at idle, peaking at 50 W during hard drive spinup or heavy CPU usage, so this limits about 3-4 systems per 192 W PSU. Eventually I may have to give in a add a large (but low rpm!) fan or two on the back door, we’ll see.
If you’re looking for kit, I recommend mini-box.com. They stock several mini-ITX boards, invented the picoPSU, and have some nice simple compact cases. Particularly I like them because they’re local; I can either drive over to pick up parts if I want or USPS delivery is basically next day.
So ghetto
Maximum swag, this would be perfect for a small office to