Archive for the ‘ceph’ Category

A bucket of squid

June 10, 2015 5 comments

Since my last post I’ve been tinkering with my little Pi cluster a bit, but mostly trying to sort out a better case and power supply. Although I do like the Lego aesthetic it’s far too likely to be randomly disassembled by my kids! The power supply was also an issue, as the little USB powered hub only had a 3A supply and was struggling with six Pis.

So I’ve rehoused the cluster into the case shown below, and I think now have a (still WIP) good solution for the power supply. The picture shows five Pi 2 B cluster nodes (plus one Pi B+ for cluster control), though I’m planning to add a sixth Pi 2 B cluster node on the left hand side. It’s hard to see, but there’s also an eight port Ethernet switch under the Pis.

## See the bottom of this post for updates ##

Pi chassisUnfortunately the case was damaged in transit, and is now held together with super glue! To their credit the vendor quickly provided a full refund, and I’m ordering a replacement. Until that arrives I’m using the damaged case as a proof of concept.

Pi casePi caseApart from the damage in transit, I’m very happy with the case. There was a 120mm fan under the top vent, but I removed that and the system remains cool-ish with only passive air flow.

For the power supply I’m using four quad-USB car charging adapters, all connected to a laptop power supply. You can see them in the top left and right of the case photos. The USB charging adapters can tolerate between 8-20V and output 5V and 6A, 30W, across four USB ports each. Perfect for my small Pi cluster, and with the potential for an in-car mobile Pi cluster (just kidding!).

USB powerCurrently the power supply is still a bit primitive, with no power switch or soft power down option. So I need to manually shut down all the cluster nodes before disconnecting the power. Which brings me to the WIP section…

I’m planning to use an ATXRaspi board to manage power to the Pi control node, and modify the startup/shutdown process to then manage power to the six Pi cluster nodes. The control node will connect via GPIO to a solid state relay to switch the cluster power on and off. I’m considering using two SSRs so that I can selectively fail half of the cluster nodes.

Pi case power planPower On

  1. Connect mains power, no power to RPi nodes (option to continuously power Ethernet switch)
  2. Press ATXRaspi switch, enable power to RPi control node (and Ethernet switch) and trigger startup script
    1. Startup script activates the DC relay and enables power to the RPi cluster nodes

Power Off

  1. Press ATXRaspi switch, this triggers shutdown script on RPi control node
    1. Shutdown script executes remote shutdown of RPi Cluster nodes via SSH
    2. Shutdown script deactivates the DC relay and powers off the RPi cluster nodes
    3. Shutdown script completes the shutdown and powers off of the RPi control node
      (and Ethernet switch)
  2. Disconnect mains power

I’ll update next when the parts arrive and I see if this actually works… 🙂

## Update 1 – It Lives! ##

I found some time this evening to pull the cluster apart and test the ATXRaspi and relays that arrived last week.

Looks good! Button control works great and the control node startup (/etc/rc.local) triggers the relays via the GPIO connections.

Testing ATXRaspiNow to put it all back together and test the cluster node shutdown scripts.

## Update 2 – Back in the box ##

Here are some photo’s of the reassembled cluster, now with working soft power on/off controls.

Pi cluster

Front top view – showing power button and ATXRaspi (lower right side)

Pi cluster

Front left view – showing the USB power adapters that are controlled by the relays

Pi Cluster

Front right view – showing the power control relays

Ceph is running nicely, and I’ve updated the CRUSH map to split the pg copies between the odd/even nodes, as they hang off different USB power supplies.

ceph_osd_treeAnd some glamour shots 🙂

Pi ClusterPi ClusterPi ClusterI’ve also uploaded the pictures here –

Categories: ceph, Raspberry Pi Tags:

My little Cephalopony

April 9, 2015 Leave a comment

In order to teach myself a bit about Ceph, I’d like to introduce my modestly scalable, low performance, gigabyte-scale Raspberry Pi storage cluster!

Ceph Pi clusterThat’s five Raspberry Pi v2, each with the standard quad-core ARM Cortex-A7 CPU (over-clocked to 1 GHz), 1 GB memory, and four 8 GB USB memory sticks for storage. When I was a lad that was a lot…

Installation was embarrassingly easy, mostly following instructions from here, though I used Minibian as the OS instead of Raspbian. Along the way I also discovered the Mosh shell, which makes remote connection over varied, slow, and intermittent networks tolerable.

Obviously this is an educational build and not intended for serious use, though it’s already clear that 8 GB USB sticks are too small. Their appeal was low cost and the “Christmas tree” effect of so many blinkenlights.

ceph@piceph1:~$ ceph -s
cluster 661a43a5-e9cc-4350-851d-109d07c44773
health HEALTH_OK
monmap e1: 1 mons at {}, election epoch 2, quorum 0 piceph1
osdmap e24: 5 osds: 5 up, 5 in
pgmap v49: 192 pgs, 3 pools, 0 bytes data, 0 objects
25694 MB used, 9240 MB / 36923 MB avail
192 active+clean

I’ll update with more detail as I figure out just what to do with this “data centre in a lunch box” 🙂

Categories: ceph, Raspberry Pi