Gluster
Introduction
First tried to use ceph for a home cluster, but the overhead was just to great to handle with my small scale operation. Want something that I could run on on next to nothing, and not really expect great speeds, but high reliability.
Really liked ceph and its resilliance tho. But guess it was just to much to expect it to work with my ideas.
Toyed with the idea of using some s3fs thing, or gluster, but for this atempt I'm giving gluster a try.
Setup
Want to try to use docker/podman/container sollution, why ? Because I want to be able to issolate where its messing with my machine, and be able to replicate the setup easily.
Simple Dockerfile:
FROM fedora RUN dnf install -y glusterfs-server CMD ["glusterd", "-N", "--log-file", "-"]
I build this dockerimage as glusterfs
To run, i set up a macvlan for podman, lost documentation how I done this, TODO find out
Now to run the container I do:
podman run --name glusterfs -d --network macnet32 --hostname dell1 -v $PWD/cert:/etc/ssl -v /gluster/lib:/var/lib/glusterd -v /mnt/sdb:/mnt/brick --ip 192.168.0.231 --rm --privileged glusterfs
To mount a volume (that I somehow created sigh):
mount -t glusterfs 192.168.0.231:/vol test2
Start a second node..
From the "master" node i.e. the one you initiated first kinda:
gluster peer probe 192.168.0.233
Remove node
First remove all bricks from the node in use, (i have a broken cluster ofcouse):
gluster volume remove-brick vol replica 1 192.168.0.232:/mnt/brick/test2 force gluster peer detach 192.168.0.232
Add Brick
gluster volume add-brick vol 192.168.0.233:/mnt/brick/test2
Bitrot
Enable:
gluster volume bitrot vol enable
Trigger scrub (for testing mostly i guess), think ondemand means now is demand, not to only trigger ondemand:
gluster volume bitrot vol scrub ondemand
Heal
How the hell is this done, copy the file manually to the disk that is broken ?
Bitchy volym
When you messed around to much with a volume, it might sometimes help to restart glusterd
Certificate
Activate:
gluster volume set vol client.ssl on gluster volume set vol server.ssl on gluster volume set vol auth.ssl-allow workstation