01 Jan 0001, 00:00

Gluster

Introduction

First tried to use ceph for a home cluster, but the overhead was just to great to handle with my small scale operation. Want something that I could run on on next to nothing, and not really expect great speeds, but high reliability.

Really liked ceph and its resilliance tho. But guess it was just to much to expect it to work with my ideas.

Toyed with the idea of using some s3fs thing, or gluster, but for this atempt I'm giving gluster a try.

Setup

Want to try to use docker/podman/container sollution, why ? Because I want to be able to issolate where its messing with my machine, and be able to replicate the setup easily.

Simple Dockerfile:

FROM fedora

RUN dnf install -y glusterfs-server

CMD ["glusterd", "-N", "--log-file", "-"]

I build this dockerimage as glusterfs

To run, i set up a macvlan for podman, lost documentation how I done this, TODO find out

Now to run the container I do:

podman run --name glusterfs -d --network macnet32 --hostname dell1 -v $PWD/cert:/etc/ssl -v /gluster/lib:/var/lib/glusterd -v /mnt/sdb:/mnt/brick --ip 192.168.0.231 --rm --privileged glusterfs

To mount a volume (that I somehow created sigh):

mount -t glusterfs 192.168.0.231:/vol test2

Start a second node..

From the "master" node i.e. the one you initiated first kinda:

gluster peer probe 192.168.0.233

Remove node

First remove all bricks from the node in use, (i have a broken cluster ofcouse):

gluster volume remove-brick vol replica  1   192.168.0.232:/mnt/brick/test2 force
gluster peer detach 192.168.0.232

Add Brick

gluster volume add-brick vol 192.168.0.233:/mnt/brick/test2

Bitrot

Enable:

gluster volume bitrot vol enable

Trigger scrub (for testing mostly i guess), think ondemand means now is demand, not to only trigger ondemand:

gluster volume bitrot vol scrub ondemand

Heal

How the hell is this done, copy the file manually to the disk that is broken ?

Bitchy volym

When you messed around to much with a volume, it might sometimes help to restart glusterd

Certificate

Activate:

gluster volume set vol client.ssl on
gluster volume set vol server.ssl on

gluster volume set vol auth.ssl-allow workstation