A distributed MinIO installation on Rocky Linux 9 across five nodes, three data disks each; the same steps work unchanged on RHEL 9 and AlmaLinux 9. Every command below runs on all nodes — that is the part people get wrong, because a node with a different configuration does not fail loudly, it just refuses to join.
Before you start, several caveats that did not exist when this was first written. MinIO relicensed from Apache 2.0 to AGPLv3 back in 2021, and then in 2025 removed the administration console from the community build — what is left on port 9001 is a basic object browser. Since October 2025 there are also no official community binaries, RPMs or container images, and the repository was archived in April 2026. The server still works and everything below still applies, but bucket policies, lifecycle rules, replication and user management are now mc commands or a paid AIStor licence. Plan for the command line.
Firewall ports a MinIO installation needs
firewall-cmd --permanent --zone=public --add-port=9000/tcp
firewall-cmd --permanent --zone=public --add-port=9001/tcp
firewall-cmd --reload
9000 is the S3 API, 9001 the console. On a distributed deployment the nodes also talk to each other on 9000, so this rule is not only for clients.
Time must be in sync
date
Not a formality. A distributed MinIO installation signs every inter-node request the way S3 signs a client request, and MinIO requires all nodes to be within 15 minutes of each other — past that, signature verification fails and the node's requests are rejected. date across nodes is the eyeball test; this is the real one:
systemctl enable --now chronyd
chronyc tracking
If Leap status is not Normal on every node, fix that before going further.
Disks: XFS, labelled, mounted by label
mkfs.xfs /dev/sdb -L DISK1
mkfs.xfs /dev/sdc -L DISK2
mkfs.xfs /dev/sdd -L DISK3
mkdir /mnt/disk1
mkdir /mnt/disk2
mkdir /mnt/disk3
# /etc/fstab
LABEL=DISK1 /mnt/disk1 xfs defaults,noatime 0 2
LABEL=DISK2 /mnt/disk2 xfs defaults,noatime 0 2
LABEL=DISK3 /mnt/disk3 xfs defaults,noatime 0 2
mount -a
df -h
Mounting by label rather than by /dev/sdX is the important detail. Device names are assigned in discovery order and can change across a reboot or a controller replacement; mount the wrong disk into the wrong position and MinIO sees a drive whose contents do not match where it thinks they are.
noatime is there because MinIO has no use for access timestamps. The kernel default is relatime, so the cost is small rather than a write per read — but on a drive doing nothing except object storage it is a write that buys you nothing.
UUID= is the paranoid version of the same idea: a label can be duplicated by a disk carried in from another machine, a UUID cannot. blkid reads them.
Install the server
# This no longer works - every path under dl.min.io returns HTTP 410 Gone:
# wget https://dl.min.io/server/minio/release/linux-amd64/archive/minio-20230504214430.0.0.x86_64.rpm
That was the original step and it is worth keeping visible, because the reason it fails is not obvious: MinIO stopped publishing community binaries, RPMs and images in October 2025 and archived the repository in April 2026. min.io/download now offers only AIStor, the commercial product. The AGPLv3 community edition is source only, so you build it:
dnf install golang git
go install github.com/minio/minio@latest
install -m 0755 ~/go/bin/minio /usr/local/bin/minio
Build once and copy the same binary to every node. A version mismatch across a distributed set is one of the ways a cluster comes up degraded without an obvious error. Note that with no RPM there is no packaged minio.service either — take the unit file from the archived repository, or write one that reads /etc/default/minio.
The service account
groupadd -r minio-user
useradd -M -r -g minio-user minio-user
chown minio-user:minio-user /mnt/disk1 /mnt/disk2 /mnt/disk3
-M means no home directory and -r a system account. If you later place TLS certificates under /home/minio-user, create that directory yourself — the account does not have one by default.
The configuration, identical on every node
# /etc/default/minio
MINIO_VOLUMES="https://minio-prod-0{1...5}.domain.local:9000/mnt/disk{1...3}/minio"
MINIO_OPTS="--console-address :9001"
MINIO_ROOT_USER=...
MINIO_ROOT_PASSWORD=...
The {1...5} and {1...3} are MinIO's own expansion notation, and the line above describes the whole cluster: five hosts, three drives each, fifteen drives in total. The same string goes on every node — each one works out which part of it is itself.
Fifteen drives also decides your erasure coding. MinIO groups drives into erasure sets and applies a default parity of EC:4 to any set of eight or more — four parity shards, so you can lose four drives. With three drives per node that is one whole node plus one spare drive; two nodes down is six drives, past quorum, and the data is unreadable until one comes back. If surviving two node failures matters, raise it before the first start:
MINIO_STORAGE_CLASS_STANDARD="EC:6"
Confirm what you actually got from the startup log — MinIO prints the set count and drives per set on the first boot of a pool, and the erasure set size cannot be changed afterwards. A MinIO installation with too few drives will start happily and give you far less protection than you assumed.
Every hostname in that line must resolve on every node, forwards and backwards. Name resolution is the most common reason a distributed MinIO installation sits in a retry loop at startup.
TLS certificates
The account has no home directory yet, so create the path on every node first:
mkdir -p /home/minio-user/.minio/certs/CAs
chown -R minio-user:minio-user /home/minio-user
scp CAs/public.crt [email protected]:/home/minio-user/.minio/certs/CAs/public.crt
scp public.crt [email protected]:/home/minio-user/.minio/certs/public.crt
scp private.key [email protected]:/home/minio-user/.minio/certs/private.key
Repeat for each node, then fix ownership:
chown minio-user:minio-user /home/minio-user/.minio/certs/CAs/public.crt
chown minio-user:minio-user /home/minio-user/.minio/certs/public.crt
chown minio-user:minio-user /home/minio-user/.minio/certs/private.key
The certificate has to be valid for the names in MINIO_VOLUMES — a certificate issued for the load balancer alone leaves the nodes unable to verify each other.
Start it
systemctl enable minio.service
systemctl start minio.service
If systemd reports status=203/EXEC with a bare Permission denied, that is SELinux on a binary you built rather than installed from a package — the RPM used to label it for you:
restorecon -v /usr/local/bin/minio
ausearch -m avc -ts recent
Verify
systemctl status minio
journalctl -u minio -n 50
The log is where a distributed deployment tells you what is wrong: unresolvable peers, a certificate mismatch, a drive that is not mounted. Wait for all nodes before judging — the first ones to start will report the others as unreachable until they are up, which is normal and looks alarming.
Then check the cluster from a client:
mc alias set prod https://s3-prod.domain.local:9000 <root-user> <root-password>
mc admin info prod
Note the port: mc speaks S3 and must hit the API on 9000, not the console on
or a distribution package.)
- (
mcis no longer downloadable either —go install github.com/minio/mc@latest,
mc admin info lists every node and every drive with its state; fifteen drives online out of fifteen is the answer you want. But that only proves the nodes can see each other. This proves the data path and the certificate chain:
mc mb prod/testbucket
mc cp /etc/hosts prod/testbucket/
mc ls prod/testbucket
mc rb --force prod/testbucket
Once there is data in it, the next thing worth knowing is that deleting an object does not free the space — how to permanently delete objects from disk covers the lifecycle rule that actually reclaims it.
