Create a Ceph Cluster on Ubuntu with cephadm

Ceph is the distributed storage system behind most serious open-source private clouds: it pools the disks across many servers into a single, self-healing cluster that serves block, object, and file storage at once. On Ubuntu 24.04 the supported way to deploy it is cephadm, which bootstraps and manages the cluster as containers via systemd — no fragile hand-rolled package installs.

This walkthrough builds a small but real Ceph cluster from scratch: prepare the hosts, bootstrap the first monitor with cephadm, add OSDs from raw disks, check cluster health, and create your first storage pool. It mirrors how clouditiv runs Ceph as the storage backbone underneath OpenStack.

Prerequisites

You need three Ubuntu 24.04 hosts (one is enough to learn, but three give you real replication), each with at least one empty, unpartitioned disk for an OSD beyond the OS disk. You also need time-synchronised clocks (chrony), passwordless SSH from the bootstrap node to the others as root, and at least 4 GiB RAM per node. Pick a private management network — the examples use 10.10.10.0/24.

Step 1: Prepare every host

On all nodes, install prerequisites and ensure time sync — Ceph is intolerant of clock skew between monitors:

sudo apt update
sudo apt install -y chrony lvm2 podman
sudo systemctl enable --now chrony

Confirm each host can resolve the others by name. Add entries to /etc/hosts if you have no internal DNS:

sudo tee -a /etc/hosts <<'EOF'
10.10.10.11 ceph-1
10.10.10.12 ceph-2
10.10.10.13 ceph-3
EOF

Step 2: Install cephadm on the bootstrap node

On the first node (ceph-1), install the cephadm tool. The distro package is fine, but pinning a release with the cephadm script is cleaner:

sudo apt install -y cephadm ceph-common

Verify it is available:

cephadm version

Step 3: Bootstrap the cluster

Bootstrapping creates the first monitor and manager, generates keys, and launches the dashboard. Point it at the bootstrap node's IP:

sudo cephadm bootstrap --mon-ip 10.10.10.11

When it finishes, cephadm prints the dashboard URL and a one-time admin password — save them. From here, run Ceph commands inside the cephadm shell, which has the admin keyring mounted:

sudo cephadm shell -- ceph -s

Expect a HEALTH_WARN at first (too few OSDs); that is normal before you add disks.

Step 4: Add the other hosts to the cluster

cephadm manages remote nodes over SSH using its own key. Copy the cluster's public key to each new host, then register them:

ssh-copy-id -f -i /etc/ceph/ceph.pub root@ceph-2
ssh-copy-id -f -i /etc/ceph/ceph.pub root@ceph-3
sudo cephadm shell -- ceph orch host add ceph-2 10.10.10.12
sudo cephadm shell -- ceph orch host add ceph-3 10.10.10.13

List the hosts the orchestrator now knows about:

sudo cephadm shell -- ceph orch host ls

Step 5: Add OSDs (your storage)

An OSD (Object Storage Daemon) manages one disk. First see which devices Ceph considers available across the cluster:

sudo cephadm shell -- ceph orch device ls

The simplest approach is to let Ceph consume every available disk automatically:

sudo cephadm shell -- ceph orch apply osd --all-available-devices

To add a specific disk by hand instead, target the host and device path:

sudo cephadm shell -- ceph orch daemon add osd ceph-2:/dev/sdb

Step 6: Verify cluster health

Give the OSDs a minute to come up, then check status. A healthy three-node cluster reports HEALTH_OK:

sudo cephadm shell -- ceph -s
sudo cephadm shell -- ceph osd tree

Expected output for a good cluster looks like:

  cluster:
health: HEALTH_OK
services:
mon: 3 daemons, quorum ceph-1,ceph-2,ceph-3
mgr: ceph-1(active)
osd: 3 osds: 3 up, 3 in

Step 7: Create your first pool

Data lives in pools. Create a replicated RBD pool (three copies) and initialise it for block-device use:

sudo cephadm shell -- ceph osd pool create rbd 64 64
sudo cephadm shell -- ceph osd pool application enable rbd rbd
sudo cephadm shell -- rbd pool init rbd

Confirm the pool exists and check usage:

sudo cephadm shell -- ceph osd pool ls
sudo cephadm shell -- ceph df

Troubleshooting and pitfalls

• Stuck on HEALTH_WARN ‘too few OSDs’ — you have fewer OSDs than the pool's replication size (default 3). Add disks or lower the size for a lab.
• Device shows as unavailable — the disk has a partition table or old LVM signature. Wipe it with ceph orch device zap ceph-2 /dev/sdb --force.
• Clock skew detected — chrony is not synced on a node; Ceph monitors require tight time agreement.
• SSH errors adding hosts — the cluster public key is not in the target's authorized_keys; re-run the ssh-copy-id step.

Where to go next

With a healthy cluster and an RBD pool, the payoff is attaching that resilient storage to virtual machines. Continue with using Ceph RBD as KVM/libvirt VM storage. If you are heading toward a full cloud, the same pools back Cinder block volumes in OpenStack.

Conclusion

You have bootstrapped a self-healing Ceph cluster with cephadm, added OSDs, reached HEALTH_OK, and created a pool ready for real workloads. Ceph is precisely the storage backbone clouditiv operates beneath OpenStack — replicated, scalable, and fault-tolerant by design. Running Ceph in production means capacity planning, failure-domain design, and 24/7 monitoring; if you would rather consume that resilience than operate it, clouditiv runs a sovereign, ISO 27001 / BSI C5 private cloud on Ubuntu 24.04 + OpenStack 2025.2 with Ceph storage and your data in Germany. See our on-premise cloud solution.