MicroK8s with external etcd⚓︎
Warning
Canonical explicitly discourages the use of etcd within MicroK8s. For almost every use case, the zero ops approach using dqlite is a more resilient and easier to manage solution.
MicroK8s supports integration with an external etcd cluster for storing Kubernetes data. This can provide improved performance and reliability for your MicroK8s cluster, but comes with an additional maintenance overhead.
Common commands⚓︎
Configuring etcdctl⚓︎
To configure etcdctl to connect to the etcd cluster, you need to set the
ETCDCTL_API environment variable to 3 and the ETCDCTL_ENDPOINTS
environment variable to the etcd cluster endpoint. For example:
export ETCDCTL_API=3
export ETCDCTL_ENDPOINTS='https://node1:2379,https://node2:2379,https://node3:2379'
If you are using client certificates for authentication, you can set the
ETCDCTL_CERT_FILE, ETCDCTL_KEY_FILE, and ETCDCTL_CA_FILE environment
variables to the paths of the client certificate, client key, and CA certificate
files, respectively:
export ETCDCTL_CERT_FILE=/path/to/client.crt
export ETCDCTL_KEY_FILE=/path/to/client.key
export ETCDCTL_CA_FILE=/path/to/ca.crt
Check the status of the etcd cluster⚓︎
etcdctl endpoint status --write-out=table
Check the health of the etcd cluster⚓︎
etcdctl endpoint health --write-out=table
FAQ⚓︎
How do I back up my etcd data?⚓︎
You can back up your etcd data by using the etcdctl snapshot save command. Refer to the etcd backup and restore documentation for detailed instructions.
How do I restore my etcd data from a backup?⚓︎
You can restore your etcd data from a backup using the etcdctl snapshot restore command. Refer to the etcd backup and restore documentation for detailed instructions.
How do I monitor the health of my etcd cluster?⚓︎
You can monitor the health of your etcd cluster using the etcdctl endpoint health command. Additionally, you can use monitoring tools like Prometheus to collect metrics from etcd. Refer to the etcd monitoring documentation for more information.
How do I manage etcd cluster membership?⚓︎
You can manage etcd cluster membership using the etcdctl member add, etcdctl member remove, and etcdctl member list commands. Refer to the etcd cluster reconfiguration documentation for detailed instructions.
What should I do if my etcd cluster becomes unavailable?**⚓︎
Consult the etcd disaster recovery documentation for guidance on recovering from etcd failures.
etcd has run out of space, what should I do?⚓︎
If etcd has run out of space, no changes can be made to the cluster and it is put into "alarm" mode.. You can try to free up space by deleting old snapshots or compacting the etcd database.
To compact the database and defragment it, you can use the following commands:
# identify current revision
etcdctl endpoint status --write-out="json" | egrep -o '"revision":[0-9][0-9]*'
# compact to that revision
etcdctl compact <revision>
# Defragment
etcdctl --command-timeout=300s defrag
Refer to the etcd maintenance documentation for more information.
You might also want to enable auto compaction to prevent this from happening in the future.
To disable the alarm and resume normal operation, you can use the following command:
etcdctl alarm disarm
Troubleshooting⚓︎
MicroK8s reports cluster not running but processes are active⚓︎
If microk8s status reports the cluster is not running, but htop shows MicroK8s processes are still active, and microk8s start does not resolve the issue, the internal certificates may have expired.
Check the certificate status:
microk8s refresh-certs -c
If certificates are expired, refresh them:
microk8s refresh-certs