etcd Security & Backup | Kubernetes Security

etcd Security Overview

etcd is Kubernetes' data store—it holds all cluster configuration, state, and secrets. Compromise of etcd means complete cluster compromise. Securing etcd requires encryption, access control, backup strategies, and disaster recovery.

⚠️ Critical: etcd contains all cluster data including unencrypted secrets by default. It must be protected with the highest level of security.

Encryption at Rest

Configure etcd Encryption

Enable encryption for etcd data in Kubernetes:

# In /etc/kubernetes/manifests/kube-apiserver.yaml
- --encryption-provider-config=/etc/kubernetes/enc/encryption-config.yaml
- --encryption-provider-config-automatic-reload=true
            

Encryption Configuration

apiVersion: apiserver.config.k8s.io/v1
kind: EncryptionConfiguration
resources:
- resources:
  - secrets
  providers:
  - aescbc:
      keys:
      - name: key1
        secret: 
  - identity: {}
            

Generate Encryption Key

# Generate 32-byte key and base64 encode
head -c 32 /dev/urandom | base64
            

etcd Peer Security

TLS Configuration

Peer TLS: Encrypt communication between etcd members
Client TLS: Encrypt API server to etcd communication
Certificate Management: Regular certificate rotation

Configure Peer TLS

--cert-file=/etc/kubernetes/pki/etcd/server.crt
--key-file=/etc/kubernetes/pki/etcd/server.key
--peer-cert-file=/etc/kubernetes/pki/etcd/peer.crt
--peer-key-file=/etc/kubernetes/pki/etcd/peer.key
--peer-trusted-ca-file=/etc/kubernetes/pki/etcd/ca.crt
            

Access Control

Restrict etcd port (2379) to authorized clients only
Use firewall rules to block direct access
Only API server should directly access etcd

etcd Backup Strategy

Manual Backup

# Snapshot etcd database
sudo etcdctl --endpoints=127.0.0.1:2379 snapshot save backup.db

# Verify snapshot
sudo etcdctl --write-out=table snapshot status backup.db
            

Automated Backup with Velero

Use Velero for cluster-wide backup including etcd:

Automated backup schedules
Point-in-time recovery
Multi-cloud support
Restore to different clusters

Backup Locations

S3/Cloud Storage: Offsite encrypted backups
Network Storage: NFS or similar for quick recovery
Multiple Regions: Geo-redundant backups

Disaster Recovery

Restore from Backup

# Stop all API servers
# Stop all etcd members

# Restore from backup on one member
sudo etcdctl snapshot restore backup.db \
  --data-dir=/var/lib/etcd \
  --initial-cluster=etcd0=https://master1:2380 \
  --initial-advertise-peer-urls=https://master1:2380 \
  --name=etcd0

# Start etcd and API servers
            

Recovery RTO/RPO

RTO (Recovery Time Objective): Minimize recovery time
RPO (Recovery Point Objective): Minimize data loss
Regular backup testing essential
Document and practice recovery procedures

💡 Recommendation: Back up etcd every hour and test recovery procedures quarterly.

etcd Security Checklist

✓ Enable encryption at rest for etcd data
✓ Configure TLS for peer and client communication
✓ Implement access control to etcd
✓ Enable audit logging for API server
✓ Regular automated backups (hourly)
✓ Offsite backup storage
✓ Test disaster recovery quarterly
✓ Monitor etcd performance and health
✓ Keep etcd updated