Introduction#
Kubernetes Custom Resource Definitions (CRDs) let you add your own resource types to the Kubernetes API. Operators combine CRDs with controllers that watch those resources and reconcile the actual state toward desired state. This pattern is how databases, message brokers, and complex stateful workloads are managed declaratively on Kubernetes.
Custom Resource Definitions#
A CRD defines a new API resource type with its schema.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
name: databases.example.com
spec:
group: example.com
versions:
- name: v1alpha1
served: true
storage: true
schema:
openAPIV3Schema:
type: object
properties:
spec:
type: object
properties:
engine:
type: string
enum: [postgres, mysql]
version:
type: string
replicas:
type: integer
minimum: 1
maximum: 5
storageGB:
type: integer
status:
type: object
properties:
phase:
type: string
connectionString:
type: string
subresources:
status: {} # allow status updates separately from spec
scope: Namespaced
names:
plural: databases
singular: database
kind: Database
shortNames: [db]
After applying this CRD, you can create Database objects:
1
2
3
4
5
6
7
8
9
10
apiVersion: example.com/v1alpha1
kind: Database
metadata:
name: my-postgres
namespace: production
spec:
engine: postgres
version: "15"
replicas: 3
storageGB: 100
Writing an Operator with controller-runtime#
The operator pattern uses a reconcile loop: watch for changes to your CRD, compare actual state to desired state, take actions to converge them.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
// main.go
package main
import (
"context"
"fmt"
ctrl "sigs.k8s.io/controller-runtime"
"sigs.k8s.io/controller-runtime/pkg/client"
appsv1 "k8s.io/api/apps/v1"
corev1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/runtime"
examplev1 "example.com/operator/api/v1alpha1"
)
type DatabaseReconciler struct {
client.Client
Scheme *runtime.Scheme
}
// Reconcile is called whenever a Database resource changes
func (r *DatabaseReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {
log := ctrl.LoggerFrom(ctx)
// Fetch the Database resource
db := &examplev1.Database{}
if err := r.Get(ctx, req.NamespacedName, db); err != nil {
return ctrl.Result{}, client.IgnoreNotFound(err)
}
// Desired StatefulSet
desired := r.buildStatefulSet(db)
// Get existing StatefulSet
existing := &appsv1.StatefulSet{}
err := r.Get(ctx, client.ObjectKey{
Name: db.Name,
Namespace: db.Namespace,
}, existing)
if err != nil {
// Create if not found
log.Info("Creating StatefulSet", "name", db.Name)
if err := r.Create(ctx, desired); err != nil {
return ctrl.Result{}, fmt.Errorf("creating statefulset: %w", err)
}
} else {
// Update if replica count changed
if *existing.Spec.Replicas != int32(db.Spec.Replicas) {
existing.Spec.Replicas = int32ptr(int32(db.Spec.Replicas))
if err := r.Update(ctx, existing); err != nil {
return ctrl.Result{}, fmt.Errorf("updating statefulset: %w", err)
}
}
}
// Update status
db.Status.Phase = "Running"
if err := r.Status().Update(ctx, db); err != nil {
return ctrl.Result{}, fmt.Errorf("updating status: %w", err)
}
return ctrl.Result{}, nil
}
func (r *DatabaseReconciler) buildStatefulSet(db *examplev1.Database) *appsv1.StatefulSet {
replicas := int32(db.Spec.Replicas)
return &appsv1.StatefulSet{
ObjectMeta: metav1.ObjectMeta{
Name: db.Name,
Namespace: db.Namespace,
OwnerReferences: []metav1.OwnerReference{
*metav1.NewControllerRef(db, examplev1.GroupVersion.WithKind("Database")),
},
},
Spec: appsv1.StatefulSetSpec{
Replicas: &replicas,
Selector: &metav1.LabelSelector{
MatchLabels: map[string]string{"app": db.Name},
},
// ... rest of spec
},
}
}
func int32ptr(i int32) *int32 { return &i }
func main() {
mgr, _ := ctrl.NewManager(ctrl.GetConfigOrDie(), ctrl.Options{})
(&DatabaseReconciler{
Client: mgr.GetClient(),
Scheme: mgr.GetScheme(),
}).SetupWithManager(mgr)
mgr.Start(ctrl.SetupSignalHandler())
}
Operator SDK and Kubebuilder#
Most operators are scaffolded with Kubebuilder or Operator SDK rather than written from scratch.
1
2
3
4
5
6
7
8
9
10
11
# Scaffold a new operator project
kubebuilder init --domain example.com --repo example.com/database-operator
# Create an API (CRD + controller)
kubebuilder create api --group example --version v1alpha1 --kind Database
# Generate CRD manifests from Go struct annotations
make manifests
# Run controller locally against the cluster
make run
Owner References and Garbage Collection#
When the operator creates child resources, set owner references so Kubernetes garbage-collects them when the parent is deleted.
1
2
3
4
5
// The OwnerReference in buildStatefulSet above does this automatically.
// When the Database CR is deleted, the StatefulSet is automatically deleted too.
// Verify owner references
kubectl get statefulset my-postgres -o jsonpath='{.metadata.ownerReferences}'
Status Conditions#
Use standard condition types for status reporting:
1
2
3
4
5
6
7
8
9
import "k8s.io/apimachinery/pkg/api/meta"
meta.SetStatusCondition(&db.Status.Conditions, metav1.Condition{
Type: "Ready",
Status: metav1.ConditionTrue,
ObservedGeneration: db.Generation,
Reason: "DatabaseRunning",
Message: "All replicas are running",
})
Conclusion#
CRDs and operators bring Kubernetes’ declarative model to any stateful system. The reconcile loop (fetch desired state → compare to actual state → apply changes) is a powerful pattern that handles partial failures gracefully: if the reconciler crashes mid-way, it simply runs again and converges the state. Use Kubebuilder to scaffold operators rather than writing controller-runtime boilerplate manually.