Building a Kubernetes operator with Python

Kubernetes has become the industry standard for container orchestration. Managing complex applications on Kubernetes can be challenging, requiring manual intervention and specialized knowledge. This is where Kubernetes Operators simplify things.

An Operator is a software extension to Kubernetes that simplifies the deployment and management of complex applications. They do this by using Custom Resources. They encapsulate operational knowledge and best practices, automating tasks like scaling, updates, and failure recovery.

This blog post will guide you through developing a Kubernetes Operator using Python. We'll cover key concepts, choose a framework, walk through a simple example, and provide kubectl commands for deployment and verification.

Why Python for Kubernetes Operators?

While Go is the dominant language in the Kubernetes ecosystem, Python offers several advantages for Operator development:

• Ease of Use: Python is renowned for its simplicity and ease of learning.
• Rich Ecosystem: Python boasts a vast collection of libraries, including those for Kubernetes interaction (like pykube and kopf).
• Rapid Development: Python's dynamic typing and concise syntax facilitate faster development cycles.

Understanding Key Concepts

Before diving into code, let's clarify some essential concepts:

• Custom Resource Definition (CRD): CRDs extend the Kubernetes API by defining new object types called Custom Resources, representing your application. They specify the schema and validation rules for these resources.
• Custom Resource (CR): A CR is an instance of a CRD. It holds the specific configuration and state of your application.
• Controller: A controller is a software component that watches for changes to CRs and takes actions to reconcile the desired state with the actual state.
• Reconciliation Loop: This is the core logic of an Operator. It continuously compares the desired state defined in CRs with the actual state of the cluster and takes actions to rectify any discrepancies.

Image credit: https://iximiuz.com/en/posts/kubernetes-operator-pattern/

Choosing a Framework: Kopf

Kopf (Kubernetes Operator Pythonic Framework) is a popular choice for developing Python Operators. It offers several benefits:

• Pythonic API: Kopf provides a user-friendly, Pythonic way to define Operator logic.
• Declarative Style: Kopf emphasizes a declarative approach, making Operator logic more readable and maintainable.
• Built-in Features: Kopf comes with features like event handling, logging, and error management.

A Simple Example

Let's illustrate these concepts with a basic Operator that manages a custom resource called "ExampleResource." This example will create a Deployment based on the specifications provided in an instance of ExampleResource.

1. Define the CRD:

Save the following CRD definition to a file named example-crd.yaml.

apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  name: exampleresources.example.com # . format
spec:
  group: example.com
  scope: Namespaced
  names:
    plural: exampleresources
    singular: exampleresource
    kind: ExampleResource
  versions:
    - name: v1alpha1
      served: true
      storage: true
      schema:
        openAPIV3Schema:
          type: object
          properties:
            spec: # Specification for the ExampleResource
              type: object
              properties:
                replicas: # Number of replicas
                  type: integer
                image: # Docker image for the custom resource
                  type: string
                # ... other properties as needed

This CRD defines an "ExampleResource" resource with properties like replicas and image. Apply this CRD to your Kubernetes cluster using the following kubectl command:

kubectl apply -f example-crd.yaml

2. Implement the Operator Logic:

Save the following Python Operator code to a file named example-operator.py.

import kopf
import pykube #library that simplifies interaction with k8s API

@kopf.on.create("example.com", "v1alpha1", "exampleresources") # Watch for ExampleResource creation events
def create_fn(spec, **kwargs): # Function to handle creation events
    kube_api = pykube.HTTPClient(pykube.KubeConfig.from_file()) # Kubernetes API client
    deployment = pykube.Deployment(kube_api, {
        # Define the deployment object based on the ExampleResource spec
        'apiVersion': 'apps/v1',
        'kind': 'Deployment',
        # ... other deployment properties
    })
    kopf.adopt(deployment) # Mark deployment as managed by the Operator
    deployment.create() # Create the deployment
    return {'message': 'Deployment created'}

This code snippet demonstrates a simple Operator that listens for the creation of "ExampleResource" resources. When a new ExampleResource is created, the Operator creates a corresponding Kubernetes Deployment based on the ExampleResource's specification.

3. Containerize the Operator

You'll need a Dockerfile to containerize your Operator. Create a file named Dockerfile in the same directory as your example-operator.py with the following content:

FROM python:3.9

WORKDIR /app
COPY example-operator.py .
COPY requirements.txt .

RUN pip install --no-cache-dir --upgrade pip \
    && pip install --no-cache-dir -r requirements.txt

CMD ["python", "example-operator.py"]

Make sure you have a requirements.txt file listing your project's dependencies (like kopf and pykube).

Build and push the Docker image:

docker build -t /example-operator:v1 .
docker push /example-operator:v1

Replace with your Docker Hub username or the repository you're using.

4. Create Operator Deployment YAML

Create a file named operator-deployment.yaml with the following content, making sure to update the image name:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: example-operator
spec:
  replicas: 1
  selector:
    matchLabels:
      app: example-operator
  template:
        metadata:
          labels:
            app: example-operator
        spec:
          containers:
          - name: example-operator
            image: /example-operator:v1

Apply the deployment to your cluster using:

kubectl apply -f operator-deployment.yaml

5. Verify Operator Pod

Make sure the Operator pod is running:

kubectl get pods -l app=example-operator

You should see a pod running for the example-operator.

6. Create an Instance of ExampleResource

Create a file named example-resource.yaml with the following content:

apiVersion: example.com/v1alpha1 
kind: ExampleResource
metadata:
  name: my-example-resource
spec:
  replicas: 3
  image: nginx:latest 

Apply this resource to your cluster:

kubectl apply -f example-resource.yaml

7. Verify Deployment Creation

The Operator should automatically create a Deployment based on the ExampleResource. Check for it using:

kubectl get deployments 

You should see a deployment created with a name derived from your ExampleResource (e.g., my-example-resource).

Additional Considerations

• Error Handling: Implement robust error handling and logging mechanisms to ensure Operator stability.
• Permissions: Define appropriate RBAC roles and role bindings to grant the Operator necessary permissions.
• Testing: Thoroughly test your Operator to ensure it behaves as expected under different scenarios.

Conclusion

Developing Kubernetes Operators with Python empowers you to automate the management of complex applications, enhancing efficiency and reducing errors. By leveraging frameworks like Kopf and understanding core concepts like CRDs, CRs, and reconciliation loops, you can create robust and effective Operators to streamline your Kubernetes workflows. Refer to the references section below for more detailed information on Operators, Kopf, and related Kubernetes concepts.

References

1. de Bree, D. (2024, April 25). Build your own Python Kubernetes Operator.
2. Bishop, S. (n.d.). Building Kubernetes Operators with Python and Kopf. Kiwi PyCon.
3. Imran, S. (2024, February 24). Empowering Kubernetes Operator Development with Kopf: Getting Started with writing Operator (Part-2). Medium.
4. Imran, S. (2024, February 17). Empowering Kubernetes Operator Development with Kopf: Introduction to Operators (Part-1). Medium.
5. Shahverdiev, J. (n.d.). Kubernetes Operator with Python. YouTube.
6. Operator pattern. (2024, July 16). Kubernetes.
7. Tiram, O. (2022, September 22). Writing Kubernetes Operators with Python. Spectro Cloud.
8. Kumar, Y. (n.d.). Writing your own Kubernetes operator and CRD in Python with kopf to enhance and harden your Kubernet. Python India.