k8s-watcher

k8s-watcher

k8s-watcher is a k8s container that updates config maps and secrets in real time, and notifies an application when changes occur. It is meant to run as a sidecar container of the application it serves.

Goals

k8s-watcher is very similar to kiwigrid/k8s-sidecar. I wrote this implementation with the following goals in mind:

• learn about k8s watch functionality
• lower system requirements: k8s-sidecar is written in Python. whereas k8s-watcher is written in Go. CPU should not be a concern for a watcher, but in a test scenario, k8s-sidecar uses 81 MB of memory, whereas k8s-watchers uses only 6 MB.
• fun! :)

Usage

The container supports the follows arguments:

  -annotation string
        Annotation name to override target-dir (default "k8s-watcher")
  -debug
        Enable debug logging
  -label-selector string
        Label selector to watch
  -mode string
        configmaps, secrets, or both (default "both")
  -namespace string
        Namespace to watch (default "default")
  -notification-delay duration
        Initial delay to retry notifications (default 1s)
  -notification-delay-factor float
        factor to increase delay on subsequent retry attempts (default 2)
  -notification-max-attempts int
        Maximum number of attempts to retry notifications (default 5)
  -notification-max-delay duration
        maximum delay to retry notifications (default 30s)
  -notification-url string
        URL to send notifications (blank disables notifications
  -target-dir string
        Target directory to store watched resources

A basic example:

apiVersion: apps/v1
kind: Deployment
[...]
      volumes:
        - name: data
          emptyDir: {}
      containers:
        - name: sidecar
          image: codeberg.org/clambin/k8s-watcher:sha-dba0d2a
          args:
            - '-label-selector=app=my-app'
            - '-mode=configmaps'
            - '-namespace=my-namespace'
            - '-target-dir=/data'
          volumeMounts:
            - mountPath: /data
              name: data

The sidecar container watches the namespace 'my-namespace' for configmaps and secrets will a label 'app=my-app'. Matching configmaps and secrets are written to the /data volume, so that an application contain can access them through the same volume.

A note on file system permissions

Since k8s-watcher also supports k8s secrets, all data is written with '0600' permissions (read/write for user, no access for group or world). By default the container runs as uid/gid 1000. To override this, set a securityContent with the required uid:

      securityContext:
        runAsUser: 2000
        runAsGroup: 2000

Overriding the target directory

The target directory can be overridden on a configspec/secret-basis by adding a notification to the configmap/secret:

apiVersion: v1
kind: ConfigMap
metadata:
  name: my-configmap
  annotations:
    k8s-watcher: /some-other-directory
  labels:
    app: my-app

The name of the annotation can be overridden using the -annotation argument

Notifying your application

The main advantage of k8s-watcher is its capability of notifying an application that a configmap or secret has changed ( e.g. so the configuration or secret can be reloaded at runtime):

apiVersion: apps/v1
kind: Deployment
[...]
      volumes:
        - name: data
          emptyDir: {}
      containers:
        - name: sidecar
          image: codeberg.org/clambin/k8s-watcher:sha-dba0d2a
          args:
            - '-label-selector=app=my-app'
            - '-mode=configmaps'
            - '-namespace=my-namespace'
            - '-target-dir=/data'
            - '-notification-url=http://localhost:8888/reload'
          volumeMounts:
            - mountPath: /data
              name: data

k8s-watcher then sends a HTTP POST to the specific URL each time a configmap or secret is found/added, updated or deleted, using the following JSON payload:

{
    "operation":"add/update/delete",
    "object_type":"ConfigMap/Secret",
    "object_name":"object-name"
}

Configuration

Since k8s-watcher uses the k8s API, you will need a ServiceAccount that has access to the right primitives. See example/manifest.yaml for an example of the ServiceAccount, Role and RoleBinding configuration.

Authors

• Christophe Lambin

License

This project is licensed under the MIT License - see the LICENSE.md file for details.