workloads

Kubernetes workloads, by its API

Returns, for each deployment, statefulset and daemonset of a Kubernetes cluster, or of one namespace, how many replicas it wants, how many are ready, and its state from the two, read from the API server with the token of a read-only service account. Two watches are offered, each one set on the probe.

Versions

Generic: the apps/v1 API of deployments, statefulsets and daemonsets, stable since Kubernetes 1.9, in every Kubernetes still supported. It stays at the common level.

How it works

Three lists, the token in the header Authorization: Bearer <token>, each page by page, following metadata.continue to the last page:

1. GET https://<host:address>:6443/apis/apps/v1/deployments?limit=500
2. GET https://<host:address>:6443/apis/apps/v1/statefulsets?limit=500
3. GET https://<host:address>:6443/apis/apps/v1/daemonsets?limit=500

or /apis/apps/v1/namespaces/<namespace>/... when the variable kubernetes:namespace names one.

{"items": [{"metadata": {"namespace": "shop", "name": "web"}, "spec": {"replicas": 3}, "status": {"readyReplicas": 2}}], "metadata": {"continue": ""}}
{"items": [{"metadata": {"namespace": "kube-system", "name": "kube-proxy"}, "status": {"desiredNumberScheduled": 5, "numberReady": 5}}], ...}

• A workload is named <kind>/<namespace>/<name> (deployment/shop/web): the selection of the probe keeps or leaves out a kind (daemonset/*) or a namespace (*/kube-system/*).
• For a deployment or a statefulset, the replicas wanted are spec.replicas and the ready ones status.readyReplicas, which Kubernetes leaves out at zero; for a daemonset, status.desiredNumberScheduled, one per node it fits, and status.numberReady.
• The state is ready when all the replicas wanted are ready, degraded when some are, down when none is, and idle when the workload wants none (scaled to zero, or a daemonset that fits no node).
• A code 401 (the token refused) or 403 (the account may not list one of the three) stops the collector misconfigured; any other code but 200 stops it unreachable. No workload at all is an answer: a namespace may be empty.

The token comes from protocol:http:token, a secret variable: it only goes in the header. The answer is not secret.

Prerequisites

• The address of the API server, its port and its certificate, as for the collector of the nodes.
• A service account allowed to list the three and nothing else, and its long-lived token in protocol:http:token. The service account and its token Secret are those of the documentation of the collector of the nodes; the rule:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: banshee-workloads
rules:
  - apiGroups: ["apps"]
    resources: ["deployments", "statefulsets", "daemonsets"]
    verbs: ["list"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: banshee-workloads
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: banshee-workloads
subjects:
  - kind: ServiceAccount
    name: banshee
    namespace: monitoring

For one namespace only, set in kubernetes:namespace, a Role and a RoleBinding in that namespace, the same rule, in place of the ClusterRole and its binding.

Variables

Outputs

Watches

• A rollout takes replicas down and brings new ones up: not-ready raises only when it lasts five minutes.

Limits

• The replicas counted are the ready ones, not the available ones: a replica counts as soon as its readiness probe passes, before the workload's minReadySeconds.
• A workload deleted is no longer listed: only missing, with its name set on the probe, sees it go.
• The jobs and cronjobs are not read.
• The address is written into the URL as is: an IPv6 address must be given as a name.
• Sources: spec.replicas (1 when not written), readyReplicas, desiredNumberScheduled and numberReady are those of the Kubernetes API reference (Deployment, StatefulSet, DaemonSet, apps/v1). Not run against a cluster.