StatefulSets are used to manage stateful applications, such as databases or other applications, that keep track of their state.By using StatefulSets, a set of pods can be deployed and scaled within a global namespace, ensuring that they are ordered and unique.Headless service is a regular Kubernetes service where the spec.clusterIP is explicitly set to "None" and spec.type is set to "ClusterIP". Instead, SRV records are created for all the named ports of service's endpoints.
Prerequisite
Onboard the clusters where your services are deployed to Tanzu Service Mesh. For a global namespace, it can be a single cluster with multiple namespaces.
Create a global namespace to which you can add the headless service. For information about creating a global namespace and adding services to it, see Connect Services Across Clusters with a Global Namespace.
Create a Headless service with the label and set the clusterIP field to None.
Context
Deploying and replicating stateful applications poses the following challenges:
Pod replicas cannot be created or deleted at the same time. StatefulSet will not create the next pod until the previous pod is up and running.
Pods are not interchangable. Pod replicas have a persistent identifier across any rescheduling.
Continous data synchronization between pods is necessary to maintain same states.
If all pods die or the clusters running these pods crash, data will be lost.Each pod should have its own persistent storage with replicable data and pod state, stored in the pod's own storage, so that when a pod dies, the Persistent Pod Identifiers will reattach the volume to the replaced pod. Reattaching persistent volumes requires remote storage.
StatefulSet pods get fixed ordered names ($statefulsetname-$ordinal). For example, if you create a StatefulSet with a name of mongo with three replicas, the replicated pods get names mongo-0, mongo-1, and mongo-2.
Most importantly, the stateful workloads often has to reach a specific pod directly (for example, during database write operations) or have pod-pod communication, without load balancing.
Headless Services are the best solution for the above issues. Tanzu Service Mesh implements Headless service for stateful applications that allows clients to directly access pods without using Kubernetes' load balancing. If you deploy an application within a global namespace as StatefulSet with Headless services, Tanzu Service Mesh supports sending traffic to those services from other pods within the cluster or from remote clusters using the Headless service name. Each pod from StatefulSet gets its own DNS name of the form: ${podname}.{governing service name}. When a pod restarts, IP address changes but the name and endpoint remains the same.
When the StatefulSet is in the same global namespace as the source traffic, then the client will use the Kubernetes service associated with the statefulset to resolve the endpoint addresses. Hostnames resolve to all statefulset pod endpoint addresses.
When a client is in a remote cluster but within the same global namespace, it resolves to a number of virtual IPs corresponding to the StatefulSet replica count.Routing is handled by existing global namespace service entries and virtual services.
Creating a Headless Service
A headless service defined in one of the namespaces of a global namespace having the following definition can be extended by creating a selectorless headless service in the other namespaces of the global namespace.
apiVersion: v1 kind: Service metadata: name: foo spec: clusterIP: None selector: key1: val1 key2: val2 ports: - port: 8080 protocol: TCP type: ClusterIP
Create a headless service in the other global namespace namespaces. The selectors which would have been defined in the spec.selector section will be defined in a custom annotation:
apiVersion: v1 kind: Service metadata: name: foo annotations: tsm.tanzu.vmware.com/endpoints.statefulset: '{"key1": "val1", "key2": "val2"}' spec: clusterIP: None ports: - port: 8080 protocol: TCP type: ClusterIPTanzu Service Mesh will auto manage the endpoints for this headless service in the namespaces where the selectorless headless service is defined.
Use case: Kafka Installation with Headless Services for access within a global namespace
Create two namespaces, 'nsOne' and 'nsTwo' on the cluster, 'clusterOne'.
k --context clusterOne create ns nsOne k --context clusterOne create ns nsTwo
Create a namespace 'nsOne' on the cluster, 'clusterTwo'.
k --context clusterTwo create ns nsOne
Create a global namespace called 'kafka-gns' having the following members: 'clusterOne' / 'nsOne', 'clusterOne' / 'nsTwo', and 'clusterTwo' / 'nsOne'.
Install Kafka on 'clusterOne' / 'nsOne'.
#add helm repohelm repo add bitnami https://charts.bitnami.com/bitnami #install kafka on clusterOne/nsOnehelm install kafka --kube-context=clusterOne -n nsOne --set replicaCount=3 bitnami/kafka
Create the following headless service with no selectors in the other namespaces of the global namespaces- 'clusterOne' / 'nsTwo' and 'clusterTwo' / 'nsOne'.
apiVersion: v1 kind: Service metadata: annotations: tsm.tanzu.vmware.com/endpoints.statefulset: '{"app.kubernetes.io/component":"kafka","app.kubernetes.io/instance":"kafka","app.kubernetes.io/name":"kafka"}' name: kafka-headless spec: clusterIP: None ports: - name: tcp-client port: 9092 protocol: TCP targetPort: kafka-client - name: tcp-internal port: 9093 protocol: TCP targetPort: kafka-internalTest the setup by running a producer and consumer from any of the global namespace members. The following command would start and open a prompt within a temporary kafka-client container.
kubectl --context=${cluster} --namespace ${ns} run kafka-client --image docker.io/bitnami/kafka:3.1.0-debian-10-r89 --rm -it --command -- shAt the prompt within the temporary kafka-client container, create a producer and a consumer using the . service name.
# create topic and populate it with messages kafka-producer-perf-test.sh --topic topic-foo --num-records 10000 --throughput -1 --record-size 1000 --producer-props bootstrap.servers=kafka:9092 # consume messages from the previously created topic kafka-consumer-perf-test.sh --bootstrap-server kafka:9092 --topic topic-foo --messages 10000 --timeout 10000
Advantages of Headless Services
Direct access to each pod.
Easy Pod discovery in the StatefulSet.
Pods can be addressed more generally by using their DNS names.
Utilizes each pod's sticky identity in a stateful service (i.e. you can address a specific pod by name).
Write operations are synchronized.
