New to Voyager? Please start here.
Placement of Ingress Pods
Voyager has rich support for how HAProxy pods are placed on cluster nodes. Please check here to understand Kubernetes’ support for pod placement.
Before You Begin
At first, you need to have a Kubernetes cluster, and the kubectl command-line tool must be configured to communicate with your cluster. If you do not already have a cluster, you can create one by using Minikube.
Now, install Voyager operator in your cluster following the steps here.
To keep things isolated, this tutorial uses a separate namespace called demo throughout this tutorial. Run the following command to prepare your cluster for this tutorial:
$ curl -fSsL https://raw.githubusercontent.com/voyagermesh/voyager/v2024.8.30/docs/examples/ingress/pod-placement/deploy-servers.sh | bash
+ kubectl create namespace demo
namespace "demo" created
+ kubectl run nginx --image=nginx --namespace=demo
deployment "nginx" created
+ kubectl expose deployment nginx --name=web --namespace=demo --port=80 --target-port=80
service "web" exposed
+ kubectl run echoserver --image=gcr.io/google_containers/echoserver:1.4 --namespace=demo
deployment "echoserver" created
+ kubectl expose deployment echoserver --name=rest --namespace=demo --port=80 --target-port=8080
service "rest" exposed
Choosing Workload Kind
By default Voyager will run HAProxy pods using Deployment. Since 8.0.1 release, Voyager can run HAProxy pods using either Deployment or DaemonSet. Set the annotation ingress.appscode.com/workload-kind on an ingress object to either Deployment or DaemonSet to enable this feature. If this annotation is missing, HAProxy pods will be run using a Deployment as before.
apiVersion: voyager.appscode.com/v1
kind: Ingress
metadata:
name: ingress-w-node-selector
namespace: demo
annotations:
ingress.appscode.com/workload-kind: DaemonSet
Using Node Selector
Node selectors can be used assign HAProxy ingress pods to specific nodes. Below is an example where ingress pods are run on node with nameminikube.
kubectl apply -f https://raw.githubusercontent.com/voyagermesh/voyager/v2024.8.30/docs/examples/ingress/pod-placement/ingress-w-node-selector.yaml
apiVersion: voyager.appscode.com/v1
kind: Ingress
metadata:
name: ingress-w-node-selector
namespace: demo
annotations:
ingress.appscode.com/type: NodePort
ingress.appscode.com/use-node-port: 'true'
ingress.appscode.com/replicas: '2'
spec:
nodeSelector:
kubernetes.io/hostname: minikube
rules:
- http:
paths:
- path: /
backend:
service:
name: rest
port:
number: 80
- path: /web
backend:
service:
name: web
port:
number: 80
If you are using official networking.k8s.io/v1 ingress api group, use ingress.appscode.com/node-selector annotation to provide the selectors. For example:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-w-node-selector
namespace: demo
annotations:
ingress.appscode.com/type: NodePort
ingress.appscode.com/use-node-port: 'true'
ingress.appscode.com/replicas: '2'
ingress.appscode.com/node-selector: '{"kubernetes.io/hostname": "minikube"}'
spec:
rules:
- http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: rest
port:
number: 80
- path: /web
pathType: Prefix
backend:
service:
name: web
port:
number: 80
Using Pod Anti-affinity
Affinity rules can be used assign HAProxy ingress pods to specific nodes or ensure that 2 separate HAProxy ingress pods are not placed on same node. Affinity rules are set via spec.affinity field in Voyager Ingress CRD. Below is an example where ingress pods are spread over run on node with nameminikube.
kubectl apply -f https://raw.githubusercontent.com/voyagermesh/voyager/v2024.8.30/docs/examples/ingress/pod-placement/ingress-w-pod-anti-affinity.yaml
apiVersion: voyager.appscode.com/v1
kind: Ingress
metadata:
name: ingress-w-pod-anti-affinity
namespace: demo
annotations:
ingress.appscode.com/type: NodePort
ingress.appscode.com/use-node-port: 'true'
ingress.appscode.com/replicas: '2'
spec:
rules:
- http:
paths:
- path: /
backend:
service:
name: rest
port:
number: 80
- path: /web
backend:
service:
name: web
port:
number: 80
affinity:
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: origin
operator: In
values:
- voyager
- key: origin-name
operator: In
values:
- voyager-ingress-w-pod-anti-affinity
topologyKey: 'kubernetes.io/hostname'
Using Taints and Toleration
Using taints and toleration, you can run voyager pods on dedicated nodes.
# taint nodes where only HAProxy ingress pods will run
kubectl taint nodes minikube IngressOnly=true:NoSchedule
kubectl apply -f https://raw.githubusercontent.com/voyagermesh/voyager/v2024.8.30/docs/examples/ingress/pod-placement/ingress-w-toleration.yaml
apiVersion: voyager.appscode.com/v1
kind: Ingress
metadata:
name: ingress-w-toleration
namespace: demo
annotations:
ingress.appscode.com/type: NodePort
ingress.appscode.com/use-node-port: 'true'
ingress.appscode.com/replicas: '2'
spec:
rules:
- http:
paths:
- path: /
backend:
service:
name: rest
port:
number: 80
- path: /web
backend:
service:
name: web
port:
number: 80
tolerations:
- key: IngressOnly
operator: Equal
value: 'true'
effect: NoSchedule
If you are using official networking.k8s.io/v1 ingress api group, use ingress.appscode.com/tolerations annotation to provide the toleration information. For example:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-w-toleration
namespace: demo
annotations:
ingress.appscode.com/type: NodePort
ingress.appscode.com/use-node-port: 'true'
ingress.appscode.com/replicas: '2'
ingress.appscode.com/tolerations: '[{"key": "IngressOnly", "operator": "Equal", "value": "true", "effect": "NoSchedule"}]'
spec:
rules:
- http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: rest
port:
number: 80
- path: /web
pathType: Prefix
backend:
service:
name: web
port:
number: 80
You can use these various option in combination with each other to achieve desired result. Say, you want to run your HAProxy pods on master instances. This can be done using an Ingress like below:
apiVersion: voyager.appscode.com/v1
kind: Ingress
metadata:
name: ingress-w-node-selector
namespace: demo
annotations:
ingress.appscode.com/type: NodePort
ingress.appscode.com/use-node-port: 'true'
ingress.appscode.com/replicas: '2'
spec:
nodeSelector:
node-role.kubernetes.io/master: ""
rules:
- http:
paths:
- path: /
backend:
service:
name: rest
port:
number: 80
- path: /web
backend:
service:
name: web
port:
number: 80
tolerations:
- effect: NoSchedule
key: node-role.kubernetes.io/master
operator: Exists