Using Traefik as a Kubernetes Ingress Controller

Overview

In Kubernetes (K8s), Ingress is an API object that manages external access to the services in a cluster, typically HTTP. Ingress can provide load balancing, SSL termination and name-based virtual hosting. If you are running web services in K8s, you would need an Ingress service to publish your web content to the internet.

In my previous post, I wrote about how to deploy Kubernetes cluster using kubeadmin. In this port, I am gonna share about using Traefik as an Ingress controller for that deployment.

What is Traefik and why I use it?

Traefik is a modern HTTP reverse proxy and load balancer that makes deploying microservices easy. It can be intergrated with your existing infrastructure (Docker, Swarm mode, Kubernetes, Marathon, Consul, Etcd, Rancher, Amazon ECS,…) quickly.

Traefik has some features that might interest you:

• Auto Discovery: Traefik can map your domain name to the backends automatically.
• Tracing: Traefik can be intergrated with Open Tracing, Jaeger & Zipkin.
• Metrics: Traefik can exports the web metrics to Prometheus, Data Dog, StatsD, InfluxDB, etc.
• HTTPS: Let’s Encrypt, ACME, custom certificates, etc.

I found Traefik is easy to use and its auto discovery feature looks awesome to me. Not only HTTP load balancing, Traefik also support TCP now - see PR-4587. However it is in Alpha release at this moment.

One downside is Traefik doesn’t support hitless reload. For me it is not a big deal, but if you care more about the availability of your web service, consider using Nginx, Envoy or HA proxy instead of Traefik.

Some information about Reload vs. Hitless reload that you might want to know:

• Restart: The process is stopped and a new one is started. All existing connection will be terminated.
• Reload: A new process is started and the old process is instructed to close its sockets once it is ready. This results in about 100ns where the old process can refuse connections.
• Hitless reload: The new process reads the connection info from old process via IPC. They hand over the jobs so that there are no refused connections.

Using Traefik in Kubernetes

Step 1: Configure RBAC

First of all, we need to grant privileges to Traefik so it can to access Pods, Endpoints, and Services running in the cluster. There are two ways to set up the proper permission: Via namespace-specific using RoleBindings or a single, global config using ClusterRoleBinding.

If you want to restrict Traefik to watch the service inside some specific namespaces only, use RoleBindings. If you just want to make it simple, use ClusterRoleBinding. In this post, I am gonna use the 2nd method. Create traefik-rbac.yaml file with following content.

traefik-rbac.yaml

---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: traefik-ingress-controller
rules:
  - apiGroups:
      - ""
    resources:
      - services
      - endpoints
      - secrets
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - extensions
    resources:
      - ingresses
    verbs:
      - get
      - list
      - watch
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: traefik-ingress-controller
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: traefik-ingress-controller
subjects:
- kind: ServiceAccount
  name: traefik-ingress-controller
  namespace: kube-system

Then deploy the resouce

$ kubectl apply -f traefik-rbac.yaml

Step 2: Deploy Traefik to the cluster

You can deploy using Deployment or DaemonSet. I am gonna go with using Deployment manifest since it secures better scalability (up and down scaling) and has good support for rolling updates. Let’s create traefik-deployment.yaml file with following content.

traefik-deployment.yaml

---
apiVersion: v1
kind: Namespace
metadata:
   name: traefik
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: traefik-ingress-controller
  namespace: kube-system
---
kind: Deployment
#apiVersion: extensions/v1beta1
apiVersion: apps/v1
metadata:
  name: traefik-ingress-controller
  namespace: kube-system
  labels:
    k8s-app: traefik-ingress-lb
spec:
  replicas: 1
  selector:
    matchLabels:
      k8s-app: traefik-ingress-lb
  template:
    metadata:
      labels:
        k8s-app: traefik-ingress-lb
        name: traefik-ingress-lb
    spec:
      serviceAccountName: traefik-ingress-controller
      terminationGracePeriodSeconds: 60
      containers:
      - image: traefik:1.7-alpine
        name: traefik-ingress-lb
        ports:
        - name: http
          containerPort: 80
        - name: https
          containerPort: 443
        - name: dashboard
          containerPort: 8080
        args:
        - --api
        - --kubernetes
        - --defaultentrypoints=http,https
        - --entrypoints=Name:https Address::443 TLS
        - --entrypoints=Name:http Address::80
        - --logLevel=INFO

Then deploy the resouce

$ kubectl apply -f traefik-deployment.yaml

Verify the deployment

$ kubectl get deployment traefik-ingress-controller -n kube-system

You should see the result

NAME                         READY   UP-TO-DATE   AVAILABLE   AGE
traefik-ingress-controller   1/1     1            1           1m

Now we need to create a Service for the Deployment above. Create traefik-service.yaml file with following content. You might want to update externalIPs to your master node public ip address.

traefik-service.yaml

---
apiVersion: v1
kind: Service
metadata:
  name: traefik-ingress-service
  namespace: kube-system
spec:
  selector:
    k8s-app: traefik-ingress-lb
  ports:
    - protocol: TCP
      port: 80
      name: http
    - protocol: TCP
      port: 443
      name: https
    - protocol: TCP
      port: 8080
      name: dashboard
  externalIPs:
  - <your-public-ip-address>

Then deploy the resouce

$ kubectl apply -f traefik-service.yaml

Verify the deployment of Service

$ kubectl get svc traefik-ingress-service -n kube-system

You should see a similar result

NAME                      TYPE        CLUSTER-IP      EXTERNAL-IP     PORT(S)                   AGE
traefik-ingress-service   ClusterIP   10.96.129.200   103.88.88.88    80/TCP,443/TCP,8080/TCP   1m

Step 3: Deploy the Traefik Dashboard (optional)

This step is optional. Traefik offers a simple dashboard to show the mapped back-end services and some statistics. If you want to use it, create traefik-dashboard.yaml file with following content. You might want to update the host value to your actual domain name which used to access the dashboard.

traefik-dashboard.yaml

---
apiVersion: v1
kind: Service
metadata:
  name: traefik-dashboard
  namespace: kube-system
spec:
  selector:
    k8s-app: traefik-ingress-lb
  ports:
  - name: dashboard
    port: 80
    targetPort: 8080
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: traefik-dashboard
  namespace: kube-system
spec:
  rules:
  - host: traefik-dashboard.ndk.name
    http:
      paths:
      - path: /
        backend:
          serviceName: traefik-dashboard
          servicePort: dashboard

Then deploy the resouce

$ kubectl apply -f traefik-dashboard.yaml

Then open the browser, go to the domain name you sepficied in the host rule. You would see the similar website.

Configure Traefik as an Ingress of your web applications

Traefik supports Name-Based routing which allows you to define a list of front-ends. There front-ends will point to a collection of back-ends which are your web application deployed in the Kubernetes cluster.

I assume that you already have a web application deployed, and a Service was configured for that Deployment. Now what you need to do is creating an Ingress manifest as below:

myweb-ingress.yaml

---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: myweb
  annotations:
    kubernetes.io/ingress.class: traefik
spec:
  rules:
  - host: myweb-address.com
    http:
      paths:
      - path: /
        backend:
          serviceName: myweb
          servicePort: 5000

Then deploy the resouce

$ kubectl apply -f myweb-ingress.yaml

Verify the deployment of Service

$ kubectl get svc myweb

You should see a similar result

NAME         TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)    AGE
myweb        ClusterIP   10.109.22.3      <none>        5000/TCP   1m

If you go to the Traefik Dashboard, you would see the new front-end and back-end discovered automatically by Traefik Ingress controller. Now open your web browser and go to http://myweb-address.com address defined in the host value above to access your web service.

SSL termination with Traefik

Before configuring SSL termination for your web service let’s go back to the Deployment of Traefik, in traefik-deployment.yaml you can see the definition for enabling HTTPS on port 443. You also need make sure you put --entrypoints=Name:https Address::443 TLS in the value list of args.

Now let’s modify the myweb-ingress.yaml file to support HTTPS as below

myweb-ingress.yaml

---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: myweb
  annotations:
    kubernetes.io/ingress.class: traefik
    ingress.kubernetes.io/ssl-redirect: "true"
spec:
  rules:
  - host: myweb-address.com
    http:
      paths:
      - path: /
        backend:
          serviceName: myweb
          servicePort: 5000
  tls:
   - secretName: myweb-cert

You might notice about the difference, now we have another annotation value which is ingress.kubernetes.io/ssl-redirect: "true". This config helps us to force redirection from HTTP to HTTPS with 301 code.

Another config is secretName in the tls section. This indicates the Kubernetes Secret that stores your website SSL certificate. Following is the command to create that secret.

$ kubectl -n default create secret tls myweb-cert --key=cert.key --cert cert.crt

What’s next?

We’ve finished the basic configurations to use Traefik as a Ingress controller in the Kubernetes cluster. For advance configs, you can prefer to their official documentation for the detail. One of the interesting parts is the Annotations that you can directly put them in your Ingress manifest file to enable, disable or adjust Traefik’s features.

For monitoring your Traefik Ingress controller, you can grab the health check and mectrics data from Treafik API or export the them to Prometheus, DataDog, StatsD or InfluxDB using Metric Definition.

Are you running Traefik? Let’s share your experience with it below :-).

UPDATE: Traefik v2

I wrote a new article for the deployment of Traefik 2 into Kubernetes here.