Deploy Kubernetes cluster using kubeadmin on Ubuntu Server

Kubernetes (k8s) is an open-source system for automating deployment, scaling, and management of containerized applications. Kubernetes is so popular nowadays. There are several ways to deploy and use it. Public cloud services such as Google, Azure, Amazon, etc are offering as well. However if you decide to deploy your own k8s cluster from the beginning, you can use kubeadmin. Let’s discover the steps…

Install Container Runtime

The container runtime is the software that is responsible for running containers. Kubernetes supports several runtimes such as Docker, rkt, runc. In this blog post, I will use Docker for the demonstration purpose. Following is the basic steps to install Docker Engine on Ubuntu Server.

$ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
$ sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
$ sudo apt-get update
$ apt-get install -y docker-ce

Each Kubernetes release supports a specific highest version of Docker Engine. You can find that information from the offical release note. So for the best compatibility, we should specify the version of Docker during the installation. For instance as the time of writting this blog post, we should install package docker-ce=18.06.0~ce~3-0~ubuntu for Kubernetes v1.13.2.

Then hold the package and start Docker service

$ sudo apt-mark hold docker-ce
$ sudo systemctl enable docker
$ sudo systemctl start docker

Install Kubernetes Master node

First, we have to add Kubernetes repository in order to have the latest packages.

$ apt-get update && apt-get install -y apt-transport-https curl
$ curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -
$ cat <<EOF >/etc/apt/sources.list.d/kubernetes.list
deb https://apt.kubernetes.io/ kubernetes-xenial main
EOF

Then update APT repository and install Kubernetes’s packages.

$ sudo apt-get update
$ sudo apt-get install -y kubelet kubeadm kubectl

Same as Docker installation, we should add above packages to package manager exclude lists. This will prevent the system from updating Kubernetes’s package which might cause out-of-sync for the cluster.

$ sudo apt-mark hold kubelet kubeadm kubectl

Now we can initialize the Master node with simple command

$ kubeadm init <agrs>

kubeadm init has serveral arguments. However you might want to customzie these following ones:

• --pod-network-cidr to specify the Pod network segment. Depends on 3rd networking plugin you are going to you, the value can be different.
• --apiserver-advertise-address to specify the ip address that Kubernetes API will advertise on. It could be your private ip address, public ip address or 0.0.0.0.
• --apiserver-cert-extra-sans to specify the domain name or ip address you want to add to the certificate sans. Without this argument, the certificate only contains the machine’s hostname and interface’s ip addresses by default.

For example, following command is to initialize the cluster with a pod network that will be used by Flannel network plugin.

$ kubeadm init --pod-network-cidr=10.244.0.0/16

If the command runs successfully, you will get a message like Your Kubernetes master has initialized successfully! and an instruction to modify your current user’s shell environment to have access to new cluster with kube* commands. Let’s follow those instruction:

$ mkdir -p $HOME/.kube
$ sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
$ sudo chown $(id -u):$(id -g) $HOME/.kube/config

There will be another important command for you to join other Kubernetes worker node to this cluster. You might see a similiar output like this.

$ kubeadm join 172.31.39.27:6443 --token bkz1q4.rw5qh1aaml8tpen8 --discovery-token-ca-cert-hash sha256:716a1df8e6751ca0b703716a58a3f89899c51e92e328054df5737b953bfc7865

The kubadmin join command comes with 24-hours TTL token by default. If you want to create a new “join” command, simply run:

$ kubeadm token create --print-join-command

If you want to make token never expire, put --ttl 0. Check kubeadm token command’s document for more detail.

Deploy Kubernetes cluster networking

The cluster networking is needed for the communication between Containers, Pod-to-Pod, Pod-to-Service and external traffic to the Service.

There are several Kubernetes networking models that you can follow. In this post I will use Flannel as an example. To deploy Flannel, just apply the config from their yaml file.

$ kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml

Show our deployment, verify the READY column is 2/2, meaning is the deployment was successfully.

$ kubectl get deployments --all-namespaces
NAMESPACE     NAME      READY   UP-TO-DATE   AVAILABLE   AGE
kube-system   coredns   2/2     2            2           3m6s

Deploy Kubernetes dashboard

$ kubectl create -f https://raw.githubusercontent.com/kubernetes/dashboard/master/aio/deploy/recommended/kubernetes-dashboard.yaml

Verify the deployment and service

$ kubectl get deployment kubernetes-dashboard -n kube-system
NAME                   READY   UP-TO-DATE   AVAILABLE   AGE
kubernetes-dashboard   1/1     1            1           2m22s

$ kubectl get svc kubernetes-dashboard -n kube-system
NAMESPACE     NAME                   TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)         AGE
kube-system   kubernetes-dashboard   ClusterIP   10.111.213.138   <none>        443/TCP         2m23s

Access to Kubernetes Dashboard

We would need to create a ClusterRoleBinding in order to have permission to access the dashboard. Create a file name dashboard-admin.yaml with following content.

dashboard-admin.yaml

apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: kubernetes-dashboard
  labels:
    k8s-app: kubernetes-dashboard
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
- kind: ServiceAccount
  name: kubernetes-dashboard
  namespace: kube-system

You might want to read more about Access Control to have proper grantting.

Then deploy the role

$ kubectl create -f dashboard-admin.yaml
clusterrolebinding.rbac.authorization.k8s.io/kubernetes-dashboard create

Now it is time to access the Dashboard. With above deployment, the dashboard will bind to cluster ip address and listen to port 443 by default. We can verify this by running following command.

$ kubectl get svc --all-namespaces
NAMESPACE     NAME                   TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)         AGE
default       kubernetes             ClusterIP   10.96.0.1        <none>        443/TCP         7m7s
kube-system   kube-dns               ClusterIP   10.96.0.10       <none>        53/UDP,53/TCP   7m3s
kube-system   kubernetes-dashboard   ClusterIP   10.111.213.138   <none>        443/TCP         2m23s

We will not able to connect to this cluster ip address directly. To access it we have to use the kubernetes proxy by running kubectl proxy command.

$ kubectl proxy
Starting to serve on 127.0.0.1:8001

Then we can access the dashboard via proxy url http://localhost:8001/api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy/

However if you are not working in the same machine as the master node. You will not able to access that loopback ip address. So instead of using Cluster-IP we will change it to NodePort.

Let’s edit the deployed service

$ kubectl edit svc kubernetes-dashboard -n kube-system

Find and change the ClusterIP to NodePort. Double check the dashboard’s address again

$ kubectl get svc kubernetes-dashboard -n kube-system
NAME                   TYPE       CLUSTER-IP       EXTERNAL-IP   PORT(S)         AGE
kubernetes-dashboard   NodePort   10.111.213.122   <none>        443:30633/TCP   4m55s

Now you can access to the dashboard direclty by using the node’s address. For example, with above output, the address will be http://10.111.213.122:30633.

Kubernetes dashboard authentication

Authenticate to Kubernetes Dashboard

You can login into Kubernetes dashboard using Kubeconfig or Token. In order to get the token, you can use following command

$ kubectl -n kube-system describe secret $(kubectl -n kube-system get secret | grep dashboard | awk '{print $1}')