Thu. Aug 11th, 2022

Aproximate time: 20 to 30 minutes

You are wanting to learn what kubernetes is and how it works. I don’t blame you. It is the new buzz these days in IT. But it is also a fascinating way to develop apps / software and test it before you go to market with it. It is also a viable way to Virtualize without having to purchase big expensive servers and run your power bill through the roof. I run 4 clusters a mixture of kubernetes and docker on (wait for it ……wait for it) Raspberry PIs model 4 b. They are a mixture of 4 – 2 gb Pis, 9 – 4 gb Pis and 4 – 8 gb Pis. So bear with us and we will try to learn ya somethin. Let’s get this party started …….

Make sure you burn the os to a SD Card then boot to your RPi use -> http://www.rpihobby.us/tips/ – RPi imager – I use 64 bit version. I choose the Ubuntu Server Lite Version.

Turn on cgroups this allows you to set limits for CPU, Memory and Disk I/O you can isolate a process and the process’s network. Kubernetes will only function with cgroups turned on.
1 – sudo nano -c /boot/firmware/cmdline.txt
2 – copy the line below and paste into the above file.
cgroup_enable=cpuset cgroup_enable=memory cgroup_memory=1 swapaccount=1

Headless mode: Add the network information in this file. This allows you to define the static ip, gw, netmask, also sets the hostname and turns off autoconfig for the main nic. Go to the end of the line and paste the following.
ip=192.168.1.50::192.168.1.1:255.255.255.0:rancherm01:eth0:off

For some reason my dns is acting weird, so I put this in to help alleviate the issue:
You can add ability to give yourself a static DNS & Set the DNS to permanent on Ubuntu Server.
3 – sudo nano -c /etc/resolv.conf
nameserver 1.1.1.1
nameserver 9.9.9.9


4 – sudo apt -y install resolvconf
5 – sudo systemctl start resolvconf.service
6 – sudo systemctl enable resolvconf.service
7 – sudo systemctl status resolvconf.service
8 – sudo nano /etc/resolvconf/resolv.conf.d/head
nameserver 1.1.1.1
nameserver 9.9.9.9

Restart the service to reflect changes:
9 – sudo systemctl restart resolvconf.service
10 – sudo systemctl restart systemd-resolved.service

11 – sudo nano -c /etc/hosts
127.0.1.1 servername
192.168.xxx.xxx mk8-r01-prod.cfs.loc mk8-r01-prod
192.168.xxx.xxx mk8-w01-prod.cfs.loc mk8-w01-prod
192.168.xxx.xxx mk8-w02-prod.cfs.loc mk8-w02-prod
192.168.xxx.xxx mk8-w03-prod.cfs.loc mk8-w03-prod

12 – sudo nano -c /etc/hostname
FQDN ex: server.domain.com

Add a Static IP to your device Ubuntu Server
13 – sudo nano -c /etc/netplan/50-tab 00-tab 01-tab # Do not use all of them only one will work.

network:
  ethernets:
    ens18:
      addresses:
      - 192.168.xxx.xxx/24
      gateway4: 192.168.xxx.xxx
      nameservers:
        addresses:
        - 1.1.1.1
        - 9.9.9.9
        search:
        - cfs.loc
  version: 2

On a Raspberry Pi use the following to config a static IP – change the example static section, it should look like the following:

14 – sudo /etc/dhcpcd.conf

# Example static IP configuration:
interface eth0
static ip_address=192.168.0.10/24
#static ip6_address=fd51:42f8:caae:d92e::ff/64
static routers=192.168.0.1
static domain_name_servers=192.168.0.1 8.8.8.8 
#fd51:42f8:caae:d92e::1

Set the timezone for our server and turn swap off: (Kubernetes will not work with swap on.)

15 - sudo timedatectl set-timezone America/New_York
16 - sudo swapoff -a

Prepare the Server for install and Config of Microk8s:
17 – sudo apt update && sudo apt -y full-upgrade

If on Ubuntu Server OS please go to step 20, otherwise run the following 2 commnds
18 – sudo apt -y install snapd && sudo reboot
19 – sudo snap install core

Add External Storage technology
20 – sudo apt install -y nfs-common snapd open-iscsi

Start and enable nfs
21 – sudo rm -f /lib/systemd/system/nfs-common.service
22 – sudo systemctl daemon-reload
23 – sudo systemctl start nfs-common
24 – sudo systemctl enable nfs-common

Start and Enable iSCSId
25 – sudo systemctl start iscsid
26 – sudo systemctl enable iscsid

Install Kubernetes (microk8s)

27 – sudo snap install microk8s –classic –channel=1.22/stable
28 – sudo microk8s status –wait-read

Build a cluster add nodes for cluster here
29 – sudo microk8s add-node

Stop here until all nodes are ready/added

Repeat steps 1 through 27 to create the worker nodes, then run step 29 to add them to the cluster.

Check your cluster health with this commend:
30 – sudo microk8s.kubectl get no

Let’s enable some kubernetes modules (Master node only please!)
31 – sudo microk8s.enable dns dashboard helm3 ingress rbac storage

Make sure all pods come up before proceeding:
32 – sudo watch -n 1 microk8s kubectl get all –all-namespaces

Add your user to microk8s group to keep from typing sudo everytime.
33 – sudo newgrp microk8s
34 – sudo usermod -a -G microk8s $USER
35 – sudo mkdir -p ~/.kube
36 – sudo chown -f -R $USER ~/.kube

Here you can install Longhorn by Rancher or nfs-subdir-external-provisioner (I will use this one today). STORAGE Alternative for dynamic persistent storage Lets configure our External Storage. In step 38 this is a 7 tb nfs shared storage device.


37 – sudo microk8s helm3 repo add nfs-subdir-external-provisioner https://kubernetes-sigs.github.io/nfs-subdir-external-provisioner/
38 – sudo microk8s helm3 install nfs-subdir-external-provisioner nfs-subdir-external-provisioner/nfs-subdir-external-provisioner \
–set nfs.server=192.168.xxx.xxx \
–set nfs.path=/export/kubetest

Configure the nfs-client as Default, Remove microk8s-hostpath as default storageclass.
39 – sudo microk8s kubectl patch storageclass microk8s-hostpath -p ‘{“metadata”: {“annotations”:{“storageclass.kubernetes.io/is-default-class”:”false”}}}’
40 – sudo microk8s.kubectl get sc
41 – sudo microk8s kubectl patch storageclass nfs-client -p ‘{“metadata”: {“annotations”:{“storageclass.kubernetes.io/is-default-class”:”true”}}}’
42 – sudo microk8s.kubectl get sc

In Portainer
Resources

This article is a living document so there for will change when it is prudent. Please check back with us for updates.

Thank you for coming by, Please contribute and come back now ya hear……

By Michael Cooper

30 years in it, Senior Systems Engineer

Leave a Reply

Your email address will not be published.