Liking cljdoc? Tell your friends :D

dda-k8s-crate

| team@social.meissa-gmbh.de | Website & Blog

This crate is part of dda-pallet.

Kubernetes setup

This crate uses Kuberentes to initialize and build a single host Kubernetes cluster.

Requirements

This crate should fullfill certain requirements for the installed Kubernetes cluster

a small k8s all in one system for serving one application.
is compatible with k8s
providing ingress for app to be installed (replacement of traditional reverse-proxy httpd)
support letsencrypt (dynamic created by https) for a defined fqdn or injected static https certs.
exposes dashboard with users defined & disabled annonymous access.

As our services need to be exposed to the outside by a URL we use Ingress to do just that. Ingress requires some sort of load balances attached in order to acquire an external IP. As we do not want to install Kubernetes on a third party cloud platform we require MetalLB to give Ingress an external IP. Ingress and MetalLB is configured in a dedicated file usually annotated with ingress_*.yml. There is a unique Ingress configuration for each type of certificate used and they can all be active at the same time and listen to different URLs. However, the config files are somewhat similar and most important config options are:

the name given for the ingress instance
the (cluster-)issuer to be used with that ingress instance
host(s) name and corresponding seceretName
the rules for redirecting and which service port to forward to

MetalLB configuration resiedes in a seperate metallb_config.yml file where the host ip needs to be changed to the actual external if of the host. metallb.yml is responsible for the installation of Metallb but does not need to be adjusted for our installation. [Ingress] [MetalLB]

Flannel networkking

Kubernetes requires a way for multiple hosts to commuicate with each other. Flanne handles the layer 3 networking of Kubernetes. We do not use any custom configuration of flannel. [Flannel] [Kubernetes Cluster Networking]

cert-manager and handling of https and certificates

Ingress together with MetalLB is not enough to handle https and ca certificates for our Kubernetes cluster. Additionally, we require cert-manager to issue certificates for us. The workflow for either letsencrypt or ca issuer requires three configs to be applied:

a secret representing the ca-key pairs needs to be created
a config file for the actual certificate needs to be created
an issuer of either self-signed, ca-issuer or letsencrypt issuer

The self-signed issuer does not require a secret or certificate config file. For the letsencrypt issuer we distinguish between an issuer for staging and one for production.

Nexus repository manager

The nexus repository manager is the first application we deploy on the Kubernetes cluster. It uses the lestsencrypt-prod-issuer and requires an ingress config file on its own. In addition to the ingress config file a nexus-storage.yml and nexus.yml config file are necessary. The storage is of kind PersistenVolume and differentiates the storage and actualy application of nexus in two seperate Pods.

Dashboard

The dashboard allows for debugging and overview of the Kubernetes cluster and should later also provide networking. To secure the dasboard, an ssh proxy is required and the dashboard can then be accessed via a http://localhost:8001/api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy in the browser.