GitOps with Argo CD on Leaseweb Kubernetes | Part 1
In this article, we will go from a default Leaseweb managed Kubernetes cluster to a production-ready cluster capable of delivering a stable and secure environment for all deployments. The goal of this article is not to present the only, or even the best, approach to achieve a production-ready cluster. As is the norm in IT, there are multiple different ways of accomplishing things, and selecting the best or most appropriate depends on the goal, the org, the people involved, and a multitude of other factors. As such, the goal here is merely to offer an approach to be tweaked and adjusted to each specific implementation.
To start, a Kubernetes cluster is needed. Follow these pages to provision your Leaseweb cluster (https://kb.leaseweb.com/kb/managing-a-kubernetes-cluster/configure-a-kubernetes-cluster/) and get the kubeconfig file (https://kb.leaseweb.com/kb/kubernetes/getting-started-with-kubernetes-overview/).
It is recommended to have at least 4 worker nodes for high availability.
Note
If you have fewer than 4 worker nodes, use the non-high availability manifests both when installing Argo CD and on the repo in the kustomization.yaml file.
Argo CD
Install Argo CD
To install Argo CD, the upstream documentation can be followed and is available here (https://argo-cd.readthedocs.io/en/stable/)
First, create the namespace where Argo CD will be installed:
kubectl create namespace argocd
Then apply the installation manifests:
kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/ha/install.yaml
Access Argo CD
To log in, the default admin password is needed. It is created during installation and saved in the argocd-initial-admin-secret. To get it from the cluster run:
kubectl -n argocd get secret argocd-initial-admin-secret -o jsonpath='{.data.password}' | base64 -d ; echo
By default, Argo CD’s API server is not exposed outside the cluster. To get access, port forwarding needs to be configured:
kubectl port-forward svc/argocd-server -n argocd 8080:443
Now it is possible to log in with the username admin and the password retrieved in the previous step by navigating to http://localhost:8080.
While Argo CD is installed and ready to be used, it is not yet production ready.
Manage Argo CD using Argo CD
Since Argo CD is a declarative, GitOps continuous delivery tool for Kubernetes, it makes sense to manage Argo CD following these same principles. To do that a git repository is needed. The creation and configuration of the repository is outside of the scope of this article.
To configure access to the repository, a secret needs to be created. Depending on the way the repository should be accessed, the secret needs to be different. To see the different examples refer to the official documentation here (https://argo-cd.readthedocs.io/en/stable/operator-manual/declarative-setup/#repositories).
In this article https will be used. Create a secret with this manifest (adjust the git repo url, password and username):
kubectl apply -f - <<EOF
apiVersion: v1
kind: Secret
metadata:
name: private-repo
namespace: argocd
labels:
argocd.argoproj.io/secret-type: repository
stringData:
type: git
url: <git repo url>
password: <password>
username: <username>
EOF
After the repository is created an ApplicationSet will be used to keep the cluster in sync with the code in the repo (adjust the git repo url):
kubectl apply -f - <<EOF
apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
name: apps
namespace: argocd
spec:
generators:
- git:
repoURL: <git repo url>
revision: HEAD
directories:
- path: apps/*/*
template:
metadata:
name: '{{path.basename}}'
spec:
project: default
source:
repoURL: <git repo url>
targetRevision: HEAD
path: '{{path}}'
destination:
server: https://kubernetes.default.svc
namespace: '{{path[1]}}'
syncPolicy:
automated:
prune: true
selfHeal: true
syncOptions:
- CreateNamespace=true
- PruneLast=true
- ServerSideApply=true
EOF
This ApplicationSet will create a namespace per directory under the apps directory on the repo and an application per directory under the specific namespace directory. For illustration:
Directory structure example:
.
├── apps
│ ├── argocd
│ │ └── argocd
│ ├── namespace1
│ │ ├── application1
│ │ ├── application2
│ │ ├── application...
│ ├── namespace2
│ │ ├── application3
│ │ ├── application4
│ │ ├── application...
│ ├── namespace...
To create a new application, simply create a new directory in the desired namespace containing the manifests as exemplified in this article.
After these bootstrap steps, it is time to start following GitOps. To start, create the argocd application.
To do that, on the repo, create the directories for the namespace and application – argocd/argocd – and then a kustomization file with the following contents:
apps/argocd/argocd/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: argocd
resources:
- base/applicationset.yaml
- https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/ha/install.yaml
And a directory called base with the ApplicationSet manifest file (adjust the git repo url):
apps/argocd/argocd/base/applicationset.yaml
apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
name: apps
namespace: argocd
spec:
generators:
- git:
repoURL: <git repo url>
revision: HEAD
directories:
- path: apps/*/*
template:
metadata:
name: '{{path.basename}}'
spec:
project: default
source:
repoURL: <git repo url>
targetRevision: HEAD
path: '{{path}}'
destination:
server: https://kubernetes.default.svc
namespace: '{{path[1]}}'
syncPolicy:
automated:
prune: true
selfHeal: true
syncOptions:
- CreateNamespace=true
- PruneLast=true
- ServerSideApply=true
This will make Argo CD declaratively manage itself.
Traefik ingress controller
At this stage, Argo CD is only accessible with port forwarding enabled. To have it accessible from the outside using a human-readable website name, traefik will be used as ingress. As stated in the introduction, an ingress controller is only one example of how to make Argo CD accessible, and traefik is only one example of an ingress controller. For other possibilities, check the official documentation (https://argo-cd.readthedocs.io/en/stable/operator-manual/ingress/). To cover the human-readable website name, a DNS name is needed. The DNS configuration is outside of the scope of this article.
To install it, create the new directories under apps for the namespace and application and add the following helm chart:
apps/traefik/traefik/Chart.yaml
apiVersion: v2
name: traefik
version: 0.1.0
dependencies:
- name: traefik
version: "*"
repository: https://traefik.github.io/charts
With traefik installed, Argo CD needs to be configured to make use of it. First, argocd-server should run with TLS disabled. To disable it, a kustomization patch will be used.
First create the patch:
apps/argocd/argocd/overlays/argocd-cmd-params-cm.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: argocd-cmd-params-cm
data:
server.insecure: "true"
And then apply it by editing the apps/argocd/argocd/kustomization.yaml file:
apps/argocd/argocd/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: argocd
resources:
- base/applicationset.yaml
- https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/ha/install.yaml
patches:
- path: overlays/argocd-cmd-params-cm.yaml
After these changes are applied to the cluster, argocd-server needs to be restarted:
kubectl rollout restart deploy argocd-server -n argocd
With TLS disabled, the ingress needs to be configured. In the apps/argocd/argocd/base directory, create the IngressRoute file (adjust the domain name):
apps/argocd/argocd/base/argo-cd-ui-ingress.yaml
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
name: argocd-server
namespace: argocd
spec:
entryPoints:
- websecure
routes:
- kind: Rule
match: Host(`argocd.<your domain name>`)
priority: 10
services:
- name: argocd-server
port: 80
- kind: Rule
match: Host(`argocd.<your domain name>`) && Header(`Content-Type`, `application/grpc`)
priority: 11
services:
- name: argocd-server
port: 80
scheme: h2c
tls:
certResolver: default
And add it under the resources of the kustomization file:
apps/argocd/argocd/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: argocd
resources:
- base/applicationset.yaml
- base/argo-cd-ui-ingress.yaml
- https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/ha/install.yaml
patches:
- path: overlays/argocd-cmd-params-cm.yaml
Make sure argocd.<your domain name> resolves to the traefik public ip address. You can retrieve it from the Kubernetes cluster:
k get svc traefik -n traefik --output jsonpath='{.status.loadBalancer.ingress[0].ip}' ; echo
Argo CD is now accessible at argocd.<your domain name>
cert-manager
At this stage, Argo CD is using a self signed certificate. To have a certificate accepted by most browsers, cert-manager with Let’s Encrypt will be used. Again this is just one of many ways of doing it. There are multiple certificate providers and different tools to implement certificates.
To install cert-manager, create the new directories under apps for the namespace and application, and a kustomization file containing the following:
apps/cert-manager/cert-manager/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- https://github.com/cert-manager/cert-manager/releases/latest/download/cert-manager.yaml
With cert-manager installed, argocd needs to be configured to make use of it.
First create an issuer (adjust the email address):
apps/argocd/argocd/base/argo-cd-issuer.yaml
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
name: argo-cd-issuer
spec:
acme:
server: https://acme-v02.api.letsencrypt.org/directory
email: <your email address>
privateKeySecretRef:
name: letsencrypt
solvers:
- selector: {}
http01:
ingress:
class: traefik
and a certificate (adjust the domain name):
apps/argocd/argocd/base/argo-cd-certificate.yaml
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: argo-cd-cert
spec:
secretName: argocd-secret
issuerRef:
name: argo-cd-issuer
kind: Issuer
commonName: argocd.<your domain name>
dnsNames:
- argocd.<your domain name>
then add them under the resources of the kustomization file:
apps/argocd/argocd/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: argocd
resources:
- base/applicationset.yaml
- base/argo-cd-issuer.yaml
- base/argo-cd-certificate.yaml
- base/argo-cd-ui-ingress.yaml
- https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/ha/install.yaml
patches:
- path: overlays/argocd-cmd-params-cm.yaml
And finally, configure the IngressRoute to use the certificate-authority-data (adjust the domain name):
apps/argocd/argocd/base/argo-cd-ui-ingress.yaml
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
name: argocd-server
namespace: argocd
spec:
entryPoints:
- websecure
routes:
- kind: Rule
match: Host(`argocd.<your domain name>`)
priority: 10
services:
- name: argocd-server
port: 80
- kind: Rule
match: Host(`argocd.<your domain name>`) && Header(`Content-Type`, `application/grpc`)
priority: 11
services:
- name: argocd-server
port: 80
scheme: h2c
tls:
certResolver: default
secretName: argocd-secret
Now argocd.<your domain name> should have a valid Let’s Encrypt certificate.
Deploying an application
Now with Argo CD properly configured, it is possible to proceed to deploying an application. As a demo, a separate namespace – demo-ns – will be created with an application – demo-app – containing a deployment running nginx with an example webpage.
First, create a new directory under apps with the name demo-ns, for the namespace, and a new directory under demo-ns with the name demo-app, for the application, and add the following manifests:
apps/demo-ns/demo-app/app.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: nginx-configmap
data:
index.html: |
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Welcome</title>
<style>
body {
font-family: 'Segoe UI', Tahoma, Geneva, Verdana, sans-serif;
text-align: center;
margin: 0;
padding: 0;
background-color: #f5f5f5;
color: #333333;
height: 100vh;
display: flex;
flex-direction: column;
justify-content: center;
align-items: center;
}
h1 {
background-color: #ff8400;
color: white;
padding: 20px 0;
width: 100%;
text-align: center;
margin: 0;
}
#img {
max-width: 100%;
height: auto;
display: block;
margin: 0 auto;
box-shadow: 0 4px 8px rgba(0,0,0,0.3);
}
#pod-info
{
background-color: #5088c7;
color: #ffffff;
padding: 20px;
width: 100%;
text-align: center;
}
</style>
</head>
<body>
<h1>Welcome to Leaseweb Managed Kubernetes</h1>
<img id="img" src="https://cdn.leaseweb.com/images/leaseweb-layers.png" alt="Leaseweb Logo">
<div id="pod-info">
<h3>Pod Info</h3>
<div>{{ .Values.podInfo }}</div>
</div>
</body>
</html>
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
spec:
replicas: 1
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:latest
volumeMounts:
- name: nginx-config
mountPath: /usr/share/nginx/html/index.html
subPath: index.html
env:
- name: podInfo
valueFrom:
fieldRef:
fieldPath: metadata.name
volumes:
- name: nginx-config
configMap:
name: nginx-configmap
---
apiVersion: v1
kind: Service
metadata:
name: nginx-service
spec:
type: LoadBalancer
selector:
app: nginx
ports:
- protocol: TCP
port: 80
targetPort: 80
To get the public IP of the service, run the command:
k get svc demo-service -n demo-ns --output jsonpath='{.status.loadBalancer.ingress[0].ip}' ; echoAnd navigate to it in a web browser:
Conclusion
In this article, we showed how to go from a freshly provisioned Leaseweb kubernetes cluster to a cluster following the GitOps principles using Argo CD and deployed an example application.
Now it is possible to proceed with other configurations – for example RBAC or SSO user management. It is recommended that the same approach used in this article to configure Argo CD using patches and overlays be used. To learn more about other Argo CD configurations, visit the official Argo CD documentation – https://argo-cd.readthedocs.io/en/stable/operator-manual/
Another good section to read from the official documentation is the best practices – https://argo-cd.readthedocs.io/en/stable/user-guide/best_practices/
Throughout this article, we have examples of applications using kustomize, helm and plain manifests. Use whichever is more appropriate to your organization or mix and match according to your needs.
Caution
Throughout this article the versions used were always the latest. In production it is recommended to pin versions and upgrade them with care.