Upgrading prokube

The top-level UPDATING.md file contains specific instructions for upgrading from one version of prokube to another.

The general procedure for updating prokube deployments is as follows:

1. Rebase your gitops branch (e.g. gitops/dev), the target revision for ArgoCD to deploy to the latest release branch.
2. Push your changes to the appropriate branch (a --force-with-lease is usually needed).
3. If no other specific upgrade instructions are given, log into ArgoCD at (https://yourdomain.tld/argocd/) and refresh and sync all apps.

GitLab

Updating GitLab is supported by prokube Ansible scripts. More info in infra/README.md

Procedure:

1. Rebase your infra/xxx branch in infra to new master/feature branch.
2. Try updating GitLab. From your laptop cd to ansible/gitlab_playbook and run:

   ansible-playbook --ssh-common-args="-i ~/.ssh/id_ed25519_xxx" --extra-vars="ansible_user='ansible' gitlab_runner_token='<token-used-to-register-gitlab-runner>'" -i ../inventory deploy.yaml --tags update_gitlab
   

3. If you encounter an error like this:

   gitlab preinstall: It seems you are upgrading from 16.5 to 16.8.
   gitlab preinstall: It is required to upgrade to the latest 16.7.x version first before proceeding.
   

   then you will have to update to an intermediate version first. You can do that by adjusting gitlab_version, gitlab_runner_version and gitlab_agent_version in ansible/gitlab_playbook/group_vars/all.yaml and ansible/microk8s_playbook/group_vars/all.yaml.

k8s

1. Commit changes from your rebasing above and push to GitLab infra/xxx to deployment's GitLab. This will trigger redeployment of k8s.
2. Drain nodes as needed:

   kubectl drain --ignore-daemonsets --delete-emptydir-data <node-name>
   kubectl uncordon <node-name>
   

3. Restart any pods that are stuck in a pending state because their PVC is on the wrong node.