Architecture Notes

Image Pull Secrets

prokube comes with a small operator that copies all secrets from the ops namespace that begin with regcred- to all other namespaces, and patches the service accounts in those namespaces to use those image pull secrets. It also updates those secrets if they get changed in the ops namespace.

Limits

prokube applies some default limits to all workspaces (i.e. all namespaces which are attached to a "Profile"). By default, those limits cap each workspace at a maximum of 100 pods. This helps prevent overloading the cluster with thousands of pods from a single workspace. The limit can be adjusted per profile by an admin directly in the cluster (by patching the Profile CR), or for all newly created profiles by changing the patch paas/ops/pk-user-management-operator/profiles/patches/patch_profiles.j2 which gets automatically applied to all newly created profiles.

To limit resource consumption like CPU, Memory or persistent volumes sizes, you can add any ResourceQuota to the patch (for newly created profiles) or edit already existing profiles.

An example patch in patch_profiles.j2 with resource quotas that limits the number of pods to 100, the number of persistent volume claims to 2 and the total size of persistent volumes of the storage class openebs-hostpath to 10Gi would look like this:

{
    "apiVersion": "kubeflow.org/v1",
    "kind": "Profile",
    "metadata": {
        "name": "{{ profile_id }}"
    },
    "spec": {
        "resourceQuotaSpec": {
            "hard": {
                "count/pods": "100",
                "persistentvolumeclaims": "2",
                "openebs-hostpath.storageclass.storage.k8s.io/requests.storage": "10Gi"
            }
        }
    }
}