Skip to content

prokube⚓︎

Welcome to prokube.

GenAI & LLM Ready⚓︎

prokube is built for the age of Generative AI. Deploy and scale Large Language Models, build agentic applications, and run GPU-accelerated inference workloads with enterprise-grade infrastructure.

  • LLM Model Serving: Deploy LLMs like Llama, Mistral, Qwen, and custom models as scalable REST endpoints with KServe and vLLM
  • GPU Acceleration: NVIDIA GPU support with timeslicing and Multi-Instance GPU (MIG) for efficient resource sharing
  • Fast Model Loading: Local model caching dramatically reduces LLM startup times for production workloads
  • Agentic Workflows: Build and orchestrate AI agents using Kubeflow Pipelines with full observability
  • Experiment Tracking: Track prompts, model versions, and evaluation metrics with MLflow
  • S3 Model Storage: Store and version large model artifacts with MinIO's S3-compatible object storage

Whether you're fine-tuning foundation models, deploying inference endpoints, or building multi-agent systems, prokube provides the complete platform for your GenAI workloads.

About this Documentation⚓︎

This documentation is divided into four sections: user, administrator, developer, and examples. The user section shows you how to use the platform and its services. The administrator section covers platform management and operations. The developer section provides technical details for integrating applications. The examples section contains practical tutorials and cookbooks.

prokube uses many best-in-class open-source tools and applications. You can find information for these on their respective websites. These resources are usually complete, well written and up-to-date. Therefore, we usually don't write our own material for those tools and instead recommend referring to their official resources for detailed information.

Platform Overview⚓︎

prokube is a data science platform integrating Kubeflow, MLflow, and many more open-source tools. prokube provides a set of tools and services to support data science teams in their daily work. It is designed for collaborative work on the whole lifecycle of data-based products.

Notebook Servers⚓︎

prokube provides notebook servers for data scientists to work on their projects. These servers can be fitted to the users' needs in various ways and provide the basic infrastructure for data science work.

Data Storage⚓︎

prokube provides the underlying storage engines and deploys various storage services on top of these. These services include:

  • MinIO, an S3 compatible object storage server
  • PostgreSQL and TimescaleDB databases
  • Kubernetes Persistent Volumes that can be used in various ways

Data scientists can use these storage options to store their data and access it from all prokube services. Many further storage services can be added, as prokube runs on a Kubernetes infrastructure.

Model Training⚓︎

prokube provides all necessary tools to train and optimize machine learning models. Katib automates hyperparameter tuning and neural architecture search, helping you find optimal model configurations without manual trial and error. The NVIDIA GPU Operator is used to provide GPU access and share resources for training deep learning models.

Model Deployment⚓︎

prokube also provides tools to deploy machine learning models. You can deploy models with KServe as REST endpoints while making use of autoscaling, authentication, monitoring and many other features.

Kubeflow Pipelines⚓︎

prokube provides the Kubeflow Pipelines engine for data scientists to create and run machine learning pipelines. These pipelines can be used to automate the training, optimization and deployment of machine learning models.

User Management⚓︎

prokube provides user management tools for administrators to manage users and profiles. Users can be added, removed, and assigned to profiles. Profiles can be created and deleted and are used to manage access to resources and data. Various identity providers can be used to authenticate users.

Kubernetes⚓︎

prokube runs on Kubernetes. Kubernetes is an open-source container-orchestration system for automating computer application deployment, scaling, and management. prokube provides users low-level access to their namespaces on the cluster to deploy their own services and applications. Administrators can manage the whole cluster on this low level and have full overview of and access to all components of the platform.

Deployment Options⚓︎

prokube can be deployed on any Kubernetes cluster that meets the requirements. The choice of Kubernetes distribution depends on your infrastructure, team expertise, and scale requirements.

For cloud deployments, we recommend using managed Kubernetes services:

  • Google Kubernetes Engine (GKE)
  • Amazon Elastic Kubernetes Service (EKS)
  • Azure Kubernetes Service (AKS)

Managed services handle control plane operations, upgrades, and provide cloud-native integrations. This is the best choice for organizations already using cloud infrastructure and wanting minimal Kubernetes operational overhead at scale.

For self-managed deployments with up to approximately 10 nodes, we recommend MicroK8s, a lightweight Kubernetes distribution from Canonical. MicroK8s is well-suited for:

  • Small to medium teams without dedicated Kubernetes operations staff
  • Development and staging environments
  • On-premise deployments where simplicity is valued
  • Quick setups with minimal operational overhead

MicroK8s provides production-grade Kubernetes with built-in high availability, automatic updates, and a straightforward installation process.

Other Kubernetes Distributions⚓︎

For larger self-managed deployments (10+ nodes) or organizations with specific requirements, prokube also works with:

  • kubeadm-based clusters for full control over cluster configuration
  • OpenShift for enterprise environments with existing Red Hat infrastructure
  • Rancher (RKE/RKE2) for multi-cluster management and hybrid environments
  • k3s for lightweight edge deployments

These options typically require more Kubernetes expertise but offer greater flexibility for complex environments.