Canonical: https://uhl.cloud/en/resources/self-hosted-ai-stack/
Language: en
Format: public page content, generated from HTML

# The Self-Hosted AI Stack

Private inference, agent memory and assistants on your own Kubernetes. Data sovereignty by architecture, not by policy - the stack we run ourselves.

BlueprintUpdated 2026-10-27

[Download the PDF](https://uhl.cloud/assets/resources/self-hosted-ai-stack-en.pdf?v=0084519c7a60)

## Why self-host AI at all

Managed AI APIs are excellent - until the data question arrives. Enquiries containing customer details, internal documents feeding an assistant, agent memory accumulating business knowledge: each leaves your perimeter when the provider is external. Self-hosting does not reject outside AI; it draws a line where sensitive context never crosses.

## The reference architecture we operate

- Kubernetes on Talos Linux - immutable, minimal attack surface, managed through GitOps so the cluster state is reviewable in Git

- Model serving for private inference - LLMs and embeddings run inside the cluster for tasks that must not leave it

- Cognee for agent memory - a knowledge-graph layer that gives agents persistent, inspectable context instead of opaque prompt stuffing

- Typebot for guided assistants - conversational intake and internal helpers, self-hosted with restricted allowed origins

- MinIO for object storage - documents, artifacts and model-adjacent data under the same sovereignty rules

- Keycloak for identity - assistants and tools authenticate against the same IdP as the team

This is not a diagram we sell. It is the stack this company's own assistants, memory and pipelines run on today.

## Sizing honestly

A CPU-only cluster handles embeddings, retrieval, orchestration and small models well. GPU nodes matter when you want fast private inference on larger models - they add cost and scheduling complexity, so the honest answer is: start with the assistants and memory layer, measure the real demand, then decide whether dedicated inference hardware pays for itself. For many organisations the pattern is hybrid - public APIs for low-sensitivity tasks, private inference where context is sensitive.

## What operating it actually means

- Versioned deployments through GitOps - every model, assistant and memory service is a reviewed change, not a console click

- Backup and restore procedures that cover agent memory and object data, with documented restore tests

- Monitoring that treats AI services like any other workload - availability, latency, resource use

- Access governed through the company IdP - assistants are not a shadow identity system

- Update paths tested on a copy before production - a broken memory layer is an outage, not an experiment

## Chat with UHL

Start a guided chat. The assistant asks a few questions and saves your enquiry for our team to review.

## Continue exploring

- [Self-hosted platforms](https://uhl.cloud/en/self-hosted/)

- [AI & automation](https://uhl.cloud/en/ai-automation/)

- [Kubernetes & Talos](https://uhl.cloud/en/kubernetes-talos/)

## Let’s discuss your project.

[Talk to UHL](https://uhl.cloud/en/contact/)