🛠️ OSS, K8s-native AI platform for distributed multi-model inference
Summary
Shaide is an open source, Kubernetes-native AI platform designed for distributed multi-model inference. It enables running multiple LLMs on your own infrastructure with independent scaling, KV-cache-aware scheduling, and an OpenAI-compatible API.
Why it’s interesting
It provides a fully reproducible, air-gapped infrastructure-as-code solution for managing multiple large language models on Kubernetes.
Target user
Engineers and organizations wanting to run multiple LLMs on their own infrastructure.
Source metrics: Points 3 · Comments 0
HN discussion · Project
Source: #HackerNews / Show HN
Summary
Shaide is an open source, Kubernetes-native AI platform designed for distributed multi-model inference. It enables running multiple LLMs on your own infrastructure with independent scaling, KV-cache-aware scheduling, and an OpenAI-compatible API.
Why it’s interesting
It provides a fully reproducible, air-gapped infrastructure-as-code solution for managing multiple large language models on Kubernetes.
Target user
Engineers and organizations wanting to run multiple LLMs on their own infrastructure.
Source metrics: Points 3 · Comments 0
HN discussion · Project
Source: #HackerNews / Show HN