🛠️ I stopped using an LLM gateway and put rate-limits/fallback in-process
Summary
VernLLM is an in-process library for TypeScript and Node that provides rate-limiting, multi-provider fallback, and circuit breaking for LLM calls without requiring a separate network gateway. It supports OpenAI-compatible APIs, Anthropic, Gemini, and Bedrock.
Why it’s interesting
It eliminates the extra network hop and security concerns of traditional LLM gateways by handling resilience features directly in-process.
Target user
TypeScript and Node developers working with LLM APIs
Source metrics: Points 3 · Comments 1
HN discussion · Project
Source: #HackerNews / Show HN
Summary
VernLLM is an in-process library for TypeScript and Node that provides rate-limiting, multi-provider fallback, and circuit breaking for LLM calls without requiring a separate network gateway. It supports OpenAI-compatible APIs, Anthropic, Gemini, and Bedrock.
Why it’s interesting
It eliminates the extra network hop and security concerns of traditional LLM gateways by handling resilience features directly in-process.
Target user
TypeScript and Node developers working with LLM APIs
Source metrics: Points 3 · Comments 1
HN discussion · Project
Source: #HackerNews / Show HN