_The intelligent (edge and LLM) proxy server for agentic applications._ Move faster by letting Arch handle the pesky heavy lifting in building agents: fast input clarification, agent routing, seamless integration of prompts with tools for common tasks, and unified access and observability of LLMs. Past the thrill of an AI demo, have you found yourself hitting these walls? You know, the all too familiar ones: You go from one BIG prompt to specialized prompts, but get stuck building routing and handoff code? You want use new LLMs, but struggle to quickly and safely add LLMs without writing integration code? Youre bogged down with prompt engineering just to clarify user intent and validate inputs effectively? Youre wasting cycles choosing and integrating code for observability instead of it happening transparently? And you think to youself, cant I move faster by focusing on higher-level objectives in a language/framework agnostic way? Well, you can! Arch Gateway was built by the contributors of Envoy Proxy with the belief that: Prompts are nuanced and opaque user requests, which require the same capabilities as traditional HTTP requests including secure handling, intelligent routing, robust observability, and integration with backend (API) systems to improve speed and accuracy for common agentic scenarios – all outside core application logic. Core Features: - 🚦 Routing. Engineered with purpose-built LLMs for fast (<100ms) agent routing and hand-off scenarios - ⚡ Tools Use: For common agentic scenarios let Arch instantly clarify and convert prompts to tools/API calls - ⛨ Guardrails: Centrally configure and prevent harmful outcomes and ensure safe user interactions - 🔗 Access to LLMs: Centralize access and traffic to LLMs with smart retries for continuous availability - 🕵 Observability: W3C compatible request tracing and LLM metrics that instantly plugin with popular tools - 🧱 Built on Envoy: Arch runs alongside app servers as a containerized process, and builds on top of Envoys proven HTTP management and scalability features to handle ingress and egress traffic related to prompts and LLMs. Jump to our docs to learn how you can use Arch to improve the speed, security and personalization of your agentic applications.
