Canal / AI Engine

Unified
AI Runtime

Provider-agnostic inference gateway built on Rust and Tokio. Abstracts 6+ LLM providers behind a single OpenAI-compatible API. Deterministic agent orchestration via StateGraph. Native Model Context Protocol integration.

Runtime Rust + Tokio

API OpenAI-Compatible

Providers 6+

License Open Source

System Architecture

Six layers. One binary.

L06

API Surface

Axum HTTP server. REST endpoints, SSE streaming, SQLx persistence. Bearer token authentication. Rate limiting and request validation.

L05

Orchestration

StateGraph execution engine. Directed graph topology for multi-agent coordination. Swarm (parallel dispatch), Plan-Execute (sequential decomposition), Expert (specialist routing).

L04

MCP Gateway

Native Model Context Protocol integration. Tool discovery, schema aggregation, server-side execution. Both client and server mode operation.

L03

Tool Execution

Sandboxed runtime with filesystem access controls. Plugin system with connector framework. Permission boundaries per tool invocation.

L02

Agent Core

Agentic loop with composable skill registry. Session persistence, context windowing, and constraint profiles. Budget enforcement and stall detection.

L01

LLM Routing

Provider-agnostic request routing with automatic failover, cost-aware model selection, and load balancing. Semantic response caching.

Integration

Drop-in API compatibility

Canal implements the OpenAI chat completions API. Redirect your existing client library to Canal's endpoint — multi-provider routing, response caching, and execution tracing are handled transparently. No application code changes required.

    example.py 
from openai import OpenAI

# Redirect to Canal gateway
client = OpenAI(
    base_url="http://localhost:3000/v1",
    api_key="your-canal-key"
)

response = client.chat.completions.create(
    model="anthropic/claude-sonnet-4-20250514",
    messages=[{"role": "user", "content": "Analyze this codebase"}],
    stream=True
)

for chunk in response:
    print(chunk.choices[0].delta.content, end="")

Anthropic Claude OpenAI GPT Google Gemini Alibaba Qwen OpenRouter Ollama (Local)

Core Systems

Engineered for production inference

Provider-Agnostic Routing

Unified interface across Anthropic, OpenAI, Google, Alibaba Qwen, OpenRouter, and local Ollama instances. Automatic failover with configurable fallback chains. Cost-aware model selection based on task classification.

StateGraph Orchestration

Deterministic multi-agent execution via directed acyclic graphs. Three coordination topologies: Swarm (parallel dispatch with handoff), Plan-Execute (decompose-then-execute), Expert (specialist routing with judge aggregation).

Composable Skill System

Agent capabilities defined as versioned, sandboxed skill units. Independent permission boundaries per skill. Stack and compose across agents. Registry-managed with runtime injection.

Constraint Profiles

Three-layer prompt constraint model: hard constraints (output format, security boundaries), soft constraints (role anchoring, reasoning mode), user preferences (custom instructions). Runtime-editable without redeployment.

UnifiedAI Runtime