Decision Layer
The Decision Layer handles adaptive routing, multi-LLM orchestration, and cost optimization.
decision.route()
Automatically select the best LLM for a given task based on requirements and constraints.
Signature
typescript
route(params: RouteParams): Promise<RouteResponse>Parameters
| Name | Type | Required | Description |
|---|---|---|---|
| task | string | Yes | Task description |
| priority | string | No | “cost” | “quality” | “speed” |
| constraints | object | No | Budget, latency limits |
Returns
typescript
{
model: string; // Selected model
provider: string; // LLM provider
estimatedCost: number; // Estimated cost in USD
estimatedLatency: number;// Expected latency in ms
reasoning: string; // Why this model was chosen
}Example
typescript
const route = await client.decision.route({
task: "Analyze legal document and extract key clauses",
priority: "quality",
constraints: {
maxCost: 0.50,
maxLatency: 5000
}
});
console.log('Selected:', route.model);
console.log('Provider:', route.provider);
console.log('Reasoning:', route.reasoning);
// Use the selected model
const result = await client.cognition.reason({
query: "Analyze document...",
model: route.model
});decision.coordinate()
Orchestrate multiple agents to work together on complex tasks.
Example
typescript
const result = await client.decision.coordinate({
task: "Research and write comprehensive report",
agents: [
{ role: "researcher", model: "gpt-4" },
{ role: "analyzer", model: "claude-3" },
{ role: "writer", model: "gpt-4" }
],
workflow: "sequential"
});
console.log('Final Output:', result.output);
console.log('Agent Contributions:', result.agentOutputs);
console.log('Total Cost:', result.totalCost);decision.optimize()
Analyze usage patterns and get recommendations for cost optimization.
Example
typescript
const optimization = await client.decision.optimize({
timeframe: "last_30_days",
goal: "reduce_cost"
});
console.log('Current Spend:', optimization.currentSpend);
console.log('Potential Savings:', optimization.potentialSavings);
console.log('Recommendations:', optimization.recommendations);
// Example recommendations:
// - Switch 60% of queries to GPT-3.5 (save $200/mo)
// - Enable caching for repeated queries (save $150/mo)
// - Use Claude for long context tasks (save $100/mo)Routing Strategies
Cost-Optimized
Minimize costs while maintaining acceptable quality. Uses cheaper models when possible.
typescript
await client.decision.route({
task: "Simple question answering",
priority: "cost"
});Quality-Optimized
Maximize output quality. Uses most capable models regardless of cost.
typescript
await client.decision.route({
task: "Complex legal analysis",
priority: "quality"
});Speed-Optimized
Minimize latency. Uses fastest available models.
typescript
await client.decision.route({
task: "Real-time chat response",
priority: "speed"
});Best Practices
- • Use adaptive routing for cost optimization
- • Set appropriate constraints for your use case
- • Monitor routing decisions with analytics
- • Use coordination for complex multi-step tasks
- • Regularly review optimization recommendations
- • Test different routing strategies in development