Architecture

Comprehensive overview of haive-mcp’s architecture enabling dynamic integration with 1900+ MCP servers.

System Overview

Haive-mcp is designed as a modular, extensible system that bridges the Haive agent framework with the Model Context Protocol ecosystem:

┌─────────────────────────────────────────────────────────────┐
│                     Haive Agent Framework                    │
├─────────────────────────────────────────────────────────────┤
│                        haive-mcp Layer                       │
│  ┌─────────────┐  ┌──────────────┐  ┌──────────────────┐  │
│  │   Agents    │  │   Discovery  │  │     Manager      │  │
│  │  • MCPAgent │  │  • Registry  │  │  • Lifecycle     │  │
│  │  • Enhanced │  │  • Search    │  │  • Installation  │  │
│  └─────────────┘  └──────────────┘  └──────────────────┘  │
├─────────────────────────────────────────────────────────────┤
│                    MCP Protocol Layer                        │
│  ┌──────────┐  ┌──────────┐  ┌──────────┐  ┌──────────┐  │
│  │  STDIO   │  │   HTTP   │  │   SSE    │  │  WebSocket│  │
│  └──────────┘  └──────────┘  └──────────┘  └──────────┘  │
├─────────────────────────────────────────────────────────────┤
│                   1900+ MCP Servers                          │
│  [Filesystem] [Database] [AI Tools] [Web] [Crypto] [More]   │
└─────────────────────────────────────────────────────────────┘

Core Components

Agent Layer

The agent layer provides MCP-enabled agents that extend Haive’s base agent capabilities:

  • MCPAgent: Base agent with MCP tool integration

  • EnhancedMCPAgent: Automatic discovery and installation from 1900+ servers

  • IntelligentMCPAgent: Task-aware tool selection

  • TransferableMCPAgent: Tool sharing between agents

Discovery System

The discovery system enables runtime tool discovery:

  • Registry: Maintains catalog of 1900+ available servers

  • Discovery Engine: Analyzes tasks to find relevant servers

  • Capability Matcher: Maps requirements to server capabilities

  • Ranking System: Prioritizes servers by relevance

Manager Component

The manager handles server lifecycle:

  • Installer: Supports NPM, Pip, Git, Docker installation

  • Process Manager: Starts, stops, monitors server processes

  • Configuration Manager: Handles server-specific settings

  • Health Monitor: Tracks server status and performance

Communication Patterns

Request-Response Flow

Agent → MCP Client → Transport → MCP Server → Tool
  ↑                                              ↓
  └──────────────── Response ←─────────────────┘

  1. Agent receives user request

  2. MCP client formats as MCP protocol message

  3. Transport layer handles communication

  4. MCP server executes tool

  5. Response flows back through layers

Async Operations

All MCP operations are asynchronous for optimal performance:

async def mcp_flow():
    # Discovery (async)
    servers = await discovery.find_servers_for_task(task)

    # Installation (async)
    await manager.install_servers(servers)

    # Execution (async)
    result = await agent.execute_tool(tool, params)

Transport Protocols

STDIO Transport

Used for command-line based servers:

  • Process-based communication

  • JSON-RPC over stdin/stdout

  • Suitable for Node.js, Python servers

  • Most common transport (60% of servers)

HTTP Transport

For web-based servers:

  • RESTful API communication

  • JSON payloads

  • Authentication support

  • Suitable for cloud services

SSE Transport

For streaming servers:

  • Server-Sent Events

  • Real-time updates

  • Unidirectional streaming

  • Used for monitoring, logs

WebSocket Transport

For bidirectional communication:

  • Full-duplex channels

  • Low latency

  • Real-time interaction

  • Advanced server features

Discovery Architecture

Three-Tier Discovery

Tier 1: Local Cache
├── Recently used servers
├── User preferences
└── Performance metrics

Tier 2: Registry Database
├── 1900+ server definitions
├── Capability mappings
└── Version information

Tier 3: Remote Discovery
├── GitHub repositories
├── NPM registry
└── Community servers

Discovery Algorithm

  1. Task Analysis: NLP processing of user request

  2. Capability Extraction: Identify required capabilities

  3. Server Matching: Find servers with capabilities

  4. Ranking: Score by relevance, popularity, performance

  5. Selection: Choose optimal server set

Installation Architecture

Multi-Stage Installation

Stage 1: Dependency Resolution
├── Check package manager
├── Resolve dependencies
└── Verify compatibility

Stage 2: Download & Install
├── Parallel downloads
├── Package installation
└── Post-install scripts

Stage 3: Verification
├── Health check
├── Tool discovery
└── Performance test

Installer Types

  • NPM Installer: Node.js packages via npx

  • Pip Installer: Python packages via pip

  • Git Installer: Direct from repositories

  • Docker Installer: Containerized servers

  • Binary Installer: Pre-compiled executables

Caching Architecture

Multi-Level Cache

L1: Memory Cache (< 1ms)
├── Active connections
├── Recent responses
└── Hot configuration

L2: Local Disk (< 10ms)
├── Server metadata
├── Tool definitions
└── Response cache

L3: Redis Cache (< 50ms)
├── Shared state
├── Distributed cache
└── Session data

Cache Strategies

  • Write-Through: Updates cache and backend

  • Write-Behind: Async cache updates

  • Cache-Aside: Lazy loading

  • Refresh-Ahead: Predictive refresh

Security Architecture

Defense in Depth

Layer 1: Authentication
├── API keys
├── OAuth 2.0
└── JWT tokens

Layer 2: Authorization
├── Role-based access
├── Tool permissions
└── Resource limits

Layer 3: Isolation
├── Process sandboxing
├── Network segmentation
└── Resource quotas

Layer 4: Monitoring
├── Audit logging
├── Anomaly detection
└── Rate limiting

Scaling Architecture

Horizontal Scaling

Load Balancer
├── Agent Pool (N instances)
├── MCP Server Pool (M instances)
└── Cache Cluster (K nodes)

Vertical Scaling

  • Resource allocation per server

  • Connection pool sizing

  • Cache size management

  • Process priority tuning

Performance Optimizations

Connection Pooling

Maintains persistent connections to frequently used servers:

  • Reduces connection overhead

  • Improves response times

  • Handles connection lifecycle

  • Automatic reconnection

Parallel Processing

Executes independent operations concurrently:

  • Parallel server installation

  • Concurrent tool execution

  • Batch operation support

  • Async/await throughout

Predictive Loading

Anticipates user needs:

  • Pre-loads likely servers

  • Warms cache with common operations

  • Background initialization

  • Speculative execution

Extensibility Points

Custom Agents

Extend base agents for domain-specific needs:

class CustomMCPAgent(MCPAgent):
    """Domain-specific MCP agent."""
    pass

Custom Servers

Create proprietary MCP servers:

class CustomServer(MCPServer):
    """Custom MCP server implementation."""
    pass

Custom Transports

Implement new transport protocols:

class CustomTransport(Transport):
    """Custom transport implementation."""
    pass

Next Steps