diff --git a/README.md b/README.md index 3cd9c9f..40cb924 100644 --- a/README.md +++ b/README.md @@ -2,17 +2,19 @@ A **Model Context Protocol (MCP) server** that enables multiple AI agents to coordinate their work seamlessly across codebases without conflicts. Built with Elixir for reliability and fault tolerance. -## ๐ŸŽฏ What is Agent Coordinator? +## What is Agent Coordinator? Agent Coordinator is a **MCP proxy server** that enables multiple AI agents to collaborate seamlessly without conflicts. As shown in the architecture diagram above, it acts as a **single MCP interface** that proxies ALL tool calls through itself, ensuring every agent maintains full project awareness while the coordinator tracks real-time agent presence. **The coordinator operates as a transparent proxy layer:** + - **Single Interface**: All agents connect to one MCP server (the coordinator) - **Proxy Architecture**: Every tool call flows through the coordinator to external MCP servers - **Presence Tracking**: Each proxied tool call updates agent heartbeat and task status - **Project Awareness**: All agents see the same unified view of project state through the proxy **This proxy design orchestrates four core components:** + - **Task Registry**: Intelligent task queuing, agent matching, and automatic progress tracking - **Agent Manager**: Agent registration, heartbeat monitoring, and capability-based assignment - **Codebase Registry**: Cross-repository coordination, dependency management, and workspace organization @@ -22,28 +24,28 @@ Instead of agents conflicting over files or duplicating work, they connect throu **Key Features:** -- **๐Ÿ”„ MCP Proxy Architecture**: Single server that proxies ALL external MCP servers for unified agent access -- **๐Ÿ‘๏ธ Real-Time Activity Tracking**: Live visibility into agent activities: "Reading file.ex", "Editing main.py", "Sequential thinking" -- **๐Ÿ“ก Real-Time Presence Tracking**: Every tool call updates agent status and project awareness -- **๐Ÿ“ File-Level Coordination**: Track exactly which files each agent is working on to prevent conflicts -- **๐Ÿ“œ Activity History**: Rolling log of recent agent actions with timestamps and file details -- **๐Ÿค– Multi-Agent Coordination**: Register multiple AI agents (GitHub Copilot, Claude, etc.) with different capabilities -- **๐ŸŽฏ Transparent Tool Routing**: Automatically routes tool calls to appropriate external servers while tracking usage -- **๐Ÿ“ Automatic Task Creation**: Every tool usage becomes a tracked task with agent coordination context -- **โšก Full Project Awareness**: All agents see unified project state through the proxy layer -- **๐Ÿ“ก External Server Management**: Automatically starts, monitors, and manages MCP servers defined in `mcp_servers.json` -- **๐Ÿ› ๏ธ Universal Tool Registry**: Proxies tools from all external servers while adding native coordination tools -- **๐Ÿ”Œ Dynamic Tool Discovery**: Automatically discovers new tools when external servers start/restart -- **๐ŸŽฎ Cross-Codebase Support**: Coordinate work across multiple repositories and projects -- **๐Ÿ”Œ MCP Standard Compliance**: Works with any MCP-compatible AI agent or tool +- **MCP Proxy Architecture**: Single server that proxies ALL external MCP servers for unified agent access +- **Real-Time Activity Tracking**: Live visibility into agent activities: "Reading file.ex", "Editing main.py", "Sequential thinking" +- **Real-Time Presence Tracking**: Every tool call updates agent status and project awareness +- **File-Level Coordination**: Track exactly which files each agent is working on to prevent conflicts +- **Activity History**: Rolling log of recent agent actions with timestamps and file details +- **Multi-Agent Coordination**: Register multiple AI agents (GitHub Copilot, Claude, etc.) with different capabilities +- **Transparent Tool Routing**: Automatically routes tool calls to appropriate external servers while tracking usage +- **Automatic Task Creation**: Every tool usage becomes a tracked task with agent coordination context +- **Full Project Awareness**: All agents see unified project state through the proxy layer +- **External Server Management**: Automatically starts, monitors, and manages MCP servers defined in `mcp_servers.json` +- **Universal Tool Registry**: Proxies tools from all external servers while adding native coordination tools +- **Dynamic Tool Discovery**: Automatically discovers new tools when external servers start/restart +- **Cross-Codebase Support**: Coordinate work across multiple repositories and projects +- **MCP Standard Compliance**: Works with any MCP-compatible AI agent or tool -## ๐Ÿš€ How It Works +## Overview ![Agent Coordinator Architecture](docs/architecture-diagram.svg) **The Agent Coordinator acts as a transparent MCP proxy server** that routes ALL tool calls through itself to maintain agent presence and provide full project awareness. Every external MCP server is proxied through the coordinator, ensuring unified agent coordination. -### ๐Ÿ”„ Proxy Architecture Flow +### Proxy Architecture Flow 1. **Agent Registration**: Multiple AI agents (Purple Zebra, Yellow Elephant, etc.) register with their capabilities 2. **External Server Discovery**: Coordinator automatically starts and discovers tools from external MCP servers @@ -52,11 +54,11 @@ Instead of agents conflicting over files or duplicating work, they connect throu 5. **Presence Tracking**: Every proxied tool call updates agent heartbeat and task status 6. **Project Awareness**: All agents maintain unified project state through the proxy layer -## ๐Ÿ‘๏ธ Real-Time Activity Tracking - FANTASTIC Feature! ๐ŸŽ‰ +## Real-Time Activity Tracking - FANTASTIC Feature! **See exactly what every agent is doing in real-time!** The coordinator intelligently tracks and displays agent activities as they happen: -### ๐ŸŽฏ Live Activity Examples +### Live Activity Examples ```json { @@ -172,12 +174,12 @@ The coordinator automatically manages external MCP servers based on configuratio **Server Lifecycle Management:** -1. **๐Ÿš€ Startup**: Reads config and spawns each external server process -2. **๐Ÿ” Discovery**: Sends MCP `initialize` and `tools/list` requests to discover available tools -3. **๐Ÿ“‹ Registration**: Adds discovered tools to the unified tool registry -4. **๐Ÿ’“ Monitoring**: Continuously monitors server health and heartbeat -5. **๐Ÿ”„ Auto-Restart**: Automatically restarts failed servers (if configured) -6. **๐Ÿ›ก๏ธ Cleanup**: Properly terminates processes and cleans up resources on shutdown +1. **Startup**: Reads config and spawns each external server process +2. **Discovery**: Sends MCP `initialize` and `tools/list` requests to discover available tools +3. **Registration**: Adds discovered tools to the unified tool registry +4. **Monitoring**: Continuously monitors server health and heartbeat +5. **Auto-Restart**: Automatically restarts failed servers (if configured) +6. **Cleanup**: Properly terminates processes and cleans up resources on shutdown ### ๐Ÿ› ๏ธ Unified Tool Registry @@ -203,14 +205,14 @@ The coordinator combines tools from multiple sources into a single, coherent int **Dynamic Discovery Process:** -1. **๐Ÿš€ Startup**: Agent Coordinator starts external MCP server process -2. **๐Ÿค Initialize**: Sends MCP `initialize` request โ†’ Server responds with capabilities -3. **๐Ÿ“‹ Discovery**: Sends `tools/list` request โ†’ Server returns available tools -4. **โœ… Registration**: Adds discovered tools to unified tool registry +1. **Startup**: Agent Coordinator starts external MCP server process +2. **Initialize**: Sends MCP `initialize` request โ†’ Server responds with capabilities +3. **Discovery**: Sends `tools/list` request โ†’ Server returns available tools +4. **Registration**: Adds discovered tools to unified tool registry This process repeats automatically when servers restart or new servers are added. -### ๐ŸŽฏ Intelligent Tool Routing +### Intelligent Tool Routing When an AI agent calls a tool, the coordinator intelligently routes the request: @@ -235,7 +237,7 @@ Agent calls "read_file" โ†’ Coordinator routes to filesystem server โ†’ Updates agent task โ†’ Sends heartbeat โ†’ Returns file content ``` -## ๐Ÿ› ๏ธ Prerequisites +## Prerequisites Choose one of these installation methods: @@ -345,339 +347,5 @@ Add this to your VS Code `settings.json`: } } } -} - } } ``` - -### 4. Test It Works - -#### Docker Testing - -```bash -# Test with Docker -docker-compose exec agent-coordinator /app/bin/agent_coordinator ping - -# Run example (if available in container) -docker-compose exec agent-coordinator mix run examples/full_workflow_demo.exs - -# View logs -docker-compose logs -f agent-coordinator -``` - -#### Manual Testing - -```bash -# Run the demo to see it in action -mix run examples/full_workflow_demo.exs -``` - -## ๐Ÿณ Docker Usage Guide - -### Available Docker Commands - -#### Basic Operations - -```bash -# Build the image -docker build -t agent-coordinator . - -# Run standalone container -docker run -d --name agent-coordinator -p 4000:4000 agent-coordinator - -# Run with custom config -docker run -d \ - -v ./mcp_servers.json:/app/mcp_servers.json:ro \ - -p 4000:4000 \ - agent-coordinator -``` - -#### Docker Compose Operations - -```bash -# Start full stack -docker-compose up -d - -# Start only agent coordinator -docker-compose up -d agent-coordinator - -# View logs -docker-compose logs -f agent-coordinator - -# Restart after config changes -docker-compose restart agent-coordinator - -# Stop everything -docker-compose down - -# Remove volumes (reset data) -docker-compose down -v -``` - -#### Development with Docker - -```bash -# Start in development mode -docker-compose -f docker-compose.yml -f docker-compose.dev.yml up - -# Interactive shell for debugging -docker-compose exec agent-coordinator bash - -# Run tests in container -docker-compose exec agent-coordinator mix test - -# Watch logs during development -docker-compose logs -f -``` - -### Environment Variables - -Configure the container using environment variables: - -```bash -# docker-compose.override.yml example -version: '3.8' -services: - agent-coordinator: - environment: - - MIX_ENV=prod - - NATS_HOST=nats - - NATS_PORT=4222 - - LOG_LEVEL=info -``` - -### Custom Configuration - -#### External MCP Servers - -Mount your own `mcp_servers.json`: - -```bash -docker run -d \ - -v ./my-mcp-config.json:/app/mcp_servers.json:ro \ - agent-coordinator -``` - -#### Persistent Data - -```bash -docker run -d \ - -v agent_data:/app/data \ - -v nats_data:/data \ - agent-coordinator -``` - -### Monitoring & Health Checks - -#### Container Health - -```bash -# Check container health -docker-compose ps - -# Health check details -docker inspect --format='{{json .State.Health}}' agent-coordinator - -# Manual health check -docker-compose exec agent-coordinator /app/bin/agent_coordinator ping -``` - -#### NATS Monitoring - -Access NATS monitoring dashboard: -```bash -# Start with monitoring profile -docker-compose --profile monitoring up -d - -# Access dashboard at http://localhost:8080 -open http://localhost:8080 -``` - -### Troubleshooting - -#### Common Issues - -```bash -# Check container logs -docker-compose logs agent-coordinator - -# Check NATS connectivity -docker-compose exec agent-coordinator nc -z nats 4222 - -# Restart stuck container -docker-compose restart agent-coordinator - -# Reset everything -docker-compose down -v && docker-compose up -d -``` - -#### Performance Tuning - -```bash -# Allocate more memory -docker-compose up -d --scale agent-coordinator=1 \ - --memory=1g --cpus="2.0" -``` - -## ๐ŸŽฎ How to Use - -Once your AI agents are connected via MCP, they can: - -### Register as an Agent - -```bash -# An agent identifies itself with capabilities -register_agent("GitHub Copilot", ["coding", "testing"], codebase_id: "my-project") -``` - -### Create Tasks - -```bash -# Tasks are created with requirements -create_task("Fix login bug", "Authentication fails on mobile", - priority: "high", - required_capabilities: ["coding", "debugging"] -) -``` - -### Coordinate Automatically - -The coordinator automatically: - -- **Matches** tasks to agents based on capabilities -- **Queues** tasks when no suitable agents are available -- **Tracks** agent heartbeats to ensure they're still working -- **Handles** cross-codebase tasks that span multiple repositories - -### Available MCP Tools - -All MCP-compatible AI agents get these tools automatically: - -| Tool | Purpose | -|------|---------| -| `register_agent` | Register an agent with capabilities | -| `create_task` | Create a new task with requirements | -| `get_next_task` | Get the next task assigned to an agent | -| `complete_task` | Mark current task as completed | -| `get_task_board` | View all agents and their status | -| `heartbeat` | Send agent heartbeat to stay active | -| `register_codebase` | Register a new codebase/repository | -| `create_cross_codebase_task` | Create tasks spanning multiple repos | - -## ๐Ÿงช Development & Testing - -### Running Tests - -```bash -# Run all tests -mix test - -# Run with coverage -mix test --cover - -# Try the examples -mix run examples/full_workflow_demo.exs -mix run examples/auto_heartbeat_demo.exs -``` - -### Code Quality - -```bash -# Format code -mix format - -# Run static analysis -mix credo - -# Type checking -mix dialyzer -``` - -## ๐Ÿ“ Project Structure - -```text -agent_coordinator/ -โ”œโ”€โ”€ lib/ -โ”‚ โ”œโ”€โ”€ agent_coordinator.ex # Main module -โ”‚ โ””โ”€โ”€ agent_coordinator/ -โ”‚ โ”œโ”€โ”€ mcp_server.ex # MCP protocol implementation -โ”‚ โ”œโ”€โ”€ task_registry.ex # Task management -โ”‚ โ”œโ”€โ”€ agent.ex # Agent management -โ”‚ โ”œโ”€โ”€ codebase_registry.ex # Multi-repository support -โ”‚ โ””โ”€โ”€ application.ex # Application supervisor -โ”œโ”€โ”€ examples/ # Working examples -โ”œโ”€โ”€ test/ # Test suite -โ”œโ”€โ”€ scripts/ # Helper scripts -โ””โ”€โ”€ docs/ # Technical documentation - โ”œโ”€โ”€ README.md # Documentation index - โ”œโ”€โ”€ AUTO_HEARTBEAT.md # Unified MCP server details - โ”œโ”€โ”€ VSCODE_TOOL_INTEGRATION.md # VS Code integration - โ””โ”€โ”€ LANGUAGE_IMPLEMENTATIONS.md # Alternative language guides -``` - -## ๐Ÿค” Why This Design? - -**The Problem**: Multiple AI agents working on the same codebase step on each other, duplicate work, or create conflicts. - -**The Solution**: A coordination layer that: - -- Lets agents register their capabilities -- Intelligently distributes tasks -- Tracks progress and prevents conflicts -- Scales across multiple repositories - -**Why Elixir?**: Built-in concurrency, fault tolerance, and excellent for coordination systems. - -## ๐Ÿš€ Alternative Implementations - -While this Elixir version works great, you might want to consider these languages for broader adoption: - -### Go Implementation - -- **Pros**: Single binary deployment, great performance, large community -- **Cons**: More verbose concurrency patterns -- **Best for**: Teams wanting simple deployment and good performance - -### Python Implementation - -- **Pros**: Huge ecosystem, familiar to most developers, excellent tooling -- **Cons**: GIL limitations for true concurrency -- **Best for**: AI/ML teams already using Python ecosystem - -### Rust Implementation - -- **Pros**: Maximum performance, memory safety, growing adoption -- **Cons**: Steeper learning curve, smaller ecosystem -- **Best for**: Performance-critical deployments - -### Node.js Implementation - -- **Pros**: JavaScript familiarity, event-driven nature fits coordination -- **Cons**: Single-threaded limitations, callback complexity -- **Best for**: Web teams already using Node.js - -## ๐Ÿค Contributing - -Contributions are welcome! Here's how: - -1. Fork the repository -2. Create your feature branch (`git checkout -b feature/amazing-feature`) -3. Commit your changes (`git commit -m 'Add some amazing feature'`) -4. Push to the branch (`git push origin feature/amazing-feature`) -5. Open a Pull Request - -See [CONTRIBUTING.md](CONTRIBUTING.md) for detailed guidelines. - -## ๐Ÿ“„ License - -This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details. - -## ๐Ÿ™ Acknowledgments - -- [Model Context Protocol](https://modelcontextprotocol.io/) for the agent communication standard -- [Elixir](https://elixir-lang.org/) community for the excellent ecosystem -- AI development teams pushing the boundaries of collaborative coding - ---- - -**Agent Coordinator** - Making AI agents work together, not against each other.