Root Signals

Root Signals is an MCP server that connects AI assistants and agents to Scorable evaluators and LLM-as-a-judge capabilities. Designed for AI engineers, prompt developers, and autonomous agents, the server allows clients like Cursor or Claude Desktop to grade model responses, test prompt templates, and enforce internal coding rules. By surfacing account-level evaluators as standardized MCP tools, agents can inspect the quality of their own outputs across dimensions such as conciseness, relevance, clarity, and faithfulness. It also supports RAG evaluation by comparing generated responses against reference context passages. While the repository provides local Docker SSE and stdio implementations, the codebase is deprecated in favor of Scorable's hosted remote MCP server endpoint, which supports extended functionality like conversation scoring and dynamic judge generation without managing local processes.

Category: AI & LLM Tooling

Tags: evaluation, llm-ops, monitoring, testing

Visit Root Signals

How to install and configure Root Signals

1. Obtain your API key by signing up or generating a key at Scorable. 2. To run via Docker using SSE transport, start the container: bash docker run -e SCORABLE_API_KEY=<your_key> -p 0.0.0.0:9090:9090 --name=rs-mcp -d ghcr.io/scorable/scorable-mcp:latest 3. Register the SSE endpoint in your client configuration (such as Cursor): json { "mcpServers": { "scorable": { "url": "http://localhost:9090/sse" } } } 4. Alternatively, configure stdio transport directly in Cursor or Claude Desktop: json { "mcpServers": { "scorable": { "command": "uvx", "args": ["--from", "git+https://github.com/scorable/scorable-mcp.git", "stdio"], "env": { "SCORABLE_API_KEY": "<myAPIKey>" } } } }

What you can do with Root Signals

• Self-improving agent outputs: Enabling Cursor Agent to evaluate initial code explanations against clarity metrics and iteratively rewrite them for higher quality. - Validating prompt templates: Scoring custom system prompts and extraction templates directly inside the IDE to verify clarity, safety, and precision before deployment. - Benchmarking RAG responses: Running faithfulness and relevance evaluators against combined user requests, generated outputs, and retrieved context chunks. - Enforcing coding policies: Evaluating generated scripts or code diffs against repository AI rules files and internal style policy documents. - LLM-as-a-judge execution: Triggering complex multi-evaluator judge collections configured in a Scorable account using judge identifiers.

Key facts

• https://github.com/root-signals/root-signals-mcp
• AI & LLM Tooling, Monitoring & Observability
• evaluation, llm-ops, monitoring, testing

Part of MCP Servers

What is Root Signals MCP?

Root Signals is an MCP server bridging Scorable API evaluation metrics with MCP clients. It allows LLMs and autonomous agents to programmatically evaluate prompt outputs, enforce coding adherence rules, and measure conversational quality against predefined standards.

What tools are available in Root Signals MCP?

The server includes tools to list evaluators and judges, run standard evaluations by ID or by name, execute LLM-as-a-judge collections, and check adherence against specified coding policy documents.

Which MCP clients work with Root Signals?

It functions with any client supporting SSE or stdio transports, including Cursor, Claude Desktop, and custom Python clients using the provided reference implementation.

Is the Root Signals MCP repository actively maintained?

The repository is deprecated and no longer updated. Scorable now provides a hosted remote MCP server over HTTP that covers these tools plus dynamic judge creation and conversation-level scoring.