CompareGraph

Many cloned repositories, one taxonomy, evidence on every cell.

CompareGraph parses many cloned repositories into one SQLite graph. It scores each clone against a taxonomy (a "family" of features) and records, per clone and feature, a verdict with a citation to a file and line range. It is one Rust binary with a CLI and an MCP server. It reads clones and never runs them. It never calls a model.

Contents

• Why CompareGraph
• Install
• Quick start
• How good is it
• Languages
• How it works
• What you get
• CLI reference
• MCP tools
• Plugins and the second engine
• Families and the seven skills
• The vault
• Configuration
• Agents, platforms, uninstall
• Library usage
• Troubleshooting
• FAQ
• Roadmap
• Documentation
• Contributing, licence, third-party code

Why CompareGraph

You have many repositories and one question: who has what? Reading each by hand does not scale, a summary without a line number cannot be checked, and notes go stale when the code moves. Five properties, each one you can test:

1. Deterministic. The same clone and version give the same store_hash. comparegraph check rebuilds and compares, and an incremental build must equal build --full (equal on both large trees in the Plan 6 report).
2. An evidence class on every answer. Each envelope says observed, derived, inferred or unknown, plus a limitations[] list of what the engine cannot see.
3. A claim needs a citation that resolves now. has and partial need --at path:start-end matching a stored node; otherwise the engine refuses. A moved citation shows as unknown* after check.
4. none needs proof of absence. Either --searched terms that find nothing, or the graph check: four channels (symbols, paths, community terms, node kinds) all empty. Any hit refuses the claim and lists the candidates.
5. No model in the engine. Scores, bands, descriptions and candidates are computed from stored facts. An agent may do the reading; the engine only stores, checks and refuses.

For: teams comparing their product with the field, due diligence on a set of projects, a newcomer mapping a field (family propose drafts the taxonomy), and coding agents that need compact facts over MCP.

Install

Two steps: the binary, then your agent.

1. The binary. One line, into ~/.local/bin, checked against the release's SHA256SUMS (linux-x64, linux-arm64, darwin-arm64, darwin-x64):

curl -fsSL https://raw.githubusercontent.com/massrelay-ai/comparegraph/main/scripts/install.sh | sh

Windows x64, in PowerShell: irm https://raw.githubusercontent.com/massrelay-ai/comparegraph/main/scripts/install.ps1 | iex. Homebrew: brew install massrelay-ai/comparegraph/comparegraph. By hand: comparegraph-<platform> and SHA256SUMS from the latest release. Any other platform, with Rust 1.85 or newer: cargo install --locked --path crates/comparegraph in the clone (--locked matters: the Perl grammar's newest release on crates.io is yanked, and only Cargo.lock resolves).

2. Your agent. Claude Code installs the seven skills and the MCP server as one plugin; it runs the comparegraph binary from your PATH:

/plugin install comparegraph --marketplace massrelay-ai/comparegraph

Any other agent (Codex, Cursor, Gemini CLI, OpenCode, Hermes, Windsurf, and more): comparegraph setup --client <name> --apply writes the entry (omit --apply for a dry run) and comparegraph doctor checks it (table). setup --apply also writes a short rules block into the agent's instruction file in the workspace and, for Claude Code, a session-start hook; the plugin ships the same hook. The server is listed in the MCP Registry as io.github.massrelay-ai/comparegraph.

Quick start

The example family pack (families/) and the sample repositories (fixtures/) ship in the repository, not in the binary, so clone it once even if you installed the binary another way; the commands below start inside that clone. The sample is fixtures/typescript (three files) against the ten-feature example pack:

mkdir ../ws && cd ../ws
comparegraph add ../comparegraph/fixtures/typescript --name tsfix   # register a clone (read, never run)
comparegraph build tsfix                                   # parse it: one run, one store hash
comparegraph family add ../comparegraph/families/example-code-graph-tools
comparegraph relate                                        # related | adjacent | unrelated
comparegraph map tsfix example-code-graph-tools            # candidates per feature, writes nothing
comparegraph claim tsfix symbols.extraction has --author me --at src/util.ts:11-11
comparegraph claim tsfix search.embeddings none --author me  # no --searched: the graph check runs
comparegraph matrix example-code-graph-tools               # features x clones, prevalence, gaps
comparegraph export vault --out ./vault                    # open ./vault in Obsidian

Every command prints one JSON envelope (add --pretty to indent it). Real output from the steps above, trimmed:

build   {"run":1,"nodes":13,"edges":21,"files_parsed":3,"communities":1,
         "store_hash":"8a64f321...","reused":false}
relate  {"repo":"tsfix","family":"example-code-graph-tools","score":0.0,"band":"unrelated"}
claim   {"feature":"symbols.extraction","verdict":"has","confidence":"derived",
         "locations":[{"path":"src/util.ts","line_start":11,"line_end":11,"stale":false}]}
claim   {"feature":"search.embeddings","verdict":"none","confidence":"inferred",
         "note":"graph-absent (names, paths, community terms, kinds; not strings, flags or config keys):
                 terms=embedding,embeddings,search; stems=embedd; channels=symbols:0,paths:0,communities:0,kinds:0"}
claim   a bad citation is refused: "src/nope.ts: file does not exist in the clone"  (error invalid_arguments)
matrix  symbols.extraction -> has (prevalence 1.0);  search.embeddings -> none;  the other eight cells "-"
check   {"repos":1,"runs_done":1,"stale_locations":0,"drifted_runs":0,"schema_ok":true,"problems":[]}
export  {"written":"./vault","repos":1,"files":27,"communities":1,"symbols":10}

relate says unrelated, as it should: a three-file fixture is not a code-graph tool, and map finds no candidates. The matrix has two claimed cells and eight unclaimed - cells.

How good is it

Every number below is copied from a report in this repository; read its caveats before quoting one. All were measured on 2026-10-13 on one shared host.

Call edges against a hand-labelled gold standard

The gold: 150 call sites per project (750 in all) drawn at random from each project's own source, labelled by two readers who worked independently under one rubric (both are language-model agents; none was run blind to the engines). final keeps only the rows both agree on. How the gold is made: fixtures/gold/README.md. Precision is over answered sites, recall over repo-target sites (shown in brackets). Kappa is Cohen's kappa on the call target between the two readers.

Pooled counts: CompareGraph 196 correct of 197 answered, 196 of 235 sites; codegraph 119 of 168 answered, 119 of 235. Sources: docs/reports/2026-10-12-engine-parity.md, sections "After Plan 6" and "codegraph head-to-head"; kappa from "After Plan 5", inter-reader agreement (pooled is_call kappa 0.94). Reproduce: scripts/gold-score.sh <project-copy> fixtures/gold/<project>/calls.jsonl --binary <comparegraph>, scripts/compare-engines.sh (all engines), scripts/h2h-codegraph.sh (codegraph).

Read it with these limits. Sites per project are 36 to 53, so per-project noise is about 0.1. caddy 1.000 means "every labelled site" (48 sites drawn before the Go work), and hono precision rests on 24 answered sites. codegraph is run in its --graph-only mode through one get_callees call per site; where it lists several same-named targets our scorer counts the site correct if the true one is among them. On hono none of its sites is answered, because the tightest node it finds there is an anonymous arrow function with no call edges; a lookup through the nearest named ancestor was not run. The floors in fixtures/gold/FLOORS.json are asserted by crates/comparegraph/tests/gold.rs and are only ever raised.

Build and query time

Caveats from the reports: build times ran at load 6 to 15 on 40 cores, so read the ratios, not the seconds. The hermes-agent build is 1.40x slower than at the end of Plan 6 lane B1, a regression that was measured and not diagnosed. Query bounds are one hub symbol (3,156 callers), at defaults, one run each. A 4.4k-file private product monorepo builds in 103 s (53k nodes) and its three harness comparators in 25 s, 81 s and 436 s (proof report, last section); the ranked map over 12 features takes 1.2 s on the product and 12.1 s on the 454k-node comparator. Reproduce: scripts/compare-engines.sh, scripts/h2h-codegraph.sh, and COMPAREGRAPH_PROFILE=1 comparegraph build <repo>.

Overlap with code-review-graph (symbols and edges)

Jaccard over nodes and resolved edges, on three projects (parity report, "After the review fixes"; symbols and extends from "After the precision and recall pass"):

Test links are the weak spot on all three projects. Reproduce: scripts/compare-engines.sh.

Conformance fixtures (reference engine)

comparegraph conform reference runs the golden fixtures in fixtures/; a provider below the floor (nodes 0.95, edges 0.85, recall and precision) is refused. Output at this commit:

Five fixtures are five small repositories: they prove the extractors against a hand-checked graph, not every language feature.

Feature mapping gold

A second gold standard scores map (feature to code) and the graph-backed absent check: 30 features of families/code-graph-tools/rows.md against six study copies of code-graph tools, 180 pairs read by two independent readers, 121 agreed, 13 unresolved, disagreements left out of the scores (method and kappa: fixtures/gold-map/README.md). The ranking was tuned on three of the clones and measured once on the other three; the held-out column is the honest number for a repository the tool has never seen.

The target (held-out recall@5 at or above 0.50 at precision@5 0.30) is open: a gold location is among the first 200 candidates for only a third of the held-out features, because the store holds symbol names, paths and community keywords and no doc text; the next lever is in the roadmap. relate bands are a known weak signal: repositories outside the family still score above 0.5 (docs/decisions.md, "Plan 7 IMP"). Sources: docs/reports/2026-10-14-mapping-gold-first-measurement.md and the parity report's "Feature-mapping baselines". Reproduce: scripts/map-score.sh, scripts/map-baselines.sh.

Languages

29 languages are parsed. Five are resolved deeply. Four have a measured gold standard. Read the tiers, not the headline.

The registry holds 32 identifiers (tsx, jsx, mjs count with their language). Unresolved languages bind a call only by name, at confidence 0.5, and only inside the same language family. Known gaps: a Java new T() has no call site, a Rust trait method and its impl share one qualified name, and Go calls on a method result, range variables and generics bind nothing yet (Plan 7 candidates). Skipped on every build: the directories node_modules, target, dist, build, .git, vendor, __pycache__, .venv, venv; files that .gitignore or .ignore exclude; minified files. Skips are counted in the run's limitations.

How it works

flowchart LR
    clones["cloned repos"] --> engines["engines: reference (tree-sitter), external plugins"]
    engines --> store[("SQLite store, schema v1")]
    store --> graph["communities, search, graph queries"]
    store --> families["families: taxonomy packs"]
    families --> relate["relate and map"]
    relate --> claims["claims with citations"]
    claims --> matrix["matrix: gaps, differentiators"]
    graph --> out["exports and MCP"]
    matrix --> out
    claims --> out

The code is arranged in rings; dependencies point inward, and the folders under crates/comparegraph/src/ carry the same names. Two engines sit in the providers ring: the built-in reference engine, and any external plugin that speaks the plugin protocol.

flowchart LR
    adapters["adapters: CLI, MCP"] --> application["application: workflows"]
    application --> providers["providers: reference tree-sitter engine, external plugins"]
    providers --> store["store: SQLite"]
    store --> domain["domain: graph, claim, evidence, run"]

The three rules

1. The engine never calls a model. Every score, band, description and candidate is computed from stored facts.
2. A claim needs a citation that resolves now. has and partial need --at path:start-end matching a stored node. none needs --searched terms that find nothing, or the graph check. Otherwise the engine refuses or writes unknown.
3. Claims and decisions are append-only, enforced by the database. A change is a new row naming the one it supersedes. The latest wins, and the trail stays.

A candidate from search or map is labelled candidate, never has. Design: docs/design.md.

What you get

Four levels. Each answer says how it knows.

A real show answer from the quick start (trimmed):

{"ok":true,"verb":"show","schema":1,
 "data":{"repo":"tsfix","lane":"study",
  "node":{"kind":"method","qualified_name":"src/util.ts::bump","path":"src/util.ts",
          "line_start":11,"line_end":11,"language":"typescript"},
  "community":"src","callers":[],"callees":["src/util.ts::add"],"stale":false,"confidence":"observed"},
 "evidence":{"confidence":"observed","limitations":[]}}

Envelope fields: ok, verb, schema, data, evidence, page, cost, error. The exit code is 0 when ok and 1 otherwise. List verbs take --limit (default 50, maximum 500) and --cursor. Error codes (from docs/catalog.json): below_floor, catalog_stale, check_failed, empty, invalid_arguments, io_error, not_found, provider_unavailable, store_error.

CLI reference

Global options on every command, before or after the verb: --root <ROOT> (the workspace, default .), --pretty, --allow-unverified (use an external provider with no manifest). comparegraph <verb> --help prints every flag.

Build and inspect

Graph queries

Families and claims

Exports and agents

MCP tools

comparegraph mcp speaks JSON-RPC 2.0 over stdio, one object per line, protocol 2025-06-18. Every tool returns the same envelope as the CLI. Eighteen tools: seventeen read-only, one write (comparegraph_claim). The list is generated into docs/catalog.json and checked in CI by comparegraph catalog --check.

There is no decide tool: adopting or rejecting a clone is a human act on the CLI. min_confidence is 0.5 (default, includes guessed hops, labelled in evidence) or 1.0. A name-only match is capped at 0.5 and never binds onto a method. Truncation (depth, max_nodes) is recorded in evidence.limitations. By hand, add this to .mcp.json:

{"mcpServers":{"comparegraph":{"command":"comparegraph","args":["--root","/path/to/workspace","mcp"]}}}

Plugins and the second engine

An external provider is any executable that speaks JSON lines on stdin and stdout: the host writes one request line, closes stdin, and reads answer lines until the first ok answer. Everything it prints is data, parsed into our own types. Register one in comparegraph.toml ([providers.mine] command = "tools/my-provider") or as comparegraph-provider-<name> on PATH, with a comparegraph-plugin.toml manifest beside it (kind, name, protocol = "1", languages, network). Without a manifest it runs only with --allow-unverified.

Guards: a 1800 s timeout (COMPAREGRAPH_PLUGIN_TIMEOUT) that kills the whole process group, a 256 MiB output cap, paths that are absolute or contain .. dropped, and a failure becomes provider_unavailable with no nodes written. Conformance is per language: comparegraph conform <name> runs the five fixtures, and build refuses only when a language present in the clone failed its fixture. Full protocol, a 25-line Python provider and the failure matrix: docs/dev/plugin-protocol.md.

The second engine. providers/crg/ is comparegraph-provider-crg, an adapter that runs code-review-graph 2.3.9 through the protocol (registered in the root comparegraph.toml; code-review-graph itself must be installed, else doctor shows it unavailable). comparegraph build <repo> --providers reference,crg builds both and prints an agreement table. On srclight: calls Jaccard 0.962, contains 0.981, imports 0.904, tests 0.594 (below the 0.8 mark), nodes 0.797 to 1.000 by kind (parity report, "Agreement between the engines"). Adapter conformance: go, python and typescript pass; java (edge recall 0.77) and rust (nodes 0.86, edge recall 0.53) stay below the floors, so the gate refuses it on clones that contain them. Through the adapter, code-review-graph scores like its native self on three gold projects and slightly higher on hono and caddy (the gold table).

To add your own engine, write a provider and pass comparegraph conform <name>; a provider below the floor is refused.

Families and the seven skills

A family is a directory with a rows.md: front matter with family: and description:, then a table with id, group, feature, definition, counts_as_has, tier (basics, common, extra or candidate) and weight, plus an optional eighth column synonyms (comma-separated words that also name the feature, e.g. reattempt, backoff). Synonyms are terms of map and of the absent check, and a claim none --searched that uses one counts as naming the feature; an undeclared unrelated word is still refused. A header with any other ninth column is refused. A feature id is <group>.<slug>: lowercase [a-z0-9-], exactly one dot, and the group column must equal the id's group (symbols.extraction has group symbols); a row that breaks it is refused when the pack loads. Two packs ship: families/example-code-graph-tools (10 features, the quick start) and families/code-graph-tools (30 features, the mapping gold). family propose <name> --repos a,b,c drafts a pack from the terms the clones share, with no model. The tier names and band thresholds (related at 0.5 or more, adjacent at 0.2 or more) are provisional, and a relatedness score is vocabulary overlap, not evidence (design notes).

The method is seven plain-markdown skills in skills/ that any agent can read. Each gives when to use the verb, the exact commands, what the engine refuses, and how to check. In Claude Code the skills and the MCP server arrive together as one plugin (Install); other agents read the skills/ directory from the clone.

The vault

comparegraph export vault --out ./vault regenerates the whole vault from the store.

vault/
  index.md      llms.txt      AGENTS.md     entry point, a short map, the short workflow for an agent
  repos/        one note per clone
  communities/  symbols/      one folder per clone
  families/     features/     one note per family; one folder per family with a note per feature and its cells
  history/      one note per clone: the dated claim and decision trail, superseded rows marked

Every note starts with a front-matter description: built from stored facts by a template, not by a model, for example function pkg/core.py::add in pkg/core.py:1-2, 3 callers, 0 callees. An agent reads the descriptions first and opens a note only when the line is not enough. At 4.4k files the vault is one file per symbol (66,970 files, 275 MB in the proof report), too large to open as one vault in a note tool.

Configuration

Agents, platforms, uninstall

comparegraph setup writes the entry; comparegraph doctor reports each client's state (aligned, stale, conflict, missing, unverified). --client all acts only on clients that look installed. setup --apply is additive and leaves a backup named <file>.bak-<unix-seconds>; a differing entry is left alone unless you pass --force.

Besides the entry, setup writes a CompareGraph block, between <!-- comparegraph --> and <!-- /comparegraph -->, into the agent's instruction file in the workspace root (--root, default the current directory; the Rules file column; only once .comparegraph/ exists there, so run comparegraph add first; skip it with --no-rules), and for Claude Code a SessionStart hook in .claude/settings.json (project scope) or ~/.claude/settings.json (user scope; skip it with --no-hook). The hook runs comparegraph doctor --brief, which prints nothing outside a workspace with a store. A block from an older release is refreshed in place; a block you edited by hand is left alone unless you pass --force. Markers quoted inside a fenced code block, or a second block, count as the block (the first marker pair wins); a close marker alone is ignored and a block is appended.

To uninstall the rules block, delete the lines from <!-- comparegraph --> to <!-- /comparegraph -->; to uninstall the hook, delete the SessionStart entry whose command is comparegraph doctor --brief.

Platforms: every release builds linux-x64, linux-arm64, darwin-arm64, darwin-x64 and win32-x64 on their own runners and ships each as a release asset listed in SHA256SUMS; the Homebrew tap follows the release. The test suite runs on linux-x64 only. Anywhere Rust 1.85+ runs, build from source. Local only: the engine makes no network call, the MCP server speaks over stdio, the store is one SQLite file you own.

Uninstall: remove the entry above (in Claude Code with the plugin, /plugin uninstall comparegraph), then delete the binary (~/.local/bin/comparegraph, brew uninstall comparegraph or cargo uninstall comparegraph), and rm -r .comparegraph in each workspace.

Library usage

The crate is comparegraph. Each verb is a function in comparegraph::application::* taking the workspace root; the CLI and MCP server are thin adapters over them.

use comparegraph::application::{add, build, matrix};
use comparegraph::providers::BuildOptions;
use std::path::Path;

let root = Path::new(".");
add::add_repo(root, Path::new("../comparegraph/fixtures/typescript"), Some("tsfix"), false)?;
let report = build::build_repo(root, "tsfix", "reference", &BuildOptions::default())?;
let m = matrix::matrix(root, "example-code-graph-tools")?;
println!("{} nodes, {} comparators", report.nodes, m.comparators);

The stable contracts: the JSON envelope (export json, every CLI answer), and crates/comparegraph/src/store/schema.sql for read-only SQL on .comparegraph/comparegraph.db. claim and decision rows are append-only: read them, do not update them.

Troubleshooting

FAQ

How is this different from code-review-graph and codegraph? Both index one repository for navigation and search; CompareGraph's unit of output is a cell in a features-by-clones matrix, with a citation. On the gold set above it scores higher on call-edge precision and recall, and builds at comparable speed (table above, with its limits). What they have that we do not:

The table follows the parity report ("Feature parity") for code-review-graph and the tool lists of codegraph's MCP server, which exposes memory, doc-index and pr_context tools. Our parser layer is taken from graphmind (MIT).

Does it use an LLM? No. An agent fits in the loop: it reads map candidates, opens the cited lines, and writes the claim with its own name as --author.

Can it adopt code for me? No. It gives you path:start-end and the clone's licence lane. Taking the code is a person's decision, recorded with decide. A port-kit export is on the roadmap.

Why are some call edges 0.5? A call matched by name only, with no import or declared type behind it, is a guess, stored at 0.5 (0.4 to 2.0 percent of call edges on the three parity projects; 24 to 33 percent on hono and caddy before Plan 6). Queries take --min-confidence 1.0 to drop them.

Is my code sent anywhere? No. The engine makes no network calls.

What does it not do? It does not diff two versions, run a clone, or infer types. A none claim means a search found nothing, not that the feature is absent. Entry points and dead code are candidates, never proof.

Roadmap

From docs/roadmap.md; none of these exist yet, and the maintainer decides the order.

• Feature stats export: per feature, how many comparators carry it and the counts behind it. Counts only; no priority formula.
• Claim import from a CSV, and family import/export for a shared taxonomy format.
• Graph-parity queries: circular dependencies, dead imports, complexity and hot paths, an architecture summary, edit context.
• A resolver for C, C++ and C# (they are parsed today, not resolved).
• A per-file analysis cache and an edge index, for the large-tree build and traverse times above.
• Go and TypeScript recall leftovers; crg adapter gaps.
• Port-kit export: for an adopted feature, the files, line ranges, licence text and attribution header.
• A text channel in the store (doc-comment and signature tokens) and a per-family synonym table, the two levers the mapping gold points at; a second signal for relate so that a repository outside a family scores below 0.5.
• Distribution: more clients (Zed, Continue) once their config shapes are verified.

Documentation

Contributing, licence, third-party code

Security reports go to support@massrelay.ai or a private advisory (SECURITY.md); conduct reports to the same address (CODE_OF_CONDUCT.md).

Read CONTRIBUTING.md and AGENTS.md first. A bug fix starts with a failing test, extraction changes extend a conformance fixture, and no model or network call enters the engine. The vendored parser layer comes from graphmind (MIT); see docs/THIRD_PARTY_NOTICES.md. Licence: Apache-2.0, see LICENSE.