signet-eval

Deterministic policy enforcement for AI agent tool calls. Every action an agent proposes passes through user-defined rules before execution. No LLM in the authorization path. Advisory nudges are separate from authorization. 25ms end-to-end.

Install

# crates.io
cargo install signet-eval

# from source
git clone https://github.com/jmcentire/signet-eval
cd signet-eval
cargo install --path .

There is no npm or PyPI package for signet-eval. The public distribution path is crates.io plus source install from GitHub. The MCP Registry listing points at the repository metadata; the runtime is the local signet-eval serve stdio server.

Quick Start

1. Hook into Claude Code — add to ~/.claude/settings.json:

{
  "hooks": {
    "PreToolUse": [{
      "matcher": "",
      "hooks": [{"type": "command", "command": "signet-eval", "timeout": 2000}]
    }]
  }
}

For Codex, enable hooks in ~/.codex/config.toml or <repo>/.codex/config.toml:

[features]
codex_hooks = true

Then add ~/.codex/hooks.json or <repo>/.codex/hooks.json:

{
  "hooks": {
    "PreToolUse": [{
      "matcher": "*",
      "hooks": [{
        "type": "command",
        "command": "signet-eval --adapter codex",
        "timeout": 30000,
        "statusMessage": "Checking Signet policy"
      }]
    }],
    "PermissionRequest": [{
      "matcher": "*",
      "hooks": [{
        "type": "command",
        "command": "signet-eval --adapter codex-permission",
        "timeout": 30000,
        "statusMessage": "Checking Signet approval policy"
      }]
    }]
  }
}

2. Done. Every tool call now passes through policy evaluation. The default policy blocks destructive operations, protects its own configuration, and allows everything else.

3. (Optional) Customize — talk to Claude with the MCP server:

claude mcp add --scope user --transport stdio signet -- signet-eval serve

Then say: "Add a $50 limit for amazon orders" or "Block all rm commands".

Default Policy

Self-protection rules are locked — they cannot be removed, edited, or reordered by the AI agent, even through the MCP management server. This prevents the agent from disabling its own guardrails.

Custom Policy

signet-eval init       # write default policy to ~/.signet/policy.yaml
signet-eval validate   # check policy for errors
signet-eval rules      # show current rules

Edit ~/.signet/policy.yaml:

version: 1
default_action: ALLOW
rules:
  - name: block_rm
    tool_pattern: ".*"
    conditions: ["contains(parameters, 'rm ')"]
    action: DENY
    reason: "File deletion blocked"

  - name: books_limit
    tool_pattern: ".*purchase.*"
    conditions:
      - "param_eq(category, 'books')"
      - "spend_plus_amount_gt('books', amount, 200)"
    action: DENY
    reason: "Books spending limit ($200) exceeded"

  - name: protect_my_config
    tool_pattern: ".*"
    conditions: ["contains(parameters, '/etc/')"]
    action: ASK
    locked: true
    reason: "System config changes require confirmation"

Rules are evaluated in order — first match wins. Multiple conditions on a rule are AND'd. Rules with locked: true cannot be modified through the MCP management server.

Advisory Injection

INJECT rules probabilistically add advisory context near the tool call that triggered them. They are nudges, not authorization: the normal ALLOW/DENY/ASK/GATE/ENSURE pass remains first-match-wins and deterministic. Injection runs afterward and only emits context when a matching inject rule fires.

rules:
  - name: maybe_remind_kindex_on_git
    tool_pattern: "^Bash$"
    conditions: ["contains(parameters, 'git ')"]
    action: INJECT
    inject:
      trigger:
        mode: exponential
        peak: 0.35
        cooldown_seconds: 300
        peak_after_seconds: 1800
        max_per_session: 3
      payload:
        text: "Before committing, check whether project `.kin` files should be included."

Trigger modes:

Payload sources:

Template substitutions are enabled by default: {tool_name}, {cwd}, {date}, and {matched_param.X}. See examples/inject_examples.yaml.

Condition Functions

Encrypted Vault

Three-tier encrypted storage with passphrase-derived key hierarchy (Argon2id + AES-256-GCM):

signet-eval setup                      # create vault with passphrase
signet-eval store cc_visa 4111...      # store Tier 3 credential
signet-eval status                     # vault status and spending
signet-eval log                        # recent action log
signet-eval unlock                     # refresh session after timeout

Credentials support scoped access via request_capability: domain restrictions, purpose constraints, per-use amount caps, and one-time tokens that auto-invalidate after a single use.

Spending limits use the vault ledger — each tool call that spends money is logged, and spend_plus_amount_gt() checks cumulative totals before allowing the next purchase.

Self-Protection

signet-eval ships with locked rules that prevent an AI agent from disabling its own policy enforcement:

1. protect_signet_dir — Denies any Write, Edit, or Bash command touching .signet/ (policy files, vault, HMAC)
2. protect_signet_binary — Denies tampering with the signet-eval binary itself
3. protect_hook_config — Requires user confirmation before modifying settings.json (where the hook is configured)
4. protect_signet_process — Denies kill/pkill/killall commands targeting signet processes

These rules are:

• Locked — MCP tools refuse to remove, edit, or reorder them
• Position-protected — Unlocked rules cannot be reordered above locked rules (first-match-wins)
• Hardcoded in defaults — If the policy file is corrupted or missing, the binary falls back to hardcoded defaults that include self-protection
• HMAC-backed — Direct file edits break the policy signature, triggering fallback to safe defaults

MCP Management Server

Manage policies conversationally through Claude:

claude mcp add --scope user --transport stdio signet -- signet-eval serve

All mutating operations auto-sign the policy when the vault is available.

MCP Proxy

Wrap upstream MCP servers with policy enforcement. The agent connects to the proxy, never directly to servers. Policy is hot-reloaded on every call.

# Configure upstream servers
cat > ~/.signet/proxy.yaml << 'YAML'
servers:
  linear:
    command: npx
    args: ["-y", "mcp-linear"]
    env:
      LINEAR_API_KEY: "your-key"
YAML

# Register proxy with Claude Code
claude mcp add --scope user --transport stdio signet-proxy -- signet-eval proxy

All Commands

Performance

Architecture

signet-eval is the enforcement layer of the Signet personal sovereign agent stack. The core principle: the authorization layer must not be an LLM. It processes structured data only — regex, comparisons, and vault queries. No natural language, no context window, no persuasion surface. A rule either matches or it doesn't.

Agent proposes action  ->  signet-eval evaluates policy  ->  allow / deny / ask
                           (deterministic, 25ms, no NLP)

Security Model

signet-eval is a seatbelt, not a cage. Understanding what it does and doesn't protect against is critical.

What it protects against

A cooperative agent making mistakes. Claude Code and Codex follow hook protocols — they call signet-eval around tool use and respect the supported responses. Within those protocols, signet-eval reliably enforces policy:

• Blocks destructive commands (rm, mkfs, piped remote execution)
• Enforces spending limits across tool calls
• Gates credential access with domain, purpose, and amount constraints
• Requires user confirmation for sensitive operations (git push --force)
• Prevents casual policy tampering via locked rules and MCP guards

This is the normal operating mode. The agent isn't trying to escape — it's trying to be helpful and the policy keeps it in bounds.

Codex support has one important semantic difference: PreToolUse is currently deny-only for enforcement, so Signet ALLOW emits no output and Signet ASK is converted to a deny at that stage. The PermissionRequest adapter explicitly allows or denies requests Codex was already going to send through approval; Signet ASK defers to Codex's normal approval prompt.

What it does NOT protect against

An adversarial agent with shell access. signet-eval runs at the same privilege level as the agent it constrains. Any process with the same UID can:

• Use shell indirection to evade string matching (alias, eval, base64 -d | sh, python3 -c "...")
• Create symlinks to protected paths under innocent names
• Read the session key file and forge HMAC signatures
• Replace the signet-eval binary itself
• Modify the hook configuration in settings.json
• Use LD_PRELOAD, ptrace, or process injection

No amount of pattern matching fixes this. Adding rules for ln invites alias. Blocking alias invites eval. Blocking eval invites python3 -c. This is an infinite regress — the fundamental issue is privilege level, not pattern coverage.

Where real containment lives

Protecting against a truly adversarial agent requires OS-level controls that operate above the agent's privilege level:

• Separate user account — run the agent as a restricted user with no write access to policy files
• Container/sandbox — Docker, Flatpak, or bubblewrap with a read-only mount for ~/.signet/
• Stripped $PATH — remove ln, alias, and other indirection tools from the agent's environment
• seccomp / AppArmor / SELinux — mandatory access controls that the agent cannot override
• Virtual environment — run Claude Code in a constrained environment with limited filesystem access

signet-eval is the policy layer within such a setup. It handles the "what should this agent be allowed to do" question with clear, auditable rules. The OS handles the "can this agent circumvent the policy" question. Neither replaces the other.

Defense in depth

The layers work together:

Without OS controls, signet-eval is a speed bump, not a wall. With them, it's the policy engine inside a secure perimeter.

License

MIT