Local-first MCP server for LLM output verification — risk signals, injection and PII checks.
io.github.subodhkc/llmverify — Model Context Protocol (MCP) Server
This MCP server provides local-first LLM output verification. It focuses on detecting risk signals and performing checks for injection and PII within LLM outputs. The server is identified by the slug io.github.subodhkc/llmverify and is described as specifically aimed at verifying outputs for these concerns.
🛠️ Key Features
Local-first LLM output verification
Risk signal detection
Injection checks
PII checks
🚀 Use Cases
Verifying LLM outputs for potential injection risks
Checking LLM outputs for personally identifiable information (PII)
Identifying risk signals in generated text
⚡ Developer Benefits
Centralized verification workflow for LLM output concerns
Programmatic checks for injection and PII based on the server’s capabilities
⚠️ Limitations
Only the described verification scope (risk signals, injection, PII) is supported based on available server data
You shipped an AI feature. Your LLM hallucinated a citation, leaked a customer's email, and followed a prompt-injection buried in user input — on the same day. llmverify is the safety layer that sits between your LLM and your users.
Local-first verification, PII redaction, prompt-injection defense, and runtime monitoring for any LLM. One npm install. Zero telemetry. No API keys on the free tier.
You build with GPT-4, Claude, Gemini, or any LLM. The model:
Hallucinates facts and citations that do not exist.
Leaks PII — emails, phone numbers, SSNs, API keys in responses.
Follows prompt injections — users trick it into ignoring your instructions.
Returns broken JSON that crashes your parser.
Drifts in quality over time, and nobody notices until a user complains.
You need a guardrail between the model and your users. That is llmverify.
Install
bash
npm install llmverify
Everything runs locally. The free tier makes zero network requests and needs no API key. Free tier limit: 500 verification calls per day (tracked locally, never sent anywhere).
What you get
Function
One-liner
What it does
verify(content)
await verify(aiResponse)
Runs hallucination, consistency, safety, and CSM6 checks; returns a risk level and findings
isInputSafe(input)
isInputSafe(userMessage)
Blocks prompt injection, jailbreaks, and malicious input before it reaches the model
redactPII(text)
redactPII(aiResponse)
Masks emails, phones, SSNs, credit cards, and API keys
containsPII(text)
containsPII(text)
Returns true if PII is present
detectAndRepairJson(...)
detectAndRepairJson(prompt, response)
Detects and repairs broken JSON output
monitorLLM(client)
monitorLLM(openaiClient)
Wraps any LLM client; tracks latency, token drift, and behavioral changes
sentinel.quick(...)
await sentinel.quick(client, model)
Runs regression tests against your model before users see changes
classify(...)
classify(prompt, response)
Intent detection, hallucination signals, and instruction compliance
auditLog(event)
auditLog({ ... })
Appends a local, hash-only audit entry for SOC 2 / HIPAA / GDPR evidence
run, prodVerify, ciVerify
await prodVerify(content)
Preset pipelines for dev, prod, strict, fast, and CI use
Quick start (30 seconds)
javascript
const { verify, isInputSafe, redactPII } = require('llmverify');
// 1. Block prompt injection before it reaches the model.if (!isInputSafe(userMessage)) {
return { error: 'Invalid input detected' };
}
// 2. Verify the model's output.const aiResponse = await yourLLM.generate(userMessage);
const result = awaitverify(aiResponse);
if (result.risk.level === 'critical') {
return { error: 'Response failed safety check' };
}
// 3. Strip PII before the response reaches a user or a log.const { redacted } = redactPII(aiResponse);
console.log(redacted);
Three lines of safety between your LLM and your users. No config file required. No API key required.
How it works
llmverify runs deterministic, pattern-based engines locally — no model calls, no network on the free tier. Same input plus same rules equals same result. Every result carries an explicit limitations array stating what was and was not checked, so you never mistake a clean score for a guarantee.
Framework alignment (baseline mapping only — not certification):
OWASP LLM Top 10
NIST AI RMF
EU AI Act
ISO 42001
CSM6 (HAIEC's 38-rule control set)
CLI
bash
# Verify a string from the terminal.
npx llmverify verify "The capital of France is London."# Start a local HTTP API for IDE / tool integration (localhost only by default).
npx llmverify-serve --port=9009
# Expose to the network only on a trusted network. There is no auth on the API.
npx llmverify-serve --host=0.0.0.0 --port=9009
The server binds to 127.0.0.1 by default, restricts CORS to localhost origins, and rate-limits clients (100 requests / 60s). It requires express (an optional dependency that installs by default).
MCP server
llmverify ships a built-in Model Context Protocol server — the same engine, exposed to MCP-compatible agents and IDEs over stdio:
Six tools: verify_llm_content, assess_hallucination_risk, check_prompt_injection, check_pii, redact_pii, get_llmverify_capabilities. Stdio-only, zero outbound network, bounded inputs/outputs, PII-filtered responses, honest notChecked/audit semantics.
Requires Node.js ≥ 20 (the MCP SDK's floor; the rest of the package supports ≥ 18). The MCP SDK and zod are regular dependencies — the mcp command lazy-loads them so other commands pay no startup cost.
See docs/MCP.md for the full tool reference and security model.
Limitations
llmverify is a triage tool, not a truth oracle. Be honest with yourself about what it can and cannot do:
It cannot definitively prove hallucinations. Hallucination signals are pattern-based. "The capital of France is London" scores low because the text looks internally consistent. Ground-truth verification requires a source document you provide.
It does not replace human review. Use it to triage, not to approve.
PII detection is regex-based. It catches standard formats (emails, US phones, SSNs, credit cards, common API keys). It misses obfuscated, image-embedded, or encoded PII. Accuracy is roughly 90% for standard formats, lower for variations.
Prompt-injection detection is pattern-based. Novel or obfuscated injections can evade it.
Free tier is 100% local. ML-enhanced features require a paid tier and an explicit API key; the free tier never makes network requests and never sends data anywhere.
If a claim matters, verify it yourself. llmverify narrows the risk surface; it does not eliminate it.
Recommendation (not legal advice): Run verify() on every model output that reaches a user, and isInputSafe() on every user input that reaches a model. Treat the risk level as a triage signal, not an approval.
Install
Configuration
Environment variables
LLMVERIFY_MCP_MAX_INPUT_CHARSdefault 1000000
Max characters accepted in a content/input argument
LLMVERIFY_MCP_TIMEOUT_MSdefault 60000
Per-tool wall-clock timeout in milliseconds
LLMVERIFY_MCP_MAX_OUTPUT_ITEMSdefault 50
Cap on any single output array
LLMVERIFY_MCP_MAX_TEXT_FIELD_CHARSdefault 2000
Cap on any single text field in output
LLMVERIFY_HOME
Base directory for engine local state
LLMVERIFY_AUDIT_DIR
Audit JSONL directory
LLMVERIFY_LOG_DIR
Operational log directory
LLMVERIFY_BASELINE_DIR
Drift baseline directory
LLMVERIFY_USAGE_FILE
Usage counter file
LLMVERIFY_AUDIT_HASH_KEYsecret
Enables keyed hmac-sha256 content hashes in audit records
LLMVERIFY_AUDIT_NO_CONTENT_HASH
Set to 1 to disable content hashes in audit records