AI Process Manager — MCP Server

Structured Windows state for AI agents — no screenshots.

This MCP server exposes the AI Process Manager local HTTP API as tools for Claude Desktop, Cursor, and any MCP client. Instead of capturing pixels (~2,765 tokens per 1080p screenshot), agents read JSON and text (~15–150 tokens per query) from processes, windows, consoles, and UI Automation trees.

Requires: AIProcessManager.exe running on Windows (system tray). Node.js ≥ 14. Zero npm dependencies.

Why this exists

Computer-use agents often "look" at the desktop via screenshots. That is slow (3–5 s), expensive in tokens, and sends pixel data through the model. AIPM answers structured questions on loopback:

Measured on a real machine: ~94–98% fewer perception tokens per action vs screenshots.

Quick start

1. Start the backend

Ensure AIProcessManager.exe is running in your system tray.

2. Configure your MCP client

Claude Desktop — claude_desktop_config.json:

{
  "mcpServers": {
    "ai-process-manager": {
      "command": "node",
      "args": ["C:\\path\\to\\ai-process-manager\\mcp\\server.js"]
    }
  }
}

Cursor and other stdio MCP clients take the same command / args pair. Optional env var AIPM_API overrides the API base URL (default: read from %LOCALAPPDATA%\AIProcessManager\endpoint.txt, else http://127.0.0.1:9147).

3. Restart the MCP client

Tools appear as ai-process-manager. Call health_check first.

Tools (20)

Read tools are annotated readOnlyHint: true — clients may auto-approve them. Action tools are readOnlyHint: false and refuse to run unless the user opts in (see Privacy).

Action tier — opt-in, off by default:

Local telemetry (writes to the local store, metadata only):

Reading deep UI trees (Chromium/Electron)

get_ui_tree defaults to depth=4, which is enough for native Win32 apps. Chromium/Electron apps (VS Code, Slack, Discord, Claude Desktop, Teams) bury content under ~10 levels of Pane/Group wrappers — at low depth the response is only empty panes. Ask for depth=15-20 there (max 30) and cap cost with max_nodes (default 200, max 1000): max_nodes is the cost brake, not depth. When the response has truncated: true, the tree was cut — repeat with a higher depth/max_nodes before concluding anything about the window.

Privacy

Nothing leaves the machine. There is no remote telemetry, no cloud service, and no account.

Network

• The backend listens on the loopback interface only (127.0.0.1:9147) — not on 0.0.0.0, so nothing on the LAN can reach it.
• Every request must carry a Host header of localhost or 127.0.0.1; anything else is rejected with 403 forbidden_host (anti DNS-rebinding, so a web page you visit cannot drive the API).
• This MCP server makes exactly one kind of outbound call: HTTP to that local address. It sends a User-Agent of mcp:<your MCP client name> so the local audit log shows which agent asked.

Read-only by default

• 15 of the 20 tools are plain GET reads, annotated readOnlyHint: true.
• The 3 action tools (ui_invoke, ui_set_value, focus_window) return 403 action_denied until the user does both: enable Agent actions in the tray menu, and add the target process to a per-app allowlist. Neither is on by default, and the setting is per app — allowing Notepad does not allow the browser.

What the telemetry stores — metadata only

Recorded: app/process name, element role (Button, Edit…), the element name the agent asked for, the action (invoke/set_value/focus), success or failure, duration in ms, and a failure reason. Two mechanical invariants make "metadata only" verifiable rather than a promise:

1. An action record stores what the agent requested, not what the app displayed. The role and name come from the agent's own role=/name_contains= arguments. The name of the element actually resolved on screen is never written, so what the tool saw never becomes telemetry.
2. The failure reason is a closed vocabulary (elemento_nao_encontrado, ui_timeout, erro_uia, outro, …). Any other string is stored as outro. An exception message — which could carry a file path or on-screen text — therefore cannot reach the disk.

Never stored: screen contents, window text read by read_window, the text typed by ui_set_value (value=), file contents, keystrokes, screenshots. The passively learned UI shape (ui_shape) holds only counts, UIA role names, depth and booleans: exposes_text says whether text exists, named_controls says how many elements have a name — never the text itself.

Masking: before any request is recorded, value= and api_key= are replaced with ***, so neither get_audit_log nor the on-disk query log can reveal typed text or a secret.

One honest nuance: the audit/query log stores the request line, so other query arguments stay readable — e.g. check_file(path=D:/videos/out.mp4) is logged as that path, and check_process(title_contains=...) keeps that fragment. It is a local log of what the agent asked for. Only value= and api_key= are masked.

Where the data lives, and how to delete it

Everything is under %LOCALAPPDATA%\AIProcessManager\:

To erase: quit the app from the tray, then delete the folder (or just db\ to reset learning and the economy counters; deleting actions.cfg turns the action tier back off). Nothing is written anywhere else, and the /audit ring buffer lives in memory only — it disappears when the app closes. You can inspect everything the store holds with get_audit_log, get_economy_stats, and GET /analytics/actions.

Troubleshooting

Every error payload carries a next_action field with the same guidance.

Coverage (measured, honest)

We publish what does not work yet on purpose — you should know the edges before relying on it.

Behaviour note: the first read of a Chromium/Electron window asks it to activate its accessibility tree — the same standard request a screen reader makes. That app then keeps computing accessibility data (a CPU cost in that app) and does not go back to sleep on its own. Native Win32 apps are unaffected.

Free vs paid

• Free & open (MIT): this MCP server.
• Free (closed): the AIProcessManager.exe backend — the sensor. Yours to run at no cost.
• Paid: AIPM Pilot, the autonomous computer-use agent that drives apps end-to-end using AIPM's structured perception. See promoflix.site.

Support the project

If AIPM saves you tokens, consider sponsoring.

License

MIT for the MCP server. The backend and AIPM Pilot are separate products.