gnome-ui-mcp

Small MCP server for GNOME Wayland desktop automation.

It exposes GNOME desktop inspection and interaction through AT-SPI for discovery and Mutter RemoteDesktop for input. In practice that means element lookup, activation, typing, screenshots, and wait helpers for the current desktop session.

For navigation it follows a snapshot-driven model: take a snapshot of the active window to get stable element uids, then click/fill/hover/fill_form those uids. Stale uids are rejected rather than silently mis-targeted, and actions auto-wait for the UI to settle. See Navigation below.

Requirements

• Linux host with GNOME Shell on Wayland
• Live local GNOME session on the machine you want to automate
• Session environment available: DBUS_SESSION_BUS_ADDRESS, XDG_RUNTIME_DIR, WAYLAND_DISPLAY, DISPLAY, XDG_SESSION_TYPE

Docker

• Docker Engine

The container must run on the same machine as the GNOME session and use the session environment plus runtime mounts.

Docker image

The recommended way to run the server is via the published GHCR image:

ghcr.io/asattelmaier/gnome-ui-mcp:latest

latest tracks the most recent release. Version tags such as v0.1.0 publish matching image tags as well.

Docker setup

Direct docker run:

docker run --rm \
  --security-opt apparmor=unconfined \
  --network host \
  --user "$(id -u):$(id -g)" \
  -e DBUS_SESSION_BUS_ADDRESS="$DBUS_SESSION_BUS_ADDRESS" \
  -e XDG_RUNTIME_DIR="$XDG_RUNTIME_DIR" \
  -e WAYLAND_DISPLAY="$WAYLAND_DISPLAY" \
  -e DISPLAY="$DISPLAY" \
  -e XDG_SESSION_TYPE="${XDG_SESSION_TYPE:-wayland}" \
  -v "$XDG_RUNTIME_DIR:$XDG_RUNTIME_DIR" \
  -v /tmp/.X11-unix:/tmp/.X11-unix:ro \
  ghcr.io/asattelmaier/gnome-ui-mcp:latest

--user "$(id -u):$(id -g)" is required so the container joins the same user session as GNOME, D-Bus, and AT-SPI.

Local development via Compose:

This path additionally requires docker compose.

1. Copy .env.example to .env
2. Adjust the values to your session
3. Run:

docker compose build
docker compose run --rm gnome-ui-mcp

Available transports

The server supports three transports, both locally and in Docker:

• stdio (default): recommended for local MCP clients that spawn the server process
• streamable-http: recommended for HTTP-based integrations on http://127.0.0.1:8000/mcp
• sse: available for backwards compatibility on http://127.0.0.1:8000/sse with message POSTs to http://127.0.0.1:8000/messages/

Examples:

gnome-ui-mcp
gnome-ui-mcp --transport streamable-http
gnome-ui-mcp --transport sse

The same flags can be passed to the Docker image by appending them after the image name:

docker run ... ghcr.io/asattelmaier/gnome-ui-mcp:latest --transport streamable-http
docker run ... ghcr.io/asattelmaier/gnome-ui-mcp:latest --transport sse

Marketplaces

The repository includes metadata for these distribution channels:

• MCP Registry via server.json
• Claude Code plugin marketplaces via .claude-plugin/plugin.json
• VS Code and GitHub agent plugin marketplaces via .github/plugin/plugin.json

Claude Code plugin marketplace

To add this repository as a plugin marketplace in Claude Code:

/plugin marketplace add asattelmaier/gnome-ui-mcp

Then install the plugin:

/plugin install gnome-ui-mcp

The plugin starts the published Docker image through scripts/run-docker-mcp.sh, so Docker and the GNOME session environment must be available on the host.

Example MCP client configuration

{
  "mcpServers": {
    "gnome-ui": {
      "command": "docker",
      "args": [
        "run",
        "--rm",
        "--security-opt",
        "apparmor=unconfined",
        "--network",
        "host",
        "--user",
        "1000:1000",
        "-e",
        "DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus",
        "-e",
        "XDG_RUNTIME_DIR=/run/user/1000",
        "-e",
        "WAYLAND_DISPLAY=wayland-0",
        "-e",
        "DISPLAY=:0",
        "-e",
        "XDG_SESSION_TYPE=wayland",
        "-v",
        "/run/user/1000:/run/user/1000",
        "-v",
        "/tmp/.X11-unix:/tmp/.X11-unix:ro",
        "ghcr.io/asattelmaier/gnome-ui-mcp:latest"
      ]
    }
  }
}

Navigation

The blessed way to drive the desktop is snapshot-first:

1. take_snapshot — capture the active window (or pass window from list_windows, or app_name). Every element gets a stable opaque uid such as 7_42.
2. click / fill / hover / fill_form — reference elements by uid. Actions auto-wait for the shell to settle and report effect verification. Pass include_snapshot=true to get a fresh snapshot of the result in the same turn.
3. select_window sets the implicit snapshot scope.

A uid is only valid for the latest snapshot: if the UI changed and a uid is rejected as stale, call take_snapshot again and use a fresh one. The lower-level path-based tools (find_elements, click_element, set_element_text, …) remain available as advanced building blocks.

Tool categories can be enabled or disabled at startup with --category NAME / --no-category NAME (for example --category navigation --category input).

API Documentation

Complete tool reference available at https://asattelmaier.github.io/gnome-ui-mcp/

Security

This server can inspect and control the active desktop session. Use it only with trusted MCP clients. Containerized execution on Ubuntu may require --security-opt apparmor=unconfined so the process can talk to the GNOME session buses.