obsbot-mcp

A cross-platform Model Context Protocol server that controls an OBSBOT Tiny 2 camera over its standard UVC/USB interface — pan/tilt/roll the gimbal, zoom, AI subject tracking, focus/exposure/white-balance/image controls, HDR and field-of-view, plus snapshot, preview, and recording — without any vendor SDK.

Install

npm install obsbot-mcp

MCP client configuration

Add a stdio server entry pointing at the installed binary (or directly at dist/index.js):

{
  "mcpServers": {
    "obsbot": {
      "command": "obsbot-mcp"
    }
  }
}

If you're running from a local checkout instead of an npm install, point command/args at node and the built entry point instead:

{
  "mcpServers": {
    "obsbot": {
      "command": "node",
      "args": ["path/to/obsbot-mcp/dist/index.js"]
    }
  }
}

Debug / diagnostics tools

By default the server advertises only the normal control surface. Pass --debug to additionally expose the diagnostics surface — the obsbot_debug_probe tool (raw XU byte get/set/query) and the raw 60-byte status block on obsbot_status:

{
  "mcpServers": {
    "obsbot": {
      "command": "node",
      "args": ["path/to/obsbot-mcp/dist/index.js", "--debug"]
    }
  }
}

With the installed binary, use "command": "obsbot-mcp" and "args": ["--debug"].

Tools

36 tools total. All names below are current as of v0.4.0 — every tool was renamed in this release and there is no backward-compatible alias; see CHANGELOG.md for the full old→new mapping if you're updating a caller.

The camera selector

Every camera-addressing tool accepts an optional camera parameter: the target camera's serial number. Omit it with a single camera attached and nothing changes — this matches the server's pre-v0.4.0, single-camera behaviour exactly. With more than one camera attached, a call that omits camera fails with an error naming every attached serial, so you always know what to pass next.

Exempt (no camera parameter, ever): obsbot_devices (enumerates the whole fleet), obsbot_capture_stop / obsbot_capture_list (address a sessionId, not a device), and obsbot_debug_probe (operates on the current diagnostics transport). Two more tools honor it only partially — see Capture below.

Multi-camera support is new in v0.4.0. It's exercised by the unit test suite against fakes; running two physical Tiny 2s at once has not yet been hardware-verified (see Known limitations).

obsbot_devices is the way to discover the serials you pass as camera: it reports each attached camera's serial (where obtainable — reading it requires briefly opening the camera), name, and status (available | bound | busy). A camera another process already holds comes back busy with no serial, since it can't be opened to read one.

Device & power

Gimbal (PTZ)

Aiming at what you can see

obsbot_capture_snapshot returns the frame as an image plus its width/height, so a model can locate something in the picture and then point the camera at it:

1. obsbot_capture_snapshot — look at the frame
2. obsbot_aim_at_pixel — pass the target's pixel and that frame's dimensions
3. obsbot_capture_snapshot again — confirm it landed, and repeat if needed

Pass the frameWidth/frameHeight from the same snapshot the pixel came from. Mixing a pixel from one frame with dimensions from another aims at the wrong place, and nothing can detect it.

The snapshot must be source: "device". obsbot_capture_snapshot can also read from source: "virtual" or "ndi", which come from OBSBOT Center's own output rather than the camera's raw stream. Those are framed and cropped by OBSBOT Center, not by this camera's optics, so the measured field-of-view constants this tool relies on don't describe them — aiming from a virtual or NDI frame lands in the wrong place with no way to detect it. Only use a device-source snapshot's pixel and dimensions here.

The tool reads the camera's magnification itself — a discrete FOV mode or a continuous zoom alike (m = 3*ratio-2, measured on hardware to better than 0.05%) — so there is no FOV or zoom argument to get wrong, and it works at any zoom. It refuses rather than guessing when AI tracking is on (tracking drives the gimbal and would fight the aim), when the FOV mode can't be decoded, when a corrupt zoom reading would resolve to an implausible magnification, or when the camera had to be woken from sleep (waking moves the gimbal, so the frame you measured no longer matches where the camera is pointing — take a fresh snapshot and retry).

Framing what you can see

obsbot_zoom_to_fit extends the same idea from a point to a region: instead of just centring on a pixel, it also zooms so that region fills the frame.

1. obsbot_capture_snapshot — look at the frame
2. Pick a bounding box around whatever should fill the frame (a face, a whiteboard, ...)
3. obsbot_zoom_to_fit — pass the box (x, y, width, height) and that frame's dimensions
4. obsbot_capture_snapshot again — confirm the framing, and repeat if needed

It shares obsbot_aim_at_pixel's refusals (AI tracking, undecodable FOV/zoom, a woken camera, a zoom still ramping, a non-16:9 frame), and adds one of its own: the region must lie within the frame — edges included, so a region that already IS the full frame is valid — with a positive width and height, or the call refuses rather than guess what a negative width or an off-frame box was supposed to mean.

margin (default 0.1, i.e. 10%) backs the requested zoom off by that fraction so the region isn't framed exactly edge-to-edge — some breathing room around it survives small aim/zoom error. The region's two axes rarely need the same zoom to fill the frame; the tool always picks the smaller of the two required magnifications, because zooming to the larger one would fill one axis by cropping the other. The result is clamped to the camera's [1x, 4x] magnification range (reported via clamped) — a region demanding more zoom than the camera has still gets the closest fit available, rather than being refused outright.

The gimbal moves before the zoom is commanded. Zoom re-centres what's already in frame but does not keep a specific pixel under the crosshair as it changes — zooming first can push the region's centre out of frame entirely, which would make the subsequent move aim at a pixel that no longer means what it did when the caller measured it.

Zoom is not instantaneous: on this hardware it ramps toward the commanded value rather than jumping to it, so a status read taken immediately after commanding it can catch it mid-transit (observed: commanding ratio 1.5 read back partway there before settling). obsbot_zoom_to_fit polls for up to 3 seconds waiting for the zoom to arrive and returns settled:false — not an error — if it didn't. A frame captured while the zoom is still moving is at an unknown magnification, so check settled before trusting a follow-up snapshot; a false just means the camera was moving slower than expected, not that anything failed.

Gimbal presets

Three on-device preset slots (1–3). Slots are create-once: obsbot_preset_save requires an empty slot (delete first to reuse one); every other preset tool requires the slot to already be occupied. Each tool re-reads the slot list after writing and returns a structured { ok:false } failure if the device didn't land the change.

Zoom

Two tools, not one — they ride different transports (standard UVC vs. the vendor command frame) and produce different physical zoom at the same commanded ratio, so merging them would silently change what ratio means. Pick by which behaviour you need.

AI tracking

Image & lens

Focus, white balance, and exposure each split into a dedicated _auto and _manual tool in v0.4.0 (previously one tool with a mode parameter) — auto and manual take different parameters, so splitting them lets each schema say exactly what it needs.

Capture

obsbot_capture_record and obsbot_capture_preview do not take camera. They select a device by source (device/virtual/ndi) through ffmpeg/ffplay, not by serial — there is no serial-to-ffmpeg-device mapping yet. obsbot_capture_snapshot honors camera only for source:"device"; for source:"virtual"/"ndi" the pixel source is still resolved by device name, independent of camera.

Diagnostics (--debug only)

¹ record/preview shell out to ffmpeg/ffplay (install: winget install Gyan.FFmpeg on Windows, brew install ffmpeg on macOS, apt install ffmpeg on Linux). snapshot does not need ffmpeg — it grabs the frame through the native helper.

Supported platforms

• Windows x64 — supported today. The native helper is built from source in native/windows/ (CMake + MSVC); the published npm package ships a prebuilt binary so end users need no toolchain.

• Linux x64 — supported from v0.2. The native helper is in native/linux/ (CMake + GCC); it uses V4L2 for standard UVC controls (zoom, focus, exposure, pan/tilt, white balance, image controls) and UVCIOC_CTRL_QUERY for vendor Extension Unit commands (gimbal speed/AI tracking, wake/sleep, HDR, FOV). Snapshots capture a MJPEG or YUYV frame via V4L2 mmap streaming and encode to JPEG using libjpeg. The linux-x64 prebuilt binary ships with the published npm package. Build dependencies: build-essential cmake libjpeg-dev libv4l-dev.

  Gimbal position reads (obsbot_gimbal_position) reflect the last-commanded value, not live in-flight position — see "Linux gimbal position feedback" below for why, and what would fix it.

• macOS 14+ (Apple Silicon and Intel) — supported. The native helper is in native/macos/ (Objective-C + IOKit/AVFoundation). It uses IOKit USB control transfers for both standard UVC controls and vendor Extension Unit commands, and AVFoundation for enumeration and snapshots. Both darwin-arm64 and darwin-x64 prebuilt binaries ship with the published npm package (darwin-x64 also covers Apple Silicon running Node under Rosetta, where process.arch reports x64). macOS 14 is the floor because the helper uses AVCaptureDeviceTypeExternal; the build pins -mmacosx-version-min so the binary does not inherit the build machine's OS as its minimum.

  Note on macOS specifically: UVCAssistant (a DriverKit system extension) owns the camera's UVC interfaces exclusively, so USBInterfaceOpen — and even USBInterfaceOpenSeize — fail with kIOReturnExclusiveAccess. The helper therefore opens the USB device, which is not locked, and issues UVC control requests on its default control endpoint. This coexists with UVCAssistant: the camera keeps working as a normal webcam while under control, so no driver-replacement step is needed.

Building the native helper (Linux)

cd native/linux
mkdir build && cd build
cmake ..
make -j$(nproc)
make install  # copies to native/prebuilt/linux-x64/

Linux gimbal position feedback is not live

obsbot_gimbal_position on Linux reports the last position obsbot_gimbal_move/ obsbot_gimbal_recenter commanded — not a live, in-flight reading. Hardware testing (2026-07-21) confirmed the OBSBOT Tiny 2's CT_PANTILT_ABSOLUTE control genuinely tracks live position — a raw USB read of that same control, bypassing the kernel, showed a real slew progressing in real time. The reason plain V4L2 (VIDIOC_G_CTRL) never sees that is that uvcvideo caches the control's value and serves the cache instead of re-querying the device (confirmed via VIDIOC_QUERY_EXT_CTRL, which reports no V4L2_CTRL_FLAG_VOLATILE). The driver invalidates that cache when the device sends a UVC Control Change interrupt — which this camera's firmware never does, and never advertises support for.

Getting a genuinely live reading through V4L2 requires briefly detaching the kernel driver from the camera's control interface and reading the control directly over raw USB — but detaching that interface (even briefly, even without writing anything) breaks any concurrent video capture on this device: streaming and control share one kernel-managed USB function, so pulling the driver off one takes both down together. That makes a libusb-based workaround incompatible with anything actually using the camera as a webcam at the same time, which ruled it out as a shipped default.

A kernel patch has been submitted upstream (media: uvcvideo: query pan/tilt position from the device on every read, July 2026 — awaiting review, not merged). It marks CT_PANTILT_ABSOLUTE volatile so the driver queries the device on every read; verified on this hardware to track a live slew through plain VIDIOC_G_CTRL, concurrently with streaming. If it is accepted, obsbot_gimbal_position becomes live on Linux with no code changes needed here. Until it ships in a kernel near you:

• obsbot_gimbal_move and obsbot_gimbal_recenter work normally — hardware-verified, repeatedly, via direct V4L2 VIDIOC_S_CTRL writes. Their target values are known and clamped before being sent, so they can't exceed the gimbal's mechanical range regardless of the missing feedback.
• obsbot_gimbal_move_speed is not available on Linux (hidden from the tool list entirely, not just refused at runtime). A speed×duration burst has no target position to clamp — without a live reading to confirm where the gimbal actually is, there's no way to bound it against the mechanical limits before it gets there. It remains available on Windows/macOS.

Building the native helper (macOS)

make -C native/macos    # -> native/prebuilt/darwin-arm64/obsbot-helper

Requires the Xcode command line tools. CMake works too (cmake -S native/macos -B native/macos/build && cmake --build native/macos/build), which is what CI uses.

Known limitations

What has actually been exercised against hardware, and what hasn't:

• The Intel (darwin-x64) helper has never been run. No Intel Mac was available to test it. It cross-compiles cleanly and is packaged, but nothing has confirmed it talks to a camera. It also covers Apple Silicon running Node under Rosetta, where process.arch reports x64 — likewise untested. Reports from Intel users are welcome.

• macOS 14 or newer is required, and macOS runtime is verified on 26.5 only. The helper uses AVCaptureDeviceTypeExternal (macOS 14+), so the build pins -mmacosx-version-min=14.0. The binary will load on 14 through 25, but behavior there is untested — in particular the UVC control path relies on UVCAssistant holding the camera's UVC interfaces while leaving the USB device itself openable. That is how current macOS behaves; older releases are unconfirmed.

• The first snapshot on macOS raises a camera permission prompt. The helper is a plain CLI tool with no bundle identifier, so macOS attributes camera access to whichever app spawned it — your MCP client — and that app is named in the prompt and holds the grant. Approve once; the grant survives helper updates, since it is keyed to the client rather than to the helper's signature.

• AI tracking overrides manual gimbal moves. When AI tracking is active (the Tiny 2's default on wake), a commanded pan/tilt executes and is then pulled back to the tracked subject — obsbot_gimbal_position shows the yaw/pitch move out and decay back to rest. This is the camera's behaviour, not a bug: turn tracking off for unopposed manual control.

• The camera may not enumerate through a USB hub or dock. A Tiny 2 connected through a USB-C dock was invisible to ioreg and system_profiler entirely — not just to this server. If obsbot_devices comes back empty, try a direct connection before assuming a software fault.

• Only the OBSBOT Tiny 2 is supported. On Windows and macOS candidacy is gated on the Remo USB vendor ID plus a known-model product ID (0x3564/0xFEF8), so no other model is detected at all — and a name-matching software source, such as the "OBSBOT Virtual Camera" that OBSBOT Center registers, is rejected because it reports no vid/pid. Linux still matches by name, because its helper does not report vid/pid yet, so a different OBSBOT may be found there — but the vendor command set is Tiny 2 specific either way. (On macOS the virtual camera cannot appear at all: the helper enumerates USB devices through the IORegistry, which a software camera never enters.)

• The vendor reply mailbox is unreliable for several seconds after a replug. On 2026-07-21 a Tiny 2 was seen returning only the host's own echoed request frame from the vendor reply mailbox (XU selector 2) — magic byte 0xaa cleared to 0x00, every other byte identical — for a continuous 3.2 s. readSerial() threw, bind() found no serial, and every tool needing a bound camera failed with "no OBSBOT camera found" while the device was plainly healthy: correct vid/pid, opened fine, XU node 2, live status block on selector 6.

  That was unexplained for a while. It is now reproducible: immediately after a USB re-enumeration. Polling readSerial every 50 ms across a replug failed 22 times in 80 attempts spread over the first 14 s, against 0 in 120 in steady state; the first read after arrival showed exactly the echoed-request signature above, and later failures showed the reply slot populated but with its magic byte still zeroed. Ruled out as causes: stale per-process device state (the same long-lived helper read a brand-new uniqueID cleanly at t+49 ms), re-opening the device (0/40 either way), and the per-transport sequence counter restarting at 1 (0/80).

  Consequence for callers: a bind attempted in the first seconds after a replug can fail even though the camera is fine. Retrying works. The arrival-driven re-bind now retries on a bounded ladder for this reason, and a readSerial failure reports what the mailbox actually held (echoed request / unparseable / a reply to another request) rather than only "no valid reply". Ruled out earlier and still ruled out: reply latency (polled 3.2 s), the wrong extension unit (the VideoControl interface exposes exactly one, bUnitID 2), the wrong wLength (every XU selector is 60 bytes by GET_LEN), the reply arriving on another selector (1–19 swept), camera sleep state, and contention from OBSBOT Center.

• Recovery after a replug is proactive, but not in every case, and it differs by platform. The server subscribes to OS camera arrival/removal events, so in the common case a replugged camera re-binds itself with no tool call — obsbot_devices reports it bound again on its own. Every cell below is hardware-measured:

  scenario	macOS	Windows	Linux
  same-port replug	proactive	proactive	next tool call
  different-port replug	proactive	next tool call	next tool call

  Where it says "next tool call", nothing is stranded — the call that follows detects the stale binding, prunes it and re-binds. It costs one failed call, which is exactly how every platform behaved before these events existed. The Windows difference comes from its arrival filter requiring a path it has already enumerated, which is also what stops the Tiny 2's audio interface from being reported as a second camera; macOS has no equivalent problem because it re-binds by serial and ignores the path. Linux emits no bus events at all yet.

  Note the interaction with the mailbox entry above: a re-bind attempted immediately after a replug can still lose the first attempt, so the server retries on a short bounded ladder.

• Two-camera operation is not yet hardware-verified. The camera selector and the per-camera device registry are covered by the unit test suite against fake transports; running two physical Tiny 2s attached at once has not been confirmed on real hardware (a second unit wasn't available). Single-camera use is unaffected either way.

• Linux gimbal position feedback is not live, and obsbot_gimbal_move_speed is unavailable there as a result. See "Linux gimbal position feedback is not live" above — a kernel patch fixing this at the source has been submitted upstream (July 2026, awaiting review). obsbot_gimbal_move and obsbot_gimbal_recenter are unaffected; both are hardware-verified to work normally. obsbot_aim_at_pixel is affected — it depends on a live pose reading to compute the aim, the same way obsbot_gimbal_position does.

• obsbot_zoom_vendor's ratio scale doesn't match obsbot_zoom_uvc's at the same ratio. A hardware snapshot comparison at ratio: 2.0 showed the vendor path framed tighter than the UVC path. Whether the vendor-side ratio encoding is off by a scale factor, or the two zoom controls simply have different physical ranges, isn't determined yet — one comparison isn't enough to tell. Tracked separately; use obsbot_zoom_uvc if you need the ratio to land exactly.

No proprietary SDK

This project speaks the camera's USB protocol directly through the OS's standard UVC driver stack and does not use, link, bundle, or ship any vendor SDK. See PROTOCOL.md for the protocol reference (frame format, checksum, command table).

How it works

The camera exposes two independent control surfaces, both reachable through the OS's standard UVC (USB Video Class) driver stack — this project never talks to the USB device directly, so the OS keeps mediating access and the camera remains usable as a normal webcam at the same time commands are sent:

• Standard UVC controls — zoom (CT_ZOOM_ABSOLUTE), focus and exposure (IAMCameraControl), gimbal position readback (UVC Pan/Tilt), and the image controls plus white balance (IAMVideoProcAmp) — are the camera's built-in UVC properties, driven via DirectShow on Windows.
• Vendor commands — gimbal moves, recenter, wake/sleep, AI tracking, HDR, and field of view — are sent through the camera's UVC Extension Unit, driven via IKsControl::KsProperty against the XU's topology node on Windows.

Both are issued through a small native helper process (obsbot-helper.exe on Windows, obsbot-helper on Linux) that the Node server spawns and talks to over a line-delimited JSON-RPC protocol on stdin/stdout. The helper is the only platform-specific piece; the codec (frame encoding, CRC-16/USB checksum, command table), transport abstraction, device manager, and MCP tool definitions are all pure TypeScript/JavaScript and shared across platforms.

Verifying against real hardware

scripts/e2e.mjs drives the built stack (dist/) against a physically connected camera: it wakes the device, zooms in, pans the gimbal, recenters, zooms back out, and puts the camera to sleep, with a short pause and console log before each step so a human can watch it happen. This moves the physical gimbal — only run it under supervision:

npm run build
node scripts/e2e.mjs

Testing changes through the live MCP tools

Two traps make it easy to test the wrong thing and believe the result. Both cost real time on 2026-07-25.

Rebuilding and reloading is not enough — kill stale server processes first. The MCP server runs from dist/, so a source change is invisible until npm run build. But reloading the server in your MCP client does not guarantee your new code executes: this project coordinates concurrent clients by electing a single owner process (see IPC-DESIGN.md), and a reload spawns a new client that forwards its tool calls to whatever owner is already running. An orphaned server from a previous session stays the owner, so the new process advertises its own up-to-date tool list while every call is executed by old code.

That failure is deceptive rather than loud: a newly added tool appears in the tool list and can be called, but behaves like the old build. Two reloads in a row will not fix it. Check for orphans before concluding anything:

# Windows
Get-CimInstance Win32_Process -Filter "Name='node.exe'" |
  Where-Object { $_.CommandLine -like "*Obsbot*" } |
  Select-Object ProcessId, CreationDate

# Linux / macOS
pgrep -af "obsbot.*dist/index.js"

Kill everything older than your build, then reload. The cheapest positive confirmation is to call a tool whose output changed — obsbot_status gaining a field, say — rather than one whose description changed, since descriptions come from the new process either way.

Frame rate selects the field of view, so the preview shows less than snapshots. At 1920×1080 this camera has two different windows onto the sensor and the frame rate picks between them — not the codec. Measured at one pose and one zoom: MJPEG@30 vs YUYV@30 came out at scale 1.00001 over 2382 inliers, i.e. the same field to within a fifth of a pixel, while MJPEG@60 is a 1.214× crop of both. obsbot_capture_preview pins 60fps for smooth motion, so it shows ~21% less of the room than obsbot_capture_snapshot, which negotiates 1080p30. Framing by eye in the preview and then aiming at a pixel from a snapshot will not agree. The geometry constants describe the 30fps field. Any measurement you make must state pixel format and frame rate; neither a resolution nor a codec alone identifies the field.

A preview holds the camera stream, so snapshots fail while one is open. obsbot_capture_preview and obsbot_capture_snapshot both need the device stream, and on Windows the second one gets Camera is in use by another application. This matters for the aim loop above, which is snapshot → aim → snapshot: stop the preview around each snapshot, or work without one. Gimbal control is unaffected — it uses control transfers, not the stream — so obsbot_aim_at_pixel itself works fine with a preview running. The error text suggests source: "virtual" or "ndi" as a workaround. That is safe for looking, and safe for aiming only if the feed is an unmodified pass-through of the camera — declare it with the source parameter on obsbot_aim_at_pixel / obsbot_zoom_to_fit and read the note it returns. A compositor that rescales or letterboxes the frame silently invalidates the geometry.