io.github.AdonaiVera/fiftyone-mcp-server

Official

by voxel51 · Python

Control FiftyOne computer vision datasets through AI assistants using 80+ operators.

Manages FiftyOne computer vision datasets with 80+ operators for AI assistants.

FiftyOne MCP Server

<div align="center"> <p align="center">

</p>

Control FiftyOne datasets through AI assistants using the Model Context Protocol

Documentation · FiftyOne Skills · FiftyOne Plugins · Discord

</div>

What is the FiftyOne MCP Server?

Enable Agents to explore datasets, execute operators, and control the FiftyOne App through natural language. This server exposes 45+ MCP tools across data operations, App UI control, and the full operator/plugin ecosystem.

code

"List all my datasets"
"Load quickstart dataset and show summary"
"Find similar images in my dataset"

The server starts with 50 built-in operators. Install plugins to expand functionality - the AI can discover and install plugins automatically when needed (brain, zoo, annotation, evaluation, and more).

Available Tools

Category	Tools	Description
📊 Dataset Management	3	List, load, and summarize datasets
🎯 App Operations	29	Control the App UI (views, panels, selection, ...)
⚡ Operator System	3	Discover and execute any FiftyOne operator
🔄 Pipelines	2	Run pipelines and manage delegated operations
🔌 Plugin Management	5	Discover, install, and manage plugins
🖥️ Session	1	Launch the FiftyOne App server
📈 Aggregations	8	Count, distinct, bounds, mean, histogram, ...
🧬 Samples	5	Add, tag, untag, and set values on samples
🗂️ Schema	2	Inspect and modify dataset field schemas
🎨 App Config	6	Color scheme, sidebar groups, active fields

Tool modes

45+ tools organized by runtime mode:

SDK: Data operations that work everywhere (datasets, aggregations, schema, samples, operators, plugins). No App connection needed.
APP: Controls the FiftyOne App UI in real time (set_view, open_panel, notify, select_samples, reload, and 25+ more). Requires a connected browser via ctx.ops.
SESSION: Bootstrap tools for starting a local App server (launch_app). Used from terminal environments.

Choosing your tools

Which tools are available depends on how you integrate the server:

Integration	Modes	Use case
FiftyOne App plugin	`app` + `sdk`	Agent panel inside the App (full UI control + data operations)
Terminal / CLI	`session` + `sdk`	Headless agent (launch the App, query data, execute operators)

Tool risk levels

Every tool is tagged with a risk level that your agent can use for auto-approval decisions:

LOW Safe to auto-execute without prompting (read-only queries, UI state changes)
OPERATOR Wraps a FiftyOne operator whose own severity should be checked before executing

Quick Start

Step 1: Install the MCP Server

bash

pip install fiftyone-mcp-server

⚠️ Important: Make sure to use the same Python environment where you installed the MCP server when configuring your AI tool. If you installed it in a virtual environment or conda environment, you must activate that environment or specify the full path to the executable.

Step 2: Configure Your AI Tool

<details> <summary><b>Claude Code</b> (Recommended)</summary>

bash

claude mcp add fiftyone -- fiftyone-mcp

</details> <details> <summary><b>Claude Desktop</b></summary>

Edit ~/Library/Application Support/Claude/claude_desktop_config.json:

json

{
  "mcpServers": {
    "fiftyone": {
      "command": "fiftyone-mcp"
    }
  }
}

</details> <details> <summary><b>Cursor</b></summary>

Add to ~/.cursor/mcp.json:

json

{
  "mcpServers": {
    "fiftyone": {
      "command": "fiftyone-mcp"
    }
  }
}

</details> <details> <summary><b>VSCode</b></summary>

Add to .vscode/mcp.json:

json

{
  "servers": {
    "fiftyone": {
      "command": "fiftyone-mcp"
    }
  }
}

</details> <details> <summary><b>ChatGPT Desktop</b></summary>

Edit ~/Library/Application Support/ChatGPT/config.json:

json

{
  "mcpServers": {
    "fiftyone": {
      "command": "fiftyone-mcp"
    }
  }
}

</details> <details> <summary><b>uvx (No Install Needed)</b></summary>

If you have uv installed:

json

{
  "mcpServers": {
    "fiftyone": {
      "command": "uvx",
      "args": ["fiftyone-mcp-server"]
    }
  }
}

This downloads and runs the latest version automatically.

</details>

Step 3: Use It

code

"List all my datasets"
"Load quickstart dataset and show summary"
"Open the map panel and show me the embeddings"
"Select samples with confidence above 0.9"
"What plugins are available? Install the brain plugin"
"Find near-duplicate images in my dataset"

Claude will automatically discover operators and execute the appropriate tools.

Contributing

We welcome contributions! Here's how to set up a local development environment:

Clone the repository

bash

git clone https://github.com/voxel51/fiftyone-mcp-server.git
cd fiftyone-mcp-server

Install dependencies
bash
```
poetry install
```
Run the server locally
bash
```
poetry run fiftyone-mcp
```

Test your changes

bash

poetry run pytest
poetry run black -l 79 src/
npx @modelcontextprotocol/inspector poetry run fiftyone-mcp

Submit a Pull Request

Resources

Resource	Description
FiftyOne Docs	Official documentation
FiftyOne Skills	Expert workflows for AI assistants
FiftyOne Plugins	Official plugin collection
Model Context Protocol	MCP specification
PyPI Package	MCP server on PyPI
Discord Community	Get help and share ideas

Community

Join the FiftyOne community to get help, share your ideas, and connect with other users:

Discord: FiftyOne Community
GitHub Issues: Report bugs or request features

</div>

Install

claude_desktop_config.json

{
  "mcpServers": {
    "fiftyone-mcp-server": {
      "command": "uvx",
      "args": [
        "fiftyone-mcp-server"
      ]
    }
  }
}