io.github.rsmdt/multimodal

Official

by rsmdt · TypeScript

Multi-provider media generation — images, video, audio, and transcription via a unified interface

Multi-provider media generation — images, video, audio, and transcription via a unified interface.

This MCP server aggregates capabilities from OpenAI, xAI, Gemini, ElevenLabs, and BFL (FLUX) to generate and edit media from text prompts, offering a single unified access point for multimodal content.

🛠️ Key Features

🎨 Image Generation via multiple providers (OpenAI, xAI, Gemini, BFL)
✏️ Image Editing across supported vendors
🎬 Video Generation from prompts (OpenAI, xAI, Gemini)
🔊 Audio Generation and transcription via supported systems
Unified interface for multi-provider media workflows

🚀 Use Cases

Create images, videos, and audio from text prompts
Edit existing media with provider-agnostic tooling
Transcribe media across formats
Integrate multimodal generation into apps with a single API

⚡ Developer Benefits

Single MCP endpoint for multimodal output
Pluggable provider backend support
Consistent interface across image, video, and audio tasks
Clear readme guidance for integration

⚠️ Limitations

Source mentions multiple providers; exact feature parity per provider may vary
Readme excerpt only; confirm current provider support and quotas in runtime
Potential latency depending on provider mix and media type

README.md11 KBView on GitHub

multimodal-mcp

Multi-provider media generation MCP server. Generate images, videos, audio, and transcriptions from text prompts using OpenAI, xAI, Gemini, ElevenLabs, and BFL (FLUX) through a single unified interface.

Features

🎨 Image Generation — Generate images via OpenAI (gpt-image-1), xAI (grok-imagine-image), Gemini (imagen-4), or BFL (FLUX Pro 1.1)
✏️ Image Editing — Edit images via OpenAI, xAI, Gemini, or BFL (FLUX Kontext)
🎬 Video Generation — Generate videos via OpenAI (sora-2), xAI (grok-imagine-video), or Gemini (veo-3.1)
🔊 Audio Generation — Text-to-speech via OpenAI (tts-1), Gemini, or ElevenLabs (Flash v2.5). Sound effects via ElevenLabs
🎙️ Audio Transcription — Speech-to-text via OpenAI (Whisper) or ElevenLabs (Scribe)
🔄 Auto-Discovery — Automatically detects configured providers from environment variables
🎯 Provider Selection — Auto-selects or explicitly choose a provider per request
📁 File Output — Saves all generated media to disk with descriptive filenames

Quick Start

Set the API key for at least one provider. Most users only need one — add more to access additional providers.

bash

# Using OpenAI
claude mcp add multimodal-mcp -e OPENAI_API_KEY=sk-... -- npx -y @r16t/multimodal-mcp@latest

# Or using xAI
# claude mcp add multimodal-mcp -e XAI_API_KEY=xai-... -- npx -y @r16t/multimodal-mcp@latest

# Or using Gemini
# claude mcp add multimodal-mcp -e GEMINI_API_KEY=AIza... -- npx -y @r16t/multimodal-mcp@latest

# Or using ElevenLabs (audio + transcription)
# claude mcp add multimodal-mcp -e ELEVENLABS_API_KEY=xi-... -- npx -y @r16t/multimodal-mcp@latest

# Or using BFL/FLUX (images)
# claude mcp add multimodal-mcp -e BFL_API_KEY=... -- npx -y @r16t/multimodal-mcp@latest

Using a different editor? See setup instructions for Claude Desktop, Cursor, VS Code, Windsurf, and Cline.

Environment Variables

Variable	Required	Description
`OPENAI_API_KEY`	At least one provider key	OpenAI API key — enables image, video, audio generation, and transcription via gpt-image-1, sora-2, tts-1, and whisper-1
`XAI_API_KEY`	At least one provider key	xAI API key — enables image and video generation via grok-imagine-image and grok-imagine-video
`GEMINI_API_KEY`	At least one provider key	Gemini API key — enables image, video, and audio generation via imagen-4, veo-3.1, and gemini-2.5-flash-preview-tts
`GOOGLE_API_KEY`	—	Alias for `GEMINI_API_KEY`; either name is accepted
`ELEVENLABS_API_KEY`	At least one provider key	ElevenLabs API key — enables audio generation (TTS, sound effects) and transcription via Flash v2.5 and Scribe v1
`BFL_API_KEY`	At least one provider key	BFL API key — enables image generation and editing via FLUX Pro 1.1 and FLUX Kontext
`MEDIA_OUTPUT_DIR`	No	Directory for saved media files. Defaults to the current working directory

Available Tools

`generate_image`

Generate an image from a text prompt.

Parameter	Type	Required	Description
`prompt`	string	Yes	Text description of the image to generate
`provider`	string	No	Provider to use: `openai`, `xai`, `google`, `bfl`. Auto-selects if omitted
`aspectRatio`	string	No	Aspect ratio: `1:1`, `16:9`, `9:16`, `4:3`, `3:4`
`quality`	string	No	Quality level: `low`, `standard`, `high`
`outputDirectory`	string	No	Directory to save the generated file. Absolute or relative path. Defaults to `MEDIA_OUTPUT_DIR` or cwd
`providerOptions`	object	No	Provider-specific parameters passed through directly

`generate_video`

Generate a video from a text prompt. Video generation is asynchronous and may take several minutes.

Parameter	Type	Required	Description
`prompt`	string	Yes	Text description of the video to generate
`provider`	string	No	Provider to use: `openai`, `xai`, `google`. Auto-selects if omitted
`duration`	number	No	Video duration in seconds (provider limits apply)
`aspectRatio`	string	No	Aspect ratio: `16:9`, `9:16`, `1:1`
`resolution`	string	No	Resolution: `480p`, `720p`, `1080p`
`outputDirectory`	string	No	Directory to save the generated file. Absolute or relative path. Defaults to `MEDIA_OUTPUT_DIR` or cwd
`providerOptions`	object	No	Provider-specific parameters passed through directly

`generate_audio`

Generate audio from text. Supports text-to-speech and sound effects. Audio generation is synchronous.

Parameter	Type	Required	Description
`text`	string	Yes	Text to convert to speech, or a description of the sound effect to generate
`provider`	string	No	Provider to use: `openai`, `google`, `elevenlabs`. Auto-selects if omitted
`voice`	string	No	Voice name (provider-specific). OpenAI: `alloy`, `ash`, `coral`, `echo`, `fable`, `nova`, `onyx`, `sage`, `shimmer`. Google: `Kore`, `Charon`, `Fenrir`, `Aoede`, `Puck`, etc. ElevenLabs: voice ID
`speed`	number	No	Speech speed multiplier (OpenAI only): `0.25` to `4.0`
`format`	string	No	Output format (OpenAI only): `mp3`, `opus`, `aac`, `flac`, `wav`, `pcm`
`outputDirectory`	string	No	Directory to save the generated file. Absolute or relative path. Defaults to `MEDIA_OUTPUT_DIR` or cwd
`providerOptions`	object	No	Provider-specific parameters passed through directly. ElevenLabs: set `mode: "sound-effect"` for sound effects, `model` for TTS model selection

`transcribe_audio`

Transcribe audio to text (speech-to-text).

Parameter	Type	Required	Description
`audioPath`	string	Yes	Absolute path to the audio file to transcribe
`provider`	string	No	Provider to use: `openai`, `elevenlabs`. Auto-selects if omitted
`language`	string	No	Language code (e.g., `en`, `fr`, `es`) to hint the transcription language
`providerOptions`	object	No	Provider-specific parameters passed through directly

`list_providers`

List all configured media generation providers and their capabilities. Takes no parameters.

Provider Capabilities

Provider	Image	Image Editing	Video	Audio	Transcription	Key Models
OpenAI	✅	✅	✅	✅	✅	gpt-image-1, sora-2, tts-1, whisper-1
xAI	✅	✅	✅	—	—	grok-imagine-image, grok-imagine-video
Gemini	✅	✅	✅	✅	—	imagen-4, veo-3.1, gemini-2.5-flash-preview-tts
ElevenLabs	—	—	—	✅	✅	eleven_flash_v2_5, scribe_v1
BFL	✅	✅	—	—	—	flux-pro-1.1, flux-kontext-pro

Image Aspect Ratios

Provider	1:1	16:9	9:16	4:3	3:4
OpenAI	✅	✅	✅	✅	✅
xAI	✅	✅	✅	✅	✅
Gemini	✅	✅	✅	✅	✅
BFL	✅	✅	✅	✅	✅

Video Aspect Ratios & Resolutions

Provider	16:9	9:16	1:1	480p	720p	1080p
OpenAI	✅	✅	✅	✅	✅	✅
xAI	✅	✅	✅	—	✅	✅
Gemini	✅	✅	—	—	✅	✅

Audio Formats

Provider	mp3	opus	aac	flac	wav	pcm
OpenAI	✅	✅	✅	✅	✅	✅
Gemini	—	—	—	—	✅	—
ElevenLabs	✅	✅	—	—	—	✅

Troubleshooting

No providers configured

code

[config] No provider API keys detected

Set at least one of OPENAI_API_KEY, XAI_API_KEY, GEMINI_API_KEY, ELEVENLABS_API_KEY, or BFL_API_KEY in the MCP server's env block.

Provider not available for requested media type

Each provider supports different media types (see Provider Capabilities). If you specify a provider that isn't configured (no API key) or doesn't support the requested media type, you'll receive an error. Omit the provider parameter to auto-select from configured providers.

Video generation timeout

Video generation polls for up to 10 minutes. If your video hasn't completed in that window, the request will fail with a timeout error. Try a shorter duration or a simpler prompt.

xAI image generation returned no data

This indicates the xAI API returned an empty response. Check that your XAI_API_KEY is valid and that your prompt does not violate xAI content policies.

Gemini image/video generation failed: 403

Verify your GEMINI_API_KEY has the Generative Language API enabled in Google Cloud Console.

Development

bash

npm run build      # Compile TypeScript to build/
npm test           # Run tests with Vitest
npm run lint       # Lint and auto-fix with ESLint
npm run typecheck  # Type-check without emitting
npm run dev        # Watch mode for TypeScript compilation

Editor Setup

Replace OPENAI_API_KEY with your provider of choice (XAI_API_KEY, GEMINI_API_KEY, ELEVENLABS_API_KEY, BFL_API_KEY). You can set multiple keys to enable multiple providers.

Claude Desktop

Add to ~/Library/Application Support/Claude/claude_desktop_config.json:

json

{
  "mcpServers": {
    "multimodal-mcp": {
      "command": "npx",
      "args": ["@r16t/multimodal-mcp@latest"],
      "env": {
        "OPENAI_API_KEY": "sk-..."
      }
    }
  }
}

Cursor

Add to .cursor/mcp.json in your project root (or ~/.cursor/mcp.json globally):

json

{
  "mcpServers": {
    "multimodal-mcp": {
      "command": "npx",
      "args": ["@r16t/multimodal-mcp@latest"],
      "env": {
        "OPENAI_API_KEY": "sk-..."
      }
    }
  }
}

VS Code (GitHub Copilot)

Add to .vscode/mcp.json in your project root:

json

{
  "servers": {
    "multimodal-mcp": {
      "command": "npx",
      "args": ["@r16t/multimodal-mcp@latest"],
      "env": {
        "OPENAI_API_KEY": "sk-..."
      }
    }
  }
}

Windsurf

Add to ~/.codeium/windsurf/mcp_config.json:

json

{
  "mcpServers": {
    "multimodal-mcp": {
      "command": "npx",
      "args": ["@r16t/multimodal-mcp@latest"],
      "env": {
        "OPENAI_API_KEY": "sk-..."
      }
    }
  }
}

Cline

Add to ~/Library/Application Support/Code/User/globalStorage/saoudrizwan.claude-dev/settings/cline_mcp_settings.json:

json

{
  "mcpServers": {
    "multimodal-mcp": {
      "command": "npx",
      "args": ["@r16t/multimodal-mcp@latest"],
      "env": {
        "OPENAI_API_KEY": "sk-..."
      }
    }
  }
}

License

MIT

Install

Configuration

Environment variables

OPENAI_API_KEYsecret

OpenAI API key for image, video, audio generation and transcription

XAI_API_KEYsecret

xAI API key for image and video generation

GEMINI_API_KEYsecret

Google Gemini API key for image, video, and audio generation

ELEVENLABS_API_KEYsecret

ElevenLabs API key for audio generation and transcription

BFL_API_KEYsecret

BFL API key for FLUX image generation and editing

MEDIA_OUTPUT_DIR

Directory for saved media files (defaults to cwd)

Add to VS Code Add to Cursor

claude_desktop_config.json

json

{
  "mcpServers": {
    "multimodal": {
      "command": "npx",
      "args": [
        "-y",
        "@r16t/multimodal-mcp@1.3.1"
      ],
      "env": {
        "OPENAI_API_KEY": "<YOUR_OPENAI_API_KEY>",
        "XAI_API_KEY": "<YOUR_XAI_API_KEY>",
        "GEMINI_API_KEY": "<YOUR_GEMINI_API_KEY>",
        "ELEVENLABS_API_KEY": "<YOUR_ELEVENLABS_API_KEY>",
        "BFL_API_KEY": "<YOUR_BFL_API_KEY>",
        "MEDIA_OUTPUT_DIR": "<YOUR_MEDIA_OUTPUT_DIR>"
      }
    }
  }
}

🛠️ Key Features

🚀 Use Cases

⚡ Developer Benefits

⚠️ Limitations

multimodal-mcp

Features

Quick Start

Environment Variables

Available Tools

generate_image

generate_video

generate_audio

transcribe_audio

list_providers

Provider Capabilities

Image Aspect Ratios

Video Aspect Ratios & Resolutions

Audio Formats

Troubleshooting

No providers configured

Provider not available for requested media type

Video generation timeout

xAI image generation returned no data

Gemini image/video generation failed: 403

Development

Editor Setup

Claude Desktop

Cursor

VS Code (GitHub Copilot)

Windsurf

Cline

License

Install

Configuration

Related servers

`generate_image`

`generate_video`

`generate_audio`

`transcribe_audio`

`list_providers`