This MCP server analyzes a video’s mood and pace, then recommends license-safe background music (BGM) using a specified mix spec. The repository is described as focused on pairing video audio characteristics with music selection that accounts for licensing constraints.
🛠️ Key Features
Analyze video mood
Analyze video pace
Recommend license-safe BGM
Use a mix specification during recommendations
🚀 Use Cases
Selecting background music for video content based on mood and pace
Generating BGM recommendations that aim to be license-safe
Applying an explicit mix spec to music selection
⚡ Developer Benefits
Inputs centered on video mood and pace
Output tied to license-safe BGM recommendations
Recommendation workflow supports a mix spec parameterization
⚠️ Limitations
Server description provides only high-level behavior (no tool list, capabilities, or configuration details).
Video-native MCP server for license-safe BGM. Watches the footage — not the script — and returns a shortlist, a 12–20s hook, and an ffmpeg ducking spec.
Sonicmatch — video in, license-safe BGM out. Ingest, Analyze, Match, Mix.
Package / CLI: sonicmatch-mcp. A Model Context Protocol server for Claude Desktop, Cursor, Codex, and other MCP clients. Drop an Instagram Reel, YouTube Short, or TikTok-style clip. Get royalty-free / Creative Commons matches with the license printed on every row.
Video-to-BGM already exists. The wedge is not “I also match music”:
it watches the footage, not the script
it returns a hook window + ffmpeg ducking spec
it is agent-native
it prints the license instead of lying
Catalog quality will kill or save this. More tools will not.
code
Video or URL in
→ scene / mood / pace / speech analysis
→ license-safe BGM shortlist
+ beat/cut hints
+ optional mix preview
Do not treat this as “script in → YouTube Music search out.” That already exists (mcp-bgm-recommender). Sonicmatch watches the video.
You own
You do not own
Local file / public URL ingest
Platform music licenses
Mood, energy curve, speech vs silence, scene cuts
Meta/TikTok “trending audio” graph
CC / royalty-free catalogs + optional paid adapters
Spotify / IG official libraries
Ranked tracks, preview URLs, mix spec, ffmpeg
Auto-publish to Instagram
North star:ingest_video → analyze_video_music → recommend_bgm → preview_mix → export_mix_spec
License warning (read this)
The code is MIT.
Every track has its own license. It is printed on every recommendation.
Nothing here is an official Instagram sticker, TikTok Commercial Music Library track, or YouTube Audio Library API result.
Do not recommend commercial pop unless the adapter is explicitly a user-owned licensed library.
CC-BY still needs attribution. CC-BY-NC is not ok for ads / shops. Non-commercial tracks are never auto-recommended.
For ads / shops, wire a user-owned Artlist / Epidemic JSON (examples/user_library.example.json). Do not scrape those sites.
Content ID can still hit you if you point at the wrong source. A CC label is not a waiver.
Demo
I dropped ./clip.mp4. Analyze it for an Instagram Reel and recommend 5 instrumental BGMs. Then mix the top pick with ducking and give me the ffmpeg command.
That prompt is the product. Install below, wire Claude Desktop, Cursor, or Codex, and paste it.
Quick start
Requires Python 3.10+ and ffmpeg / ffprobe on PATH. yt-dlp is optional and off by default (SONICMATCH_ALLOW_YTDLP=0) because platform extractors break and may violate ToS. Prefer a local file.
The PyPI project is sonicmatch-mcp. The console script is sonicmatch-mcp, which is what uvx and pipx run. python -m sonicmatch is the same server. The PyPI badge stays empty until the first tag is published. Install from git until then. Maintainer steps: PUBLISH.md.
# same clone, uv
uv venv && uv pip install -e ".[dev]"
uv run sonicmatch-mcp
bash
pytest # generates tiny color mp4s with ffmpeg
ruff check src tests
The seed catalog is about 20 Reel-editor rows (cafe, product, talking-head, travel, food, fashion, event) plus a vocal fixture and a CC-BY-NC fixture. It runs without API keys. Gemini, Jamendo, and Freesound are optional and degrade with a note in the tool response. Seed rows have no hosted audio on purpose — preview_mix synthesizes a demo bed. For real ads, point SONICMATCH_LIBRARY_PATH / EPIDEMIC_LIBRARY_PATH / ARTLIST_LIBRARY_PATH at JSON you already licensed.
Variable
Required
What it does
GEMINI_API_KEY
no
Gemini video understanding for analyze_video_music
JAMENDO_CLIENT_ID
no
Jamendo search
FREESOUND_API_KEY
no
Freesound beds and loops
SONICMATCH_LIBRARY_PATH
no
User-owned library JSON. Also EPIDEMIC_LIBRARY_PATH, ARTLIST_LIBRARY_PATH
SONICMATCH_ALLOW_YTDLP
no
1 turns on platform URL ingest. Default 0
SONICMATCH_MAX_DOWNLOAD_MB
no
Download cap. Default 200
SONICMATCH_HTTP_HOST / SONICMATCH_HTTP_PORT
no
--http bind. Default 127.0.0.1:8765
Features
Watches the picture. Gemini video understanding when GEMINI_API_KEY is set; otherwise local ffmpeg / audio heuristics (optional Whisper, PySceneDetect, librosa).
License on every row. CC / royalty-free / user-owned library. Non-commercial tracks are never auto-recommended. Generated beds stay out of auto recs.
Editor-shaped output. 12–20s hook in/out, speech ducking, ffmpeg recipe, mix spec for CapCut / Premiere / DaVinci / your agent.
Agent-native MCP. stdio for Claude Desktop and Cursor; streamable HTTP for web editors. Never ships raw multi-MB video through the payload — you get an asset_id.
Works without API keys. 20-track seed catalog for Reels. Optional Jamendo, Freesound, and user-owned Artlist / Epidemic JSON.
Cuts on the beat.suggest_cuts snaps scene cuts to a BPM grid and returns EDL-ish intro / peak / outro.
Brand lock. Save BPM / mood / no-vocals kits and pass brand_kit=… into recommend_bgm.
Series, not one-offs.analyze_batch (max 20) clusters mood and returns one shared mini-playlist.
Honest about closed graphs. No fake “trending audio,” official IG stickers, or Content-ID-safe stamps. yt-dlp platform ingest is opt-in.
Tools
Tool
What it does
status
ffmpeg / keys / seed count / day-1 risk gates
ingest_video
Local path or HTTPS URL → asset_id (never video bytes). Platform URLs need SONICMATCH_ALLOW_YTDLP=1
analyze_video_music
Mood, energy curve, speech, scenes, hook window, BPM, search queries
recommend_bgm
3–7 ranked tracks + why + license + hook in/out
search_music
Free-text / BPM / mood over seed + optional catalogs
Mix spec + ffmpeg + attribution (no render unless render=true)
suggest_cuts
Beat grid, snapped scene cuts, EDL, intro / peak / outro
generate_bed
Demo bed marked source=generated. Requires i_understand_not_commercially_cleared=true
save_brand_kit
Persist BPM / moods / no-vocals for recommend_bgm(brand_kit=…)
analyze_batch
Up to 20 clips → mood cluster + shared mini-playlist
Also ships a prompt template: “Score this video like an IG music sticker.”
Product rules (Instagram-like, not Instagram)
Prefer instrumental when speech_coverage > 0.25
Recommend a hook window, not the whole song
Show why (cuts at 0.8s average, 112 BPM, warm gold hour)
Always return license + attribution text
3–7 tracks, not 40
User can override mood / genre / no-lyrics / platform / energy
Never claim “cleared for Instagram official sticker” unless it actually is
Example
Prompt:
I dropped ./clip.mp4. Analyze it for an Instagram Reel and recommend 5 instrumental BGMs. Then mix the top pick with ducking and give me the ffmpeg command.
The agent should call, in order:
ingest_video with the local path. The result is an asset_id, not video bytes.
analyze_video_music with platform_hint set to instagram_reel.
recommend_bgm with that asset_id and max_results of 5.
preview_mix on the top track with ducking, then export_mix_spec.
A recommendation row carries score, reason, suggested_in_out (the hook), ducking, license_ok_for_platform, and content_id_risk (unknown or likely, never cleared). Seed fixtures include seed_golden_hour_latte (CC0-1.0) and seed_indie_porch_110 (CC-BY-4.0). They have no hosted audio. preview_mix synthesizes a demo bed. CC-BY-NC rows are searchable and are not auto-recommended.
After PyPI, use examples/cursor.pypi.mcp.json (uvx with args ["sonicmatch-mcp"]). From a clone: "command": "uv", "args": ["--directory", "/absolute/path/to/sonic-match-mcp", "run", "sonicmatch-mcp"]. After pip install, "command": "python3", "args": ["-m", "sonicmatch"] works if that interpreter has the package.
Codex
Codex reads ~/.codex/config.toml. The same file is shared with the Codex IDE extension and the ChatGPT desktop app. Works today — examples/codex.config.toml:
codex mcp list should show sonicmatch. A trusted project can use .codex/config.toml instead of the home file.
User-owned Epidemic/Artlist JSON shape: examples/user_library.example.json. Registry metadata: server.json. HTTP editors can point at http://127.0.0.1:8765/mcp after sonicmatch-mcp --http.
Hosted (sonic.codecrafter.dev)
The GitHub repo stays MIT. Running it yourself is not metered.
The public endpoint charges credits per tool call. A trial key is 25 credits. Pack prices: Reel 500 credits / $9, Studio 3000 credits / $39. Card checkout is not connected on the host yet.
--http has no authentication. Keep it on loopback. The Docker image binds 0.0.0.0 so the container port works — do not publish that port to the internet. See SECURITY.md.
Hard rule: never send raw multi-MB video through the MCP payload. Store locally, pass an asset_id. Loopback, file://, and private IPs are rejected (SSRF).
VideoSonic profile
Analysis returns structured JSON, not a paragraph:
json
{"duration_sec":18.4,"aspect":"9:16","content_type":"lifestyle","has_speech":true,"speech_coverage":0.62,"existing_music":false,"overall_mood":["warm","playful"],"energy_mean":0.62,"energy_curve":[{"t":0,"energy":0.3},{"t":4,"energy":0.8}],"pacing":"fast-cut","scenes":[{"start":0,"end":3.2,"description":"cafe exterior","energy":0.4}],"hook_window":[9.0,15.0],"suggested_bpm":[95,118],"avoid":["dark cinematic drone","aggressive trap","lyrics-dense"],"search_queries":["warm acoustic pop instrumental cafe"],"platform_hint":"instagram_reel","analyzer":"local"}
Primary: Gemini video understanding when GEMINI_API_KEY is set.
Fallback: ffmpeg scene cuts + WAV energy / silence / ZCR heuristics. Optional faster-whisper, scenedetect, librosa if installed (pip install 'sonicmatch-mcp[local-vl]').
Music hub
Pluggable, license-first. v0 ships:
Adapter
When
License reality
Seed catalog (data/seed_tracks.json)
always
20 CC0 / CC-BY Reel beds + a vocal fixture + a CC-BY-NC fixture (NC is never auto-recommended)
you already licensed it; we do not scrape paid sites
Generate
generate_bed
always source=generated; local sine demo unless you swap a real model
Ranking (weighted): mood/energy → instrumental if speech → duration/loop → BPM vs cut rate → license fit → tag embedding cosine → user constraints. recommend_bgm drops non-commercial and generated tracks instead of downranking them.
Tracks are indexed in SQLite (~/.cache/sonicmatch-mcp/db/tracks.sqlite) with a 24-d tag embedding. If lancedb is installed (pip install 'sonicmatch-mcp[embeddings]'), vectors are also upserted there.
Seed tracks have no remote audio files on purpose (you should host files you actually have the rights to). preview_mix synthesizes a CC0 demo bed so the mixer still runs offline. generate_bed is a catalog-miss fallback and is not cleared for ads.
Docker
bash
docker build -t sonicmatch-mcp .
# Loopback-only publish. The process inside the container has no HTTP auth.
docker run --rm -p 127.0.0.1:8765:8765 -v sonic-cache:/data/cache sonicmatch-mcp
Roadmap
Catalog > new tools.
Freesound adapter (loops / beds)
Tag embeddings in SQLite (+ optional LanceDB extra)
Epidemic Sound / Artlist as user-owned JSON plugins (no scrape)
Beat-grid vs scene-cut suggestions (EDL-ish suggest_cuts)
MCP registry listing (server.json)
Generate tool, marked source=generated (local demo; swap a real model at your own legal risk)
Official MCP registry listing via GitHub Release MCPB (see PUBLISH.md)
Non-commercial licenses excluded from auto recommend_bgm
20 seed beds a Reel editor would actually keep, with audio you host
User-owned Artlist / Epidemic JSON as the default path for ads
Real CLAP audio embeddings
PyPI project metadata, console script, and tag-triggered Trusted Publishing (.github/workflows/release.yml)
First upload on PyPI (uvx sonicmatch-mcp). Maintainer steps are in PUBLISH.md. The name was unregistered as of 2026-10-21.
Why this can be a good open-source project
Yes if you nail: (1) video-native analysis, (2) license honesty on every row, (3) editor-shaped output (hook in/out, ducking, mix spec), (4) a catalog someone would keep.
No if you only wrap YouTube Music search, or if the first five recs sound like leftover stock beds.
Day-1 risk gates (enforced in code, not slogans):
Risk
Gate
Content ID
Every rec/search/get_track includes content_id_warning. CC/RF is never "Content-ID-safe". content_id_risk is unknown or likely, never cleared.
yt-dlp ToS / broken extractors
Platform URL ingest is off unless SONICMATCH_ALLOW_YTDLP=1. Failures map to YTDLP_EXTRACTOR and tell you to pass a local file.
Upload size / SSRF
HTTPS-only remote ingest, no file:// / loopback / private IPs, SONICMATCH_MAX_DOWNLOAD_MB (default 200) on files, HTTP, and yt-dlp --max-filesize.