GLBForge — the web-readiness layer for AI-generated 3D

glbforge.dev · raw AI mesh in, shipped asset out

The deterministic layer after 3D generation, built for agents and CI. Two jobs: the agent's eyes while a mesh is being edited (inspect reads it semantically in tens of milliseconds, diff says what the last edit broke), and the gate before it ships (analyze → optimize → verify, with versioned performance budgets as a CI-able contract and "no visible loss" measured as an SSIM number). Also forges 2D artwork into watertight 3D, routes photographs to generators, and exports GLB / STL / USDZ.

Status

• ✅ glbforge inspect — the edit-loop read (sub-second): one shell or floating pieces, watertight or holes/overlaps, size in metres, up axis, origin placement, unapplied/mirrored transforms — each finding a versioned rule id (topo/open-edges from core-geometry@1) with measured/heuristic certainty, a likely cause with confidence, and a concrete fix. front is never guessed. Same thing as the MCP inspect tool: call it after every edit.
• ✅ glbforge diff <before> <after> — what changed since the last edit, including what it broke: per-part size deltas ("'legs' is 30% narrower along X"), triangle and shell deltas, topology regressions (was watertight, now has 3 open loops), origin drift, node transforms, meshes added or removed, and --visual front/side/top/iso SSIM with cameras fixed to the before framing. Same thing as the MCP diff tool.
• ✅ glbforge analyze — budget report card + named lint rules
• ✅ glbforge optimize — weld/simplify/LOD/compress to hit the budget
• ✅ glbforge scaffold — emit a React Three Fiber viewer for the optimized asset
• ✅ glbforge meshy — generate/download via Meshy REST API (image/text → 3D, --optimize glue)
• ✅ glbforge ship <input> — one call from a GLB, a logo, or a photo to a budget-gated asset: route (optimize / forge / generate) → analyze → optimize → gate. --json emits the route, the forge's decision and the full optimization report as one document.
• ✅ MCP server wrapping all of the above (@glbforge/mcp, 28 tools, dev.glbforge/glbforge in the official MCP registry)
• ✅ GLBForge Studio (glbforge ui) — local web UI: drag-drop analyze, one-click optimize with a before/after compare slider in the viewport, logo forging, Meshy generation with live progress, STL export. Zero hosting; runs entirely on your machine.
• ✅ glbforge extrude — deterministic logo/graphic → 3D (no AI, no credits): traces the silhouette (marching squares → Douglas-Peucker → earcut), extrudes it watertight, and projects the source image back on as the texture. For 2D artwork (logos, wordmarks) this beats image-to-3D generation outright: ~1.5k tris vs ~2M, exact silhouette, original colors. --bevel adds a signage-style rounded rim (miter-limited, clamp-probed insets + a crack stitching pass keep it watertight even on thin graffiti strokes; verified 0 boundary / 0 non-manifold edges on real logo fixtures). Photographic inputs are detected and routed to Meshy instead — or, with --matte auto, lifted off their background first (see below). Also exposed as the extrude_image MCP tool.

--matte auto — lift the subject, forge the sticker

A photograph has no alpha and no white ground, so the forge refuses it. The matte (matte/border@2, core/src/extrude/matte.ts) is the third answer: it takes the frame edge as the background, grows it inward through pixels of that colour, and calls what survives the subject — the idea behind a phone's "lift subject", done with plain connectivity, so it adds no model download, runs identically in Node and the browser, and is byte-for-byte deterministic. Colour decides each pixel's class and connectivity decides what happens to it, which is what keeps a mug's handle open: an enclosed region of background colour is a hole, not subject. Small enclosed regions (highlights, JPEG speckle) are filled, small disconnected pieces are dropped as debris. Matching is chromaticity-aware in one direction — a pixel that is close in colour and darker than the background is a shadow of it, so a cast shadow stops being welded to the object, while a white mug on a grey desk (brighter, same hue) is never absorbed. A 3x3 majority filter smooths the mask before tracing, so JPEG ringing along an edge does not become hundreds of contours.

The tolerance tunes itself. tuneMatte sweeps an eight-rung ladder and keeps the best-scoring cut, because confidence falls off at both ends of the knob — too tight leaves the background attached, too loose eats the object. The Studio does it on drop (the slider opens on the tuned value, tagged auto), --matte-tolerance auto does it on the CLI and prints the ladder, and the MCP's matte_preview does it by default.

Look before you forge. A tolerance you cannot see is a tolerance you cannot choose, so the cut is previewable everywhere it is offered: glbforge extrude photo.jpg --matte auto --matte-preview cut.png writes the cut (subject opaque, removed background ghosted) and prints its numbers without building geometry; extrude_image takes matte_preview: true and returns that image plus nextActions naming the tolerances to try; and the Studio redraws the cut live under the lift button as the slider moves.

A mask is an inference, so it reports like one — coverage, pieces, holes, and a confidence built from how uniform the edge was, how much contrast the cut ran through, and whether the result is a sensible size. Below 0.4 the forge refuses with those numbers rather than handing back a blob; the MCP surfaces the accepted case as a SUBJECT_LIFTED diagnostic so an agent knows the silhouette was inferred, not read. --matte auto on the CLI, matte on extrude_image, and ✂ Lift subject and forge in the Studio, offered exactly where the refusal used to be a dead end.

Usage

pnpm install && pnpm build
node packages/cli/dist/index.js analyze fixtures/veiled-guardian.glb --profile mobile-hero
node packages/cli/dist/index.js optimize fixtures/veiled-guardian-tex4k.glb -o out.web.glb --lods 40000,10000
node packages/cli/dist/index.js scaffold out.web.glb -o viewer && cd viewer && pnpm install --ignore-workspace && pnpm dev
node packages/cli/dist/index.js ui model.glb   # GLBForge Studio on localhost:5177

inspect <file> (glb/gltf/usdz/usda/usdc) answers what an agent editing a mesh gets wrong blind: ~0.1 s on a 150k-triangle asset, ~0.8 s on a raw 2M-triangle generation (the welded topology pass itself is ~70 ms per 150k triangles). --profile authoring (default) treats topology problems as warnings because they are most likely the last edit's doing; a budget profile (mobile-hero) reports them as info. --packs core-geometry@1,core-scene@1 pins rule packs, --no-topology skips the welded pass (rules are listed as skipped, never silently absent), --strict exits 1 on warnings, --json emits the full report. --expect "chair, Z-up, meters, single-shell, 0.4-1.2m tall, front -Y, watertight, origin base" turns the run into a contract (intent@1): shell count, watertight, size range and origin are measured and fail as errors (exit 1); a bare category gives a plausibility warning from a coarse size table with a stated confidence; front is recorded as declared and never measured; Z-up on a GLB is informational because glTF is Y-up by definition. Tokens the parser cannot place are reported, not dropped.

glbforge usage is the local, opt-in counter behind the one metric that says whether GLBForge is in the loop: invocations per asset. Off by default; opt in with glbforge usage --enable or GLBFORGE_USAGE=1. Every inspect / diff / analyze (and every MCP call) then appends one JSON line to ~/.config/glbforge/usage.jsonl (GLBFORGE_CONFIG_DIR to move it). Nothing is ever sent anywhere; --clear deletes it. Because every edit changes the file's hash, assets are grouped into lineages: the same path in the same session, the same path within two hours, a diff edge (before → after), or an explicit --lineage <id>. The report gives median / p90 invocations per lineage and the share of assets inspected at least five times.

analyze flags: --profile mobile-hero|desktop-hero|product-configurator, --json, --no-topology. Exits non-zero when the asset is over budget — wire it into CI like a linter. optimize flags: --target <tris>, --lods a,b, --no-textures, --no-compress, --no-verify. LOD files (--lods 40000,10000) are geometry-only: materials stripped, positions welded, meshopt-simplified, and when non-manifold topology stalls the simplifier (stacked forge layers, doubled shells) they fall back to grid vertex clustering so the target is always reached — the CLI and the Action report say grid-clustered when that happened.

Measured on the included Meshy 7 fixture (4K textures, 1.99M tris): 89.4MB → 5.5MB (93.8% smaller), score 25 → 100, ~7s. Pipeline: dedup → weld → meshopt-simplify to budget (error ladder) → fill missing normals → texture resize + WebP (normal maps near-lossless) → prune → EXT_meshopt_compression → perceptual verification.

"No visible loss" is measured, not claimed

Every optimize/ship renders the asset from four fixed cameras before and after (deterministic software rasterizer, 2x supersampled, smooth shading, base-color textures) and scores the pairs with SSIM. The weakest view must clear the profile's minSsim floor (mobile-hero 0.94, product-configurator 0.95, desktop-hero 0.96) or the report card gets an error-severity fidelity/perceptual finding and the command exits non-zero, exactly like a budget violation. A passing run records the number as an info finding, so the score ships with the report everywhere (CLI, --json, MCP, Studio).

  visual fidelity ✓ SSIM 96.7%  weakest view 95.8% @ verify_135 · floor 94.0% · geometric deviation ≤ 0.1%

On the fixture: the budget pass (1.99M → 150k tris) measures 0.958; forcing 40k tris drops to 0.896 (fails, and it should — hair strands merge); 10k measures 0.73. glbforge verify <candidate> <reference> scores any two files the same way. The cameras are fixed to the reference's bounds so a shifted or shrunken result cannot re-frame itself into a good score.

Packages

MCP server

.mcp.json registers the server for Claude Code automatically (build first: pnpm build). For other clients:

claude mcp add glbforge -- node /path/to/XUI/packages/mcp/dist/index.js

Built for agents. Every tool answers one envelope — { ok, summary, duration_ms, errors[], data } — where errors[] carries every diagnostic with a stable code (docs/error-codes.md), a severity and the prim path it refers to, and every response validates against schemas/<tool>.output.json. validate, inspect_geometry, inspect_animation, inspect_materials, analyze_performance and inspect_all read GLB, glTF, USDZ, USDA and USDC (pure-TypeScript USD readers, single layer); render / render_animation_strip draw any of them, posed at any animation time. Mutating tools take dry_run and return a diff plus post_validation, and report every change they made on their own (generated normals, joined primitives, transcoded textures…) as a coded entry. Results are compact cards (verdict, key numbers, every error plus top findings, nextActions, a drillDown pointer) with inspect_report for the full findings/textures/topology on demand; every tool that touches a GLB returns a rendered thumbnail or 2x2 turntable, and a failing optimization returns a reference | result | change-heatmap sheet so the agent sees where the loss is (compare_glb for any two files). capabilities says which providers have keys and whether KTX2 is available before a plan is made; every written file reports its sha256; read-only tools carry readOnlyHint so clients can auto-approve them. Generation tools are deliberately split into create/status/download — tasks take minutes, and agents poll at their own pace instead of holding a tool call open.

Animated assets

Skinned meshes and morph targets go through a bone-aware path: meshoptimizer's attribute-aware simplifier sees the skin weights (and per-target morph deltas) as vertex attributes, vertices on dominant-joint boundaries are locked, and every attribute — JOINTS_n, WEIGHTS_n, each target — is compacted with the same remap. Skins, inverse bind matrices, and animation clips survive untouched; analyze reports scene/animated-asset. Tested on a rigged cylinder with a rotation clip and a morph target (blend band preserved, weights normalized, deterministic).

USDZ for iOS AR

npx glbforge animate model.web.glb       # → model.idle.glb (looping idle clip, no rig; -p spin|bob|sway|breathe|hop)
npx glbforge companion model.idle.glb    # a talking desktop character (Electron via npx -y @glbforge/companion; --mcp registers its bridge, --hooks puts every Claude Code session on its face)
npx glbforge usdz model.web.glb          # → model.usdz (PNG textures; --jpeg for smaller)

Writes a binary USD (usdc, crate 0.8.0) layer with UsdPreviewSurface materials (base color, metallic/roughness via channel outputs, normal, occlusion, emissive, alpha mask/blend), transcodes WebP textures to PNG/JPEG, and packs a store-only zip with 64-byte-aligned payloads as the spec requires. The crate writer is GLBForge's own, pure TypeScript — its byte layout was verified section-by-section against files from Pixar's writer, and test/usd-oracle.py opens the usdc and its usda twin with Pixar's USD and checks every prim, value, connection, and relationship for equality (set GLBFORGE_PXR_PYTHON to a python with usd-core to run it). On the Meshy fixture the layer is 5.4MB binary vs 13.4MB ASCII. --usda writes the text layer instead. Animated assets export as UsdSkel: joints ordered parents-first with bind and rest transforms, SkelBindingAPI joint indices/weights on the meshes, and the first animation clip sampled at 30 fps into a SkelAnimation (translations, quaternion rotations, half scales). The oracle test asks Pixar's UsdSkel to recompute joint transforms from the result. Morph targets become UsdSkel BlendShape prims with the weight track sampled alongside (a morph-only mesh gets a one-joint skeleton so UsdSkel applies them). Primitives without a material get a neutral default one, since an unbound mesh renders as RealityKit's magenta "missing material" pattern. KTX2 inputs are rejected with guidance. Also export_usdz on the MCP server and an Export USDZ button in the Studio (in-browser too — textures transcode through a canvas). Reference it from <model-viewer ios-src="model.usdz"> for the AR button on iOS. Verified on an iPhone in AR Quick Look: flat-material, fully textured (JPEG color, PNG normal and metallic/roughness), skinned + animated, and blend-shape exports all open and play correctly.

glbforge init — make a project agent-ready

npx glbforge init            # in your project; idempotent

Writes a marker-delimited section into CLAUDE.md (what the budget is, which scripts and MCP tools to use, the forge-vs-generate routing rule, and that "no visible loss" is measured), adds glb:check / glb:analyze / glb:optimize / glb:ship / glb:verify / glb:studio npm scripts (plus glbforge as a devDependency), and registers the MCP server in .mcp.json (--client cursor|both for .cursor/mcp.json). Existing content, other MCP servers, and your own scripts are preserved; a second run reports everything unchanged. Flags: --profile, --assets <dir>, --local, --dry-run, --force, --no-claude-md, --no-scripts, --no-mcp. From then on any agent session in that project picks GLBForge up automatically.

glb:check runs glbforge audit <dir>: every GLB in the folder against the budget, non-zero exit if any fails — the pre-commit / CI gate.

GitHub Action

# .github/workflows/assets.yml
on: pull_request
permissions: { contents: write, pull-requests: write }
jobs:
  assets:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v7
        with: { fetch-depth: 0, lfs: true }
      - uses: glbforge/glbforge@main
        with:
          profile: mobile-hero
          optimize: true      # open a PR with the web-ready files

Runs only on GLB/glTF files changed in the pull request. Every run posts a sticky report-card comment and gates the check on the budget. With optimize: true it also optimizes each failing asset (<name>.web.glb beside the original, or in place with replace: true), verifies it perceptually, and opens or updates a pull request against the PR branch with the results — before/after table, visual SSIM, and the pipeline steps. Same input, profile, and CLI version always produce identical bytes, so outputs are cached by content hash (cache: true) and re-runs are no-ops. Fork PRs cannot receive a bot branch; their outputs are uploaded as a workflow artifact instead. Opening the PR needs the repository (and, for organizations, the org) setting Allow GitHub Actions to create and approve pull requests; when it is off, the run still posts the report card and uploads the optimized files as the glbforge-optimized artifact. In optimize mode the gate passes when the optimized output passes, so the fix is always one merge away. target-triangles overrides the profile's triangle target and lods: 40000,10000 adds geometry-only LOD files (<name>.web.lodN.glb) to the PR; both are part of the cache key. Inputs: profile, fail-on-budget, optimize, optimize-all, replace, verify, target-triangles, lods, cache, open-pr, pr-branch, version, token.

Meshy API key

Copy .env.example to .env and set MESHY_API_KEY (gitignored, loaded automatically, never overrides real env vars). Or keep it out of files entirely via macOS Keychain:

security add-generic-password -a "$USER" -s meshy-api-key -w
# ~/.zshrc: export MESHY_API_KEY=$(security find-generic-password -s meshy-api-key -w)

Full loop, one command:

node packages/cli/dist/index.js meshy image art.png --pbr --optimize -p mobile-hero -o hero.glb

Open-model generation (fal.ai)

True image→3D via open weights on GPU inference — often 5-25x cheaper than commercial generation. Set FAL_KEY (https://fal.ai/dashboard/keys):

node packages/cli/dist/index.js gen photo.png --model hunyuan --optimize   # Hunyuan3D-2
node packages/cli/dist/index.js gen photo.png --model trellis             # TRELLIS
node packages/cli/dist/index.js gen photo.png --model triposr             # fastest

The studio's generate panel picks up every configured provider automatically (Meshy + the open trio side by side, with per-model credit costs in hosted mode).

Rules

perf/* budget violations fail the build; geo/*, topo/*, mat/*, tex/*, scene/* describe defects typical of AI-generated assets, each with a concrete fix. See packages/core/src/rules.ts.

Design decisions

• Pure Node/TS, no Blender dependency. Analysis and optimization run on @gltf-transform + meshoptimizer + sharp — native to glTF, no lossy DCC round-trip, installable via npx, CI-friendly. Ops are designed as pluggable backends so Blender-only capabilities (retopo, UV unwrap, baking) can be added later without changing the CLI surface.
• Topology in welded space. Boundary/non-manifold counts are computed after unifying position-duplicate vertices, so unwelded exports don't produce garbage numbers.
• Budgets are profiles, not advice. An asset passes or fails a named, versioned target (mobile-hero@1 pins; bare mobile-hero = latest; a cap never changes in place). The methodology behind every cap and the version changelog are in docs/BUDGETS.md / glbforge.dev/budgets. Determinism makes it automatable.

Fixtures

• fixtures/veiled-guardian.glb — Meshy 7 High-Detail geometry-stage export (1.99M tris, POSITION-only, 34MB). Welded + manifold; tests assert we don't cry wolf.
• fixtures/veiled-guardian-tex4k.glb — same asset after the 4K texture stage (89MB). Its 77k position-duplicate vertices are UV-seam splits, not waste; the topo/unwelded rule distinguishes these (regression-tested).