Fanout

Single-binary, agent-native OpenTelemetry investigation.

Fanout ingests OpenTelemetry data, durably publishes it as atomic Parquet batches, and puts an AI agent in front of it — in one Go process with no external dependencies to operate. Persistent trace indexes serve targeted reads; embedded DuckDB handles SQL, broad scans, and rebuildable rollups. Point an SDK or Collector at it, open the browser, and ask questions about your telemetry in plain language.

There is no separate ingester, query service, metadata database, object store, or dashboard server to deploy. One binary, one data directory.

Architecture

One process owns ingest, storage, query, alerting, the agent runtime, and an MCP server. Everything below the dashed boundary is compiled into a single executable, including the React client.

Telemetry lands over OTLP/gRPC or OTLP/HTTP. Concurrent small requests may share a group-commit batch, while up to four workers independently encode and durably publish atomic Parquet directories with persistent trace indexes. Targeted trace reads go through those indexes; DuckDB scans the same Parquet for SQL and maintains rebuildable service, endpoint, and edge rollups. The browser client, an in-process agent, and any external MCP host all reach the same typed observability contract rather than issuing raw SQL.

Parquet is authoritative telemetry, DuckDB query state is rebuildable, and SQLite is reserved for transactional product state. Native compaction prepares replacements while reads continue and briefly gates readers only for the crash-safe namespace swap:

Application state (users, sessions, dashboards, alert rules, agent threads) lives in the control SQLite database and never sits on the telemetry write path. The published Parquet directories are self-describing, so startup can discover the authoritative batch set directly from the filesystem.

Performance

The independent Fanout Bench project measures authenticated ingest and optional dashboard read load against your hardware. It uses the official OpenTelemetry generator and publishes raw, reproducible evidence separately from the production binary. Ingest, indexed reads, DuckDB analytics, and native Parquet maintenance have separate coordination paths but still compete for the same CPU, memory bandwidth, filesystem cache, and disk.

The current publication candidate is 296,196 accepted OpenTelemetry items per second sustained for five minutes with traces, logs, and metrics arriving together on a machine with eight logical CPUs and 15.6 GiB of memory. It is a single run, and the benchmark harness that produced it carried uncommitted local changes, so it is not Fanout's official headline yet. The performance methodology shows the signal breakdown, quality gates, limitations, and publication bar.

How it compares

Fanout is a single node holding traces, logs, and metrics for a system you can reason about from one place. That premise, rather than any single feature, is what separates it from its neighbours.

Fanout is a single node. It has no clustering, no replication, and no object tier; a deployment that outgrows one machine's disk and CPU has outgrown Fanout.

Requirements

• Go and a C compiler with CGO_ENABLED=1 — DuckDB is a cgo dependency
• Bun — compiles the browser assets
• just — task runner
• A 32-character authentication code secret

SMTP and an AI provider are optional. Without SMTP, an operator can mint a short-lived login link from the local Fanout binary. Without an AI key, ingest, dashboards, traces, logs, metrics, and MCP continue to work; only investigation chat and AI-assisted controls are hidden.

Quick start

Native binary

Release archives support Linux and macOS on amd64 and arm64. The installer verifies the selected archive against the release checksum before extracting:

curl -fsSL https://raw.githubusercontent.com/labstack/fanout/main/scripts/install.sh | sh

Set FANOUT_VERSION=v{YYYY.M}.{N} to pin a release and FANOUT_PREFIX to choose the installation directory.

Docker

docker run --name fanout -p 7520:7520 \
  -v fanout-data:/var/lib/fanout/data \
  -e FANOUT_AUTH_CODE_SECRET=$(openssl rand -hex 32) \
  labstack/fanout:latest

Open the one-time setup URL printed by the container and create the first administrator. Fanout displays the ingest token exactly once; save it with your collector secrets. A standard OTLP/HTTP exporter can then use:

export OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf
export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:7520
export OTEL_EXPORTER_OTLP_HEADERS="authorization=Bearer%20$INGEST_TOKEN"

For a Collector on the same private container network, the forwarding side is:

exporters:
  otlp_http/fanout:
    endpoint: http://fanout:7520
    headers:
      Authorization: "Bearer ${env:INGEST_TOKEN}"

service:
  pipelines:
    traces:  { receivers: [otlp], exporters: [otlp_http/fanout] }
    metrics: { receivers: [otlp], exporters: [otlp_http/fanout] }
    logs:    { receivers: [otlp], exporters: [otlp_http/fanout] }

This assumes the Collector's existing otlp receiver and an environment variable containing the one-time token. Replace fanout with the private hostname reachable from that Collector.

The equivalent OTLP/gRPC endpoint is localhost:7520 (plaintext locally). Behind a TLS proxy, both transports use the public origin; gRPC requires HTTP/2 to Fanout. See the single-port migration notes. For later sign-in without SMTP, mint a 15-minute, single-use link against the running container's control database:

docker exec fanout fanout --config /etc/fanout/fanout.yaml \
  login-link admin@example.com

Add FANOUT_AI_API_KEY to enable chat. Configure all four SMTP settings (FANOUT_SMTP_HOST, FANOUT_SMTP_USERNAME, FANOUT_SMTP_PASSWORD, and FANOUT_SMTP_FROM) to enable email-code login.

The distroless image runs unprivileged as UID 65532. A bind-mounted host directory at /var/lib/fanout/data must be writable by that user; a named volume, as above, needs no such handling.

The image selects /etc/fanout/fanout.yaml by default. That file contains only the container listener and data-directory defaults; it does not contain credentials. Start a container-specific document from that file so it retains the shared listener and persistent data-directory settings:

cp fanout.docker.yaml fanout.yaml
# Add the remaining settings, then mount it over the image document:
docker run -v ./fanout.yaml:/etc/fanout/fanout.yaml:ro \
  labstack/fanout:latest

All surfaces use server.addr (:7520 by default). Publish only that port; OTLP/HTTP and OTLP/gRPC use the same address as the application.

From source

git clone https://github.com/labstack/fanout.git
cd fanout
just install   # browser dependencies and git hooks
just build     # browser assets, then the binaries

Run it with the minimum configuration:

export FANOUT_AUTH_CODE_SECRET=$(openssl rand -hex 32) # must be 32+ characters

./bin/fanout

Optionally set FANOUT_AI_API_KEY for chat and the SMTP variables shown above for email-code login. Without SMTP, run ./bin/fanout login-link admin@example.com from the same configuration and data directory.

Fanout serves the UI, API, MCP, OTLP/gRPC and OTLP/HTTP on http://localhost:7520. The first account created becomes the administrator and receives the ingest token once.

Point any OpenTelemetry collector or SDK at either OTLP endpoint with the ingest token. Use the separate Fanout Bench project for controlled capacity tests.

Configuration

Configuration is resolved once at startup and validated before Fanout opens data files or listeners. Sources apply in this order: built-in defaults, an optional YAML document selected with --config, then FANOUT_ environment variables. Fanout does not search for configuration or load .env files.

fanout.example.yaml is the complete commented schema:

cp fanout.example.yaml fanout.yaml
./bin/fanout --config ./fanout.yaml

Environment variables override the corresponding YAML values and are useful for container injection and secrets. An empty environment value means "no override"; use YAML for an explicit empty string. Unknown YAML keys and unknown FANOUT_ variables are startup errors, except for the service-discovery names Kubernetes and Docker link-style networking inject for a Service named fanout. All environment variables outside the FANOUT_ namespace are ignored.

YAML null values are rejected. Boolean values must use YAML 1.2 true or false (not yes, no, on, or off). If a YAML document contains a credential, Fanout requires that the file not be accessible by group or others (for example, mode 0600). The definitions live in internal/config/config.go; the settings most operators touch are:

Advanced DuckDB sizing

Most deployments should not set DuckDB variables. Give the process or container the CPU and memory limits it may use; at startup Fanout reserves headroom for Go and sizes the DuckDB connection pool from available CPUs. The resolved values and whether Fanout chose them are available in the runtime_sizing block returned by /readyz and /api/health.

These variables are escape hatches for measured, unusual workloads:

An explicit value always wins. Startup logs and /readyz report the detected host and cgroup limits, the selected source, and whether detection was incomplete. If Fanout cannot conclusively inspect a container limit, it warns; set storage.duckdb.memory or FANOUT_DUCKDB_MEMORY for a guaranteed bound.

For TLS and reverse proxies, health checks, backups and restores, upgrades, retention, and recovery, see the operator runbook.

Development

just            # list every recipe
just check      # the full gate: format, lint, asset freshness, and tests
just test       # Go tests
just ui         # rebuild the embedded browser assets

The browser workspaces build into internal/ui/dist and internal/mcp/apps, and those outputs are committed because go:embed needs them present in a source checkout. The binary is therefore only ever as fresh as the last UI build, so just ui-check rebuilds both workspaces and fails if the committed bytes no longer match. It is part of just check and runs in CI.

Lefthook runs formatting and linting on commit and the full gate on push; just install wires it up. The pre-push hook is a convenience and lefthook may skip it when it detects no changed files — CI runs the same just check unconditionally, and that is what actually enforces it.

Project layout

cmd/fanout/        process composition and the single entry point
internal/          ingest, storage, query, agent, MCP, auth, alerts
ui/host/           React AG-UI browser host (build-time)
ui/apps/           portable React MCP Apps (build-time)
docs/diagrams/     d2 sources and rendered SVG

Releases

Versions are CalVer — v{YYYY.M}.{N}, numbered from 0 within each month, so v2026.8.1 is the second release of August 2026. Pushing a tag publishes the same release manifest to Docker Hub and GHCR and moves both latest tags. GHCR remains the canonical registry and also carries development images:

Release images are multi-architecture for linux/amd64 and linux/arm64. Release archives provide Linux and macOS binaries for amd64 and arm64; every artifact is built on a native runner because DuckDB requires cgo. The complete release and verification contract is in docs/release.md.

Contributing

Issues and pull requests are welcome — see CONTRIBUTING.md. Run just check and just test-race before opening a pull request; together they match CI. For anything security-related, follow SECURITY.md instead of opening an issue.

Scope

This repository is Fanout itself, and it builds to a working binary with no other repository involved. Not included: LabStack's own deployment configuration, uptime monitoring, and public demo instance. Those describe how we operate Fanout, not what it does.

The documentation site in site/ is included, and is the exception that proves the rule: it is not marketing copy but the product's own reference, and part of it is generated. cmd/fanout-docgen writes every settings page from the same internal/config type the loader binds, and just check fails when a committed page no longer matches it. Documentation that can drift from the binary is documentation that eventually lies about it, and the only place that check can run is next to the code it checks.

License

Apache-2.0 © LabStack LLC. See NOTICE and THIRD_PARTY_NOTICES for attribution, and TRADEMARK for use of the Fanout name and logo.