

unicefstats-mcp
Experimental โ not an official UNICEF product. Verify retrieved values against the UNICEF Data Warehouse before citing in publications. See Limitations.
MCP server for UNICEF child development statistics. Query 790+ child-focused indicators across 200+ countries with disaggregations by sex, age, wealth quintile, and residence. No API key required.
Indicators cover child mortality, nutrition, education, child protection, WASH (water/sanitation/hygiene), HIV/AIDS, immunization, early childhood development, and more. Many align with SDG targets, but the dataset is broader than SDGs alone.
Data source: UNICEF SDMX API
Identity
Third-party aggregator listings (LobeHub, Smithery, mcp.so, Glama) are not controlled by the maintainer. Verify against the canonical source above.
Contents
Key documents
| Document | Description |
|---|
| PROVENANCE.md | Data origin, ownership, distribution pipeline, verification steps |
| CHANGELOG.md | Version history (v0.1.0โv1.5.5) with sources cited |
| RELEASE.md | Release process checklist and version management |
| CONTRIBUTING.md | Development setup, code style, PR guidelines |
| CODE_OF_CONDUCT.md | Contributor Covenant v2.1 |
| examples/RESULTS.md | Full 300-query benchmark analysis with EQA decomposition |
| examples/LITERATURE_REVIEW.md | Literature review: MCP servers for official statistics โ ecosystem, patterns, evaluation, 15 papers |
| examples/LANDSCAPE.md | Landscape analysis: 20 servers reviewed in depth (March 2026), 100 catalogued with status labels (October 2026) โ timeline, feature matrix, strengths/weaknesses |
| examples/results/related_work.md | Annotated bibliography โ 15 papers on tool-augmented hallucination |
| examples/results/statistical_summary.md | Wilcoxon, bootstrap CI, McNemar tests on benchmark results |
| examples/MCP-DIRECTORY-STATS.md | Comprehensive directory of all official statistics MCP servers |
| examples/mcp-smoke-test/MULTIMODEL_SMOKETEST.md | Cross-model smoke test (Anthropic / OpenAI / Google / OpenRouter) for the v0.7.3 cross-provider generalisation question โ design, rubric, ~$1 default run, path to full mini-EQA |
How it relates to the unicefdata packages
unicefstats-mcp is not a replacement for the unicefdata packages in Python, R, or Stata. They serve different audiences:
| unicefstats-mcp | unicefdata (Python/R/Stata) |
|---|
| Audience | AI assistants (Claude, Cursor, Copilot) | Data scientists, researchers, analysts |
| Interface | MCP protocol (tool calls via JSON) | Native language API (library(), import, ssc install) |
| Use case | Conversational data exploration, quick lookups, AI-assisted analysis | Reproducible research, ETL pipelines, statistical analysis |
| Output | JSON (compact or full) optimized for LLM context | DataFrames, tibbles, Stata matrices |
| Scripting | No โ single queries via AI chat | Yes โ full programmatic control, loops, joins, transforms |
| Caching | Delegates to unicefdata | Built-in SDMX response caching |
| Bulk download | Limited (max 500 rows per call) | Unlimited โ designed for full dataset pulls |
Under the hood, unicefstats-mcp wraps the unicefdata Python package. Every tool call ultimately calls unicefdata.unicefData() or its metadata functions. Think of the MCP as a thin AI-friendly interface on top of the same data layer.
When to use which:
- Use unicefstats-mcp when you're chatting with an AI and want to quickly explore indicators, check values, or compare countries
- Use unicefdata (Python/R/Stata) when you're writing scripts, building dashboards, running regressions, or doing any reproducible analytical work
How it compares to other data MCPs
Servers are named by repository and carry a status label (official, independent, community or commercial) as defined in Status labels. "โ" means the feature is not documented in that repository's public README. See About these tables for how counts are taken and how to propose a correction.
| Feature | unicefstats-mcp (this repo) | data360-mcp (World Bank, official) | fred-mcp-server (community) | world_bank_mcp_server (community) |
|---|
| Tools | 9 (search โ metadata โ data โ code โ identity + strict canonical lookup) | 16 (search indicators/datasets โ metadata โ data โ disaggregation โ codelist lookup โ list; summarize / rank / compare countries; country-group expansion; single- and multi-indicator Vega-Lite viz specs; API-URL; plus an interactive-choice UI tool) | 3 (browse โ search โ get) | 1 (get only) |
| Indicators | 790+ child-focused indicators | Data360 catalogue, multiple databases (WB_WDI, WB_SSGD, โฆ) | 800,000+ economic series | ~1,600 indicators |
| Countries | 200+ (ISO3) | 200+ (ISO3 REF_AREA) | US-focused (some intl) | 200+ (ISO2) |
| Disaggregations | Sex, age, wealth quintile, residence | Sex, age, urbanisation (per indicator, via data360_get_disaggregation) | Frequency, seasonal adjustment | None |
| MCP Prompts | 5 (compare_indicators, trend_analysis, country_profile, sdg_progress, write_unicefdata_code) | 7 | None | None |
| MCP Resources | 6 (system-prompt, context, llm-instructions, categories, countries, glossary) | 11 (system-prompt, context, agent-recipe, databases, codelists, data-schema, data-filters, metadata-fields, search-usage, viz/chart-grammar, โฆ) | None | None |
| Output modes | Compact (5 cols) / Full (all cols) | SDMX-shaped JSON with a per-observation claim_id (8-char "proof-carrying numbers" hash); compact serializer for aggregation tools | JSON | CSV |
| Data summary | Value range, year range, country count | summarize_data, rank_countries, compare_countries tools; Vega-Lite chart specs | None | None |
| Pagination metadata | total_rows_available vs rows_returned | limit/offset with total_count, has_more, next_offset (MCP best-practice pattern) | limit/offset | None (hardcoded 20K) |
| Input validation | ISO3, sex, wealth, residence validated | Codelist resolution tool + server-side country-coverage check (required_country) | Zod schemas | None |
| Error guidance | error + tip with next steps | Structured error_code (e.g. http_error:search) + message, raised as MCP ToolError | HTTP status text | Raw exception |
| API key | Not required | Not required | FRED_API_KEY required | Not required |
| Truncation handling | rows_truncated flag + filter tips | Payload stripping to core fields with claim_id preserved (docs/payload_analysis.md) | None | None |
| Published | PyPI | PyPI | npm | No |
Landscape: MCP servers for official statistics
This project is part of a growing ecosystem of MCP servers for international and official statistics. Tables group servers by the data source they wrap; the Status column says who maintains them, using the labels defined in Status labels.
About these tables
Status labels
Status describes who maintains a server, not its quality or the quality of its data.
- Official: presented as its own service by the organisation that produces the data, or that operates the platform it is served from: announced on its website, in a press release or at a launch event, or documented as a service it offers. Publication under the organisation's account is not enough by itself. A qualifier marks maturity where the organisation states it: (beta), (lab), (pilot).
- Independent: built by staff of that organisation, or by one of its units, and not presented by the organisation as its own service: a personal project, as stated in the README or the maintainer's profile, or a prototype. This repository is in this category, as a personal project.
- Community: built by individuals or groups with no stated affiliation to the data producer.
- Commercial: offered by a company as part of a product or a hosted or paid service, with no stated affiliation to the data producer.
Official refers to the server, not the data. Servers whose data are not official statistics, such as research estimates, are listed separately in the directory; a research producer that maintains its own server is labelled producer-maintained, not official.
When the evidence is ambiguous, we do not claim official or independent status: the row is labelled community and the note says why. Affiliations we cannot see can be added through the landscape-correction issue template below.
Counts and corrections
The comparison and landscape tables are a snapshot, not a registry. Every count (tools, prompts, resources, stars) describes what is visible on each repository's public default branch on the date of the commit that last touched the row โ currently 2026-10-24 to 2026-10-27 โ and the ecosystem moves faster than this README. The same rule applies to every server, including this one: we do not use figures supplied privately by a maintainer unless a public artefact backs them.
If you maintain one of these servers, or know one that is missing or mis-described, please open a landscape-correction issue with the repository link and what should change. Corrections, additions and clarifications are all welcome, and are applied with the date they were verified.
The World Bank entries were corrected in October 2026 after Olivier Dupriez and Aivin Solatorio of the World Bank pointed out that a community wrapper had been labelled as the Bank's server and the official data360-mcp was missing. Aivin's comparison of the official server is the basis of the data360-mcp column above. Thank you to both. The naming rule, the Status column and these label definitions came out of that exchange.
Sancera was added to the directory in November 2026 at the request of its maintainer, @aalbin2, who proposed the row in issue #1: the first addition received through the correction template. Thank you.
UN Agencies
International Organizations
National Statistics Offices
Known gaps
Community servers now cover UNESCO UIS (since February 2026), WHO GHO and UN Comtrade (since May 2026) and ILOSTAT (since August 2026), so those are no longer open gaps; none is maintained by the agency itself. Still no dedicated MCP server found (2026-10-26) for UNDP/HDI, the UNSD SDG Global Database API, UN DESA Population, UNAIDS, WFP or WTO, though the UN System Data Commons endpoint carries some SDG series and, checked on 1 November 2026, HDI, UNAIDS and population series; BIS has no dedicated server, though the community aggregator openecon-data queries it. No official server was found from Eurostat, the IMF, the OECD or any central bank. The tables above are a selection; the directory below lists 100 servers for official statistics, plus research and other related sources and a long-tail list.
Full directory with install commands: MCP-DIRECTORY-STATS.md
Relationship to sdmx-mcp
sdmx-mcp is a generic SDMX protocol MCP server, developed in UNICEF's Chief Statistician's office as a prototype and published under the unicef-drp account. UNICEF has not announced an official MCP server, so it is labelled independent here (see Status labels). The two servers are complementary, not competing:
| unicefstats-mcp (this repo) | sdmx-mcp |
|---|
| Scope | UNICEF child development data only | Any SDMX registry (UNICEF, Eurostat, OECD, ...) |
| Tools | 7 (analyst-friendly, 4-step workflow) | 23 (SDMX power-user, structural queries) |
| Data layer | Wraps unicefdata Python package | Direct SDMX REST API calls via httpx |
| Output | Formatted for LLMs (compact tables, summaries, tips) | Raw SDMX-JSON/CSV |
Benchmark comparison withdrawn (15 November 2026). Earlier versions of this page set the two servers' benchmark scores side by side (accuracy, fabrication, cost and latency). That comparison has been withdrawn. The sdmx-mcp figures came from one March 2026 run, on a different sample and under an older scoring rule than the figures now reported for unicefstats-mcp, and a check in November 2026 found that the fabrication rates published for sdmx-mcp did not match that run's own results file. A fair comparison needs both servers run on the same sample under the same scoring rule, and that has not been done. Results for unicefstats-mcp alone are in examples/RESULTS.md.
When to use which:
- Use unicefstats-mcp for UNICEF child development analysis, through a short workflow with output formatted for language models
- Use sdmx-mcp when you need to query non-UNICEF SDMX registries, explore dataflow structures, or work with hierarchical codelists
Quick Start
pip install unicefstats-mcp
Claude Code
Add to ~/.claude/.mcp.json:
{
"mcpServers": {
"unicefstats": {
"command": "unicefstats-mcp"
}
}
}
Cursor / VS Code
Add to your MCP settings:
{
"unicefstats": {
"command": "unicefstats-mcp"
}
}
What v1.1.0 adds
v1.1.0 is additive: v1.0.0's ambiguity_flag, candidates, and abstain_instruction still fire unchanged on ambiguous queries. v1.1.0 layers four new advisory envelope fields on top:
requires_confirmation โ true when the server wants the assistant to pause for user input
recommended โ the server's preferred next action (code, country, year, or tool)
assistant_guidance โ short natural-language hint the LLM can paraphrase
next_step โ a structured tool/parameter suggestion the LLM can chain into
Decision order (4 stages): strict canonical lookup โ ambiguity check (v1.0.0 fields) โ confirmation gate (requires_confirmation) โ advisory hints (recommended / assistant_guidance / next_step).
Verdict: ALLOW (no behavioural regression; advisory fields fire on 23/30 Sonnet and 26/30 Haiku paired stuck queries; see internal/v1.1.0_design/ab_results.md).
What v1.5.x adds (Jun 2026)
The v1.5.x series targeted the long tail of cases where the v1.0.0/v1.1.0 ambiguity gate fired but the model still couldn't pick the right indicator. Six releases shipped over 7 days, all additive to the v1.1.x decision order.
| version | scope | mechanism |
|---|
| v1.5.0 | Search-layer ambiguity reductions | Single-dataflow guard at heuristic Path B; all-matches accumulation in the differentiator; tiered fuzzy synonym scorer (data360-mcp pattern); cooperative abstain_instruction (faostat-mcp pattern) |
| v1.5.1 | Tier 2c misroute fixes | Jaccard threshold bump (70โ75); substring tier length-ratio guard (โค1.5); explicit antonym synonyms (unimproved drinking water โ WS_PPL_W-UI); explicit qualifier synonyms (antenatal care 4+ visits โ MNCH_ANC4) |
| v1.5.2 | Custodian-agency defaults | CURATED_PREFERRED entries default education completion (ED_CR_L1/L2/L3_UIS_MOD) and out-of-school rate (ED_ROFST_L1/L2/L3_UIS_MOD) queries to the UIS modelled series per SDG 4 custodian convention. WASH JMP headlines (WS_PPL_W-I, WS_PPL_H-B) and the antonym-safe WS_PPL_W-UI ordering guard. Every entry's dimension_hint explicitly lists the alternative versions in the data warehouse |
| v1.5.3 | Long-tail curated clusters | PT_F intimate-partner violence (SDG 5.2.1), PT_F sexual violence by 18 (SDG 16.2.3), MNCH_BIRTH18, PT_CHLD_5-17 child labour, ED_ROFST_L02, HVA_PED_LOST (UNAIDS AIDS-orphaned children) |
| v1.5.4 | Copilot-driven correctness fixes | Removed misleading 15-19 synonym from the PT_F 15-49 default; reworded HVA_PED_LOST dimension_hint to say "AIDS-caused combined" (not "all-cause") |
| v1.5.5 | Batch-use hardening (opt-in) | Two new env vars: UNICEFSTATS_SDMX_THROTTLE_MS (inter-call delay) and UNICEFSTATS_SDMX_RETRY_403=1 (exponential backoff 30/60/120s on 403s). Default behaviour unchanged for normal use; recommended for large-batch / benchmark scenarios |
Empirical validation โ v1.5.5 1484-cell paired re-validation against the v1.2.4 Jun 1 baseline measured mean EQA 0.749 (+29.1pp, p = 2ร10โปยนโฐโต) on the v9 Arm B sample (a different, larger benchmark than the 300-query EQA test below). See Benchmark Results for both scoreboards.
Curated-agency default policy โ for indicator families where the catalogue ships both an administrative-data series and a custodian-agency modelled series (UIS for education, JMP for water/sanitation, WHO/UNICEF Joint Estimates for nutrition), v1.5.x defaults natural-language queries to the modelled / custodian series and surfaces the other variants in dimension_hint. Users who explicitly want the raw / administrative figures can re-route via the hint; the LLM has full visibility of both.
| Tool | Purpose | API call? |
|---|
search_indicators(query, limit) | Find indicators by keyword | No |
list_categories() | Browse thematic groups (CME, NUTRITION, EDUCATION, ...) | No |
list_countries(region) | List countries with ISO3 codes | No |
get_indicator_info(code) | Full metadata, SDMX details, available disaggregations | No |
lookup_by_code(code) | v1.0.0 Strict canonical-code lookup (rejects natural-language input with abstain instruction) | No |
get_temporal_coverage(code) | Available year range and country count | Yes (lightweight) |
get_data(indicator, countries, ...) | Fetch observations with optional disaggregation filters | Yes |
get_api_reference(language, function) | unicefdata package API reference (Python/R/Stata) | No |
get_server_metadata() | Server identity, version, provenance, data source | No |
Workflow
1. search_indicators("child mortality") โ find indicator codes
2. get_indicator_info("CME_MRY0T4") โ check disaggregations & SDMX details
3. get_temporal_coverage("CME_MRY0T4") โ check year range
4. get_data("CME_MRY0T4", ["BRA", "IND"]) โ fetch data
5. get_api_reference("python", "unicefData") โ get code template to continue in a script
Resources
The server exposes six MCP resources clients can load for guidance and reference data:
| URI | Purpose |
|---|
unicef://system-prompt | Recommended system prompt โ operating loop + temporal-frontier check + anti-extrapolation directive (load at session start) |
unicef://llm-instructions | Full DO/DON'T rules, common mistakes, and anti-fabrication guidance |
unicef://context | Runtime context โ current_date / current_year for temporal-query sanity checks |
unicef://categories | All indicator categories with counts |
unicef://countries | ISO3 codes and country names |
unicef://glossary | Disaggregation codes and indicator-prefix legend |
The system-prompt and context resources address the T2 hallucination failure mode (model fabricating values for years beyond the data frontier). Pattern adopted from the World Bank data360-mcp server. See CHANGELOG entry for v0.5.0.
Scope: UNICEF DW indicators only
unicefstats-mcp searches and serves the UNICEF Data Warehouse SDMX catalog โ approximately 790 child-focused indicators across mortality, nutrition, education, child protection, WASH, HIV/AIDS, immunization, and early childhood development. Indicator codes follow UNICEF conventions (e.g. CME_MRY0T4, NT_ANT_HAZ_NE2, ED_ANAR_L1).
Codes from other organisations are out of scope. The MCP cannot find, resolve, or fetch them because they do not exist in the UNICEF SDMX catalog. Examples of out-of-scope code families:
| Source organisation | Prefix examples | Typical content |
|---|
| World Bank WDI | SI.POV.DDAY, NY.GDP.PCAP.PP.KD, SE.PRM.NENR | Poverty, GDP, broader education |
| World Bank ASPIRE | per_allsp.cov_pop_tot, per_* | Social protection coverage |
| ILO / ILOSTAT | EIP_*, EAP_*, DF_*_SEX_RT | Labour, employment, NEET |
| UNESCO UIS | UIS.* | Education finance, learning outcomes |
These codes will return no hits from search_indicators and a not-found error from get_data. The refusal is correct behaviour, not a bug.
For cross-organisation queries, use a sister MCP:
data360-mcp โ the World Bank's official server for its Data360 platform, covering WDI, ILO, UN, and other official providers under one tool surface.
world_bank_mcp_server โ community wrapper (anshumax) around the World Bank Open Data API only; not a World Bank product.
For SDMX power-user queries against arbitrary registries (Eurostat, OECD, national NSOs), see sdmx-mcp โ the generic SDMX protocol server discussed under Relationship to sdmx-mcp.
The scope boundary was sharpened in v1.1.1 after the v9 edge-test sample surfaced three prompts whose ground-truth codes (SI.POV.DDAY, NY.GDP.PCAP.PP.KD, per_allsp.cov_pop_tot) sit in the World Bank universe rather than the UNICEF Data Warehouse. Full write-up: internal/v1.1.0_design/ambiguity_forensic.md sections C-4 to C-6 (dev repo only).
Understanding UNICEF indicator codes
Why this matters โ the MCP's semantic layer
UNICEF SDMX indicator codes look cryptic on first contact โ PT_F_20-24_MRD_U18,
NT_ANT_HAZ_NE2, TRGT_2030_IM_DTP3 โ and most of an LLM's failure modes on this
data start with picking the wrong code. The single biggest piece of added value
this MCP offers over a raw SDMX endpoint is exposing the semantic structure
inside those codes so the assistant can disambiguate without guessing. The
implementation lives in src/unicefstats_mcp/differentiator.py (segment-level
suffix meanings and base/variant explanation) and
src/unicefstats_mcp/indicator_resolver.py (natural-language synonyms and known
ambiguous tokens). v1.1.1 added a query-aware CURATED_PREFERRED dimension_hint
that tells the resolver which code from a family to surface for a given phrasing
(e.g. "target" โ TRGT_*, "modelled" โ *_MOD, "child" โ _T total rather
than a sex-disaggregated variant).
This section documents the conventions the MCP relies on so users and downstream
agents can read codes directly, and so reviewers can audit the resolver's choices.
Anatomy of a UNICEF code
UNICEF DW codes are concatenations of dot-free, underscore-delimited segments,
read left-to-right from broadest to most specific. Take a layered protection
indicator from the catalog:
PT_F_20-24_MRD_U18
โ โ โ โ โ
โ โ โ โ โโโ Marriage cutoff: U18 = first union before age 18 (SDG 5.3.1)
โ โ โ โโโโโโโ Indicator class: MRD = ever-married / in-union
โ โ โโโโโโโโโโโโโ Age band (cohort): 20-24 = women 20-24 (retrospective denominator)
โ โโโโโโโโโโโโโโโ Population restriction: F = female-only indicator (women/girls)
โโโโโโโโโโโโโโโโโโ Family prefix: PT = Child Protection
Read out loud: "Child Protection / female respondents / cohort aged 20-24 /
ever-married / before age 18" โ i.e. the share of women aged 20-24 who were first
married before 18. Note that the F here is not a sex-disaggregation suffix
appended to a sex-neutral parent; it is a population-restriction marker built
into a family of indicators that only exist for female respondents (FGM, child
marriage, anaemia in women, antenatal care). See "Population restrictions vs sex
disaggregation" below.
The same left-to-right reading applies to the nutrition anthropometric pattern:
NT_ANT_HAZ_NE2
โ โ โ โ
โ โ โ โโโ Threshold: NE2 = below -2 SD (NE = "negative end", 2 = SD count)
โ โ โโโโโโโ Metric: HAZ = Height-for-Age Z-score
โ โโโโโโโโโโโ Sub-family: ANT = Anthropometry
โโโโโโโโโโโโโโ Family: NT = Nutrition
NT_ANT_HAZ_NE2 is the stunting prevalence indicator for children under 5. To
get the sex-stratified value, query the indicator with the SEX dimension
filter (get_data(indicator='NT_ANT_HAZ_NE2', sex='F')), not by appending _F
to the code โ see the next subsection.
And the derived-metric pattern, which the v1.1.1 query-aware scoring is
specifically tuned for:
TRGT_2030_IM_DTP3
โ โ โ โ
โ โ โ โโโโโ Antigen / dose: DTP3 = third dose of DTP
โ โ โโโโโโโโ Family: IM = Immunization
โ โโโโโโโโโโโโโ Target year: 2030 = SDG horizon
โโโโโโโโโโโโโโโโโโ Derived metric: TRGT = country-set target value
The combinatorial reading โ family โ sub-family โ metric โ threshold/age band โ
disaggregation token โ is the unwritten contract behind almost every code in the
catalog. The tables below enumerate the parts.
Topic prefixes (families)
The first segment is the topical family. The top 10โ12 prefixes in the catalog
cover the vast majority of child-focused indicators:
| Prefix | Meaning | Example code |
|---|
CME | Child Mortality Estimates โ mortality rates by age bracket (neonatal, infant, under-5, childhood, stillbirth) | CME_MRY0T4 (under-5 mortality), CME_SBR (stillbirth rate) |
NT | Nutrition โ anthropometry, anaemia, birthweight, micronutrients | NT_ANT_HAZ_NE2 (stunting), NT_BW_LBW (low birthweight) |
ED | Education โ completion rates, literacy, attendance by ISCED level | ED_CR_L1 (primary completion), ED_15-24_LR (youth literacy) |
WS | WASH โ water, sanitation, hygiene; population-level access | WS_PPL_W-SM (safely managed water) |
IM | Immunization โ vaccine coverage by antigen / dose | IM_BCG, IM_DTP3, IM_MCV1 |
MNCH | Maternal, Newborn & Child Health โ antenatal care, skilled birth attendance, early childbearing | MNCH_ANC1, MNCH_SAB, MNCH_BIRTH18 |
ECD | Early Childhood Development โ learning materials, parental stimulation, attendance | ECD_CHLD_LMPSL (learning materials) |
PT | Child Protection โ FGM, child marriage, violent discipline | PT_F_20-24_MRD_U18 (married before 18) |
HVA | HIV/AIDS โ ART coverage, prevalence, adolescent indicators | HVA_ADOL_ART_RECEIVE |
PV | Child poverty โ monetary and multidimensional | PV family |
COD | Causes of death โ disease- and condition-specific mortality / morbidity | COD_ACUTE_HEPATITIS_A |
DM | Demography โ household composition, population structure | DM_HH_U18 (households with member under 18) |
MG | Migration โ child migrants, displacement | MG family |
GN | Gender / adolescent girls (often cross-cuts NT) | GN_ANEMIA_ADOL_GRL |
TRGT | Country-set targets (see "Derived metrics" below) | TRGT_2030_* |
HAZARD | Hazard exposure (climate, conflict, disaster) | HAZARD family |
The full mapping of natural-language phrases to these prefixes lives in
indicator_resolver._SYNONYMS โ e.g. "stunting" โ NT_ANT_HAZ_NE2, "child
marriage" โ PT_F_20-24_MRD_U18, "BCG" โ IM_BCG.
Methodology and provenance suffixes
A trailing segment often signals how the value was produced, not what was
measured. These suffixes are central to the v1.1.1 query-aware scoring because
users almost always want either "survey" or "modelled" but rarely both.
| Suffix | Meaning | Example code |
|---|
_MOD | Modelled estimate (joint-estimation group output, e.g. UIS for education, IGME for mortality) | ED_CR_L1_UIS_MOD (modelled primary completion) |
_MERGE | Merged across multiple source surveys / years into a single comparable series | ECD_CHLD_LMPSL_MERGE |
_PRXY | Proxy indicator โ a related variable used in lieu of the conceptually exact one | ECD_CHLD_LMPSL_PRXY |
_NEW | New / revised methodology series (often runs alongside a legacy series for one cycle) | *_NEW |
_NUMTH | Numerator / threshold variant โ alternate cut used by some agencies | *_NUMTH |
_AGG | Aggregate (regional or income-group rollup rather than country observation) | *_AGG |
_UIS | UNESCO Institute for Statistics source / methodology | ED_CR_L1_UIS_MOD |
The MCP's differentiator.py:_SUFFIX_MEANINGS table is the canonical source.
When two codes differ only in this trailing segment, explain_difference()
reports them as base vs. variant (e.g. ECD_CHLD_LMPSL as base, _MERGE
and _PRXY as derivation variants).
Population restrictions vs sex disaggregation
This distinction is easy to miss and routinely trips up downstream agents.
Population restriction (built into the code). Some UNICEF indicators only
exist for one sex because the underlying measurement only applies to one sex:
FGM prevalence, child-marriage cohorts, anaemia in women of reproductive age,
antenatal care coverage. In those families, an F segment near the start of
the code (PT_F_*, NT_ANE_WOM_*, MNCH_BIRTH18, antenatal-care codes) is a
population-restriction marker that is part of the indicator's identity.
There is no PT_M_20-24_MRD_U18 counterpart for child marriage; the indicator
is defined on female respondents. Treat the F here as part of the indicator
name, not as a disaggregation switch.
Sex disaggregation (SEX dimension filter, at query time). For indicators
that are defined on both sexes (under-5 mortality, primary completion, stunting,
literacy), sex is not encoded in the code. It is a separate SDMX dimension โ
the SEX dimension โ with values F, M, _T (total). You select a slice
at query time:
get_data(indicator='CME_MRY0T4', sex='F')
get_data(indicator='CME_MRY0T4', sex='M')
get_data(indicator='CME_MRY0T4', sex='_T')
The differentiator.py:_SUFFIX_MEANINGS table lists F, M, _T, MF as
sex-token meanings; those entries describe the values of the SEX
dimension, not a suffix you append to an arbitrary code. Appending _F to a
code that does not already carry it (e.g. inventing NT_ANT_HAZ_NE2_F) will
not resolve โ the catalog does not contain such codes.
Age / wealth / residence disaggregation tokens
Disaggregation tokens appear either embedded in the code (when they are part of
the canonical definition, e.g. PT_F_20-24_MRD_U18) or applied at query time
via get_data() filters. The tokens below are the same in both places.
Age bands
Age is encoded as LOW-HIGH (inclusive) in the embedded form, or as a bound
token at the end of the code.
| Token | Meaning |
|---|
15-19, 20-24, 15-49 | Inclusive age band in years |
Y0 | Under 1 year (neonatal / infant variants) |
Y0T4 | 0 through 4 years inclusive (under-5) |
Y1T4 | 1 through 4 years inclusive (childhood, post-infant) |
U5, U15, U18 | "Under" threshold โ strictly below the named age |
ADOL | Adolescent (10โ19, occasionally 10โ24) |
Wealth quintile (query-time filter)
| Token | Meaning |
|---|
Q1 | Lowest quintile (poorest 20%) |
Q2, Q3, Q4 | Middle quintiles |
Q5 | Highest quintile (richest 20%) |
B20 / B40 | Bottom 20% / bottom 40% |
T20 | Top 20% |
Residence (query-time filter)
| Token | Meaning |
|---|
_T | Total |
U | Urban |
R | Rural |
Anthropometric Z-score thresholds
These appear in the NT_ANT_* family and need their own table because the
convention is non-obvious:
| Token | Meaning |
|---|
NE2 | Below -2 SD ("negative end, 2 SD") โ moderate-or-severe form |
NE3 | Below -3 SD โ severe form |
NE2_T_NE3 | Between -3 SD and -2 SD โ moderate-only form |
PO2 | Above +2 SD โ overweight side |
HAZ / WAZ / WHZ / BAZ | Height-for-age / Weight-for-age / Weight-for-height / BMI-for-age Z-scores |
Derived metrics: TRGT_ / _ARR_ / _PRJ
Three special grammars flag values that are not observations but
transformations of them. These are the cases where v1.1.1's query-aware
CURATED_PREFERRED scoring matters most. Note that ARR is a middle-segment
token (e.g. CME_ARR_U5MR, CME_ARR_SBR), not a trailing suffix.
| Pattern | Meaning | Example |
|---|
TRGT_<year>_<indicator> | Country-set target value for the named indicator at the named horizon year. TRGT_2030_IM_DTP3 is the country's 2030 target for DTP3 coverage, not the observed value. | TRGT_2030_IM_DTP3 |
*_ARR_* | Annual Rate of Reduction โ annualised percentage change derived from the underlying series; appears as a middle-segment token | CME_ARR_U5MR, CME_ARR_SBR |
*_PRJ / *_PRJ_* | Projected / forecast value rather than observed | PT_F_20-24_MRD_U15_PRJ |
v1.1.1 query-aware scoring (commit 7112e1d): the MCP only surfaces
TRGT_* codes when your natural-language query mentions target, goal,
objective, or a horizon year. A bare "DTP3 coverage in Brazil" question will
resolve to the observation series; "Brazil's 2030 DTP3 target" will resolve to
TRGT_2030_IM_DTP3. This is implemented as a hint field on CURATED_PREFERRED
entries in differentiator.py, not as a hard exclusion โ the target code is
still discoverable via direct lookup, just demoted in resolver ranking unless
the query signals intent.
The same query-aware demotion applies to _MOD (modelled) variants: queries
mentioning survey, raw, or observed prefer the un-suffixed code; queries
mentioning modelled, estimate, joint estimation prefer _MOD.
Education levels
In the ED_* family, ISCED levels are abbreviated L1 / L2 / L3:
| Token | ISCED level | Conventional name |
|---|
L1 | ISCED 1 | Primary education |
L2 | ISCED 2 | Lower-secondary education |
L3 | ISCED 3 | Upper-secondary education |
Examples: ED_ANAR_L1 (adjusted net attendance rate, primary), ED_ANAR_L2
(lower-secondary), ED_CR_L1 (completion rate, primary). Where a code stops at
L1 it is primary-only; where two levels are reported jointly the codes are
listed separately rather than concatenated.
Scope caveat. The L<n> = ISCED <n> mapping holds only inside the
ED_* family. Outside ED_*, L<n> may encode a non-ISCED level โ for
example PV_CHLD_MPI_L1 and PV_CHLD_MPI_L2 use L1 / L2 to mean severe
and moderate multidimensional-poverty deprivation respectively. Some ED_*
codes also use the two-digit form L01 / L02 to encode early-childhood or
pre-primary levels that sit below ISCED 1. Always check the indicator's name
before assuming L<n> means primary / lower-secondary / upper-secondary.
Where this is encoded in the MCP
The conventions documented above are not folklore โ they are encoded in the
MCP source and can be audited directly:
src/unicefstats_mcp/differentiator.py is the canonical reference for
segment-level meaning. The _SUFFIX_MEANINGS table maps every recognised
trailing token to a human-readable gloss; explain_difference() walks two
codes side-by-side and labels each diverging segment; the CURATED_PREFERRED
table carries per-indicator dimension_hint strings that drive the v1.1.1
query-aware scoring.
src/unicefstats_mcp/indicator_resolver.py maps natural-language phrases
to canonical codes. _SYNONYMS covers the routine cases ("stunting",
"under-5 mortality", "child marriage"); _AMBIGUOUS flags phrases that
legitimately map to more than one code and require user disambiguation;
_DISAMBIGUATION_TIPS carries the short hints surfaced in the
assistant_guidance envelope field added in v1.1.0.
unicef://glossary MCP resource โ clients that load resources at session
start get the disaggregation-code and indicator-prefix legend without having
to parse this README.
When the resolver picks a code, the chain is: natural-language query โ
indicator_resolver lookup โ if ambiguous, _AMBIGUOUS hit fires the v1.0.0
ambiguity_flag + candidate list โ if a CURATED_PREFERRED entry matches, the
dimension_hint re-ranks candidates โ the chosen code is annotated by
differentiator.py for the assistant_guidance field. Every step is
inspectable in the source; nothing in this section is heuristic on the LLM side.
Demo
Step 1: Search for indicators
>>> search_indicators("stunting", limit=3)
{
"query": "stunting",
"total_matches": 11,
"showing": 3,
"results": [
{"code": "FD_STUNTING", "name": "Moderate and severe stunting (Functional difficulties)"},
{"code": "NT_ANT_HAZ_NE2", "name": "Height-for-age <-2 SD (stunting)"},
{"code": "NT_ANT_HAZ_NE3", "name": "Height-for-age <-3 SD (severe stunting)"}
],
"tip": "Use get_indicator_info('FD_STUNTING') for full details including available disaggregations."
}
>>> get_indicator_info("CME_MRY0T4")
{
"code": "CME_MRY0T4",
"name": "Under-five mortality rate",
"description": "Probability of dying between birth and exactly 5 years of age, expressed per 1,000 live births",
"dataflow": "GLOBAL_DATAFLOW",
"sdmx_api": "https://sdmx.data.unicef.org/ws/public/sdmxapi/rest/data/UNICEF,GLOBAL_DATAFLOW,1.0/.CME_MRY0T4?format=csv",
"disaggregation_filters": {
"sex": ["_T (Total)", "M (Male)", "F (Female)"],
"wealth_quintile": ["Q1 (Lowest)", "Q2", "Q3", "Q4", "Q5 (Highest)"],
"residence": ["_T (Total)", "U (Urban)", "R (Rural)"]
}
}
Step 3: Check temporal coverage
>>> get_temporal_coverage("CME_MRY0T4")
{
"code": "CME_MRY0T4",
"start_year": 1931,
"end_year": 2024,
"latest_year": 2024,
"countries_with_data": 249,
"note": "Not all countries have data for all years. Coverage varies by country."
}
Step 4: Fetch data
>>> get_data("CME_MRY0T4", ["BRA", "IND", "NGA"], start_year=2018, end_year=2023)
{
"indicator": "CME_MRY0T4",
"countries_requested": ["BRA", "IND", "NGA"],
"total_rows_available": 18,
"rows_returned": 18,
"rows_truncated": false,
"format": "compact",
"summary": {
"value_range": {"min": 14.42, "max": 117.56, "mean": 54.78},
"year_range": {"earliest": 2018, "latest": 2023},
"countries_in_result": 3
},
"data": [
{"iso3": "BRA", "country": "Brazil", "period": 2018, "indicator": "CME_MRY0T4", "value": 15.22},
{"iso3": "BRA", "country": "Brazil", "period": 2019, "indicator": "CME_MRY0T4", "value": 15.03},
{"iso3": "BRA", "country": "Brazil", "period": 2020, "indicator": "CME_MRY0T4", "value": 14.87},
{"iso3": "BRA", "country": "Brazil", "period": 2021, "indicator": "CME_MRY0T4", "value": 14.72},
{"iso3": "BRA", "country": "Brazil", "period": 2022, "indicator": "CME_MRY0T4", "value": 14.59},
{"iso3": "BRA", "country": "Brazil", "period": 2023, "indicator": "CME_MRY0T4", "value": 14.42},
{"iso3": "IND", "country": "India", "period": 2018, "indicator": "CME_MRY0T4", "value": 36.87},
{"iso3": "IND", "country": "India", "period": 2019, "indicator": "CME_MRY0T4", "value": 34.86},
{"iso3": "IND", "country": "India", "period": 2020, "indicator": "CME_MRY0T4", "value": 32.98},
{"iso3": "IND", "country": "India", "period": 2021, "indicator": "CME_MRY0T4", "value": 31.19},
{"iso3": "IND", "country": "India", "period": 2022, "indicator": "CME_MRY0T4", "value": 29.53},
{"iso3": "IND", "country": "India", "period": 2023, "indicator": "CME_MRY0T4", "value": 27.99},
{"iso3": "NGA", "country": "Nigeria", "period": 2018, "indicator": "CME_MRY0T4", "value": 117.19},
{"iso3": "NGA", "country": "Nigeria", "period": 2019, "indicator": "CME_MRY0T4", "value": 117.37},
{"iso3": "NGA", "country": "Nigeria", "period": 2020, "indicator": "CME_MRY0T4", "value": 117.42},
{"iso3": "NGA", "country": "Nigeria", "period": 2021, "indicator": "CME_MRY0T4", "value": 117.56},
{"iso3": "NGA", "country": "Nigeria", "period": 2022, "indicator": "CME_MRY0T4", "value": 117.46},
{"iso3": "NGA", "country": "Nigeria", "period": 2023, "indicator": "CME_MRY0T4", "value": 116.82}
]
}
Key insights an AI assistant would extract from this:
- Brazil: 14.4 per 1,000 โ steadily declining, on track for SDG 3.2 target (โค25)
- India: 28.0 per 1,000 โ rapid improvement (37โ28 in 5 years), recently crossed SDG target
- Nigeria: 117 per 1,000 โ essentially flat, 4.7ร the SDG target, highest burden
Step 5: Get code template to continue in a script
>>> get_api_reference("r", "unicefData")
{
"language": "r",
"install": "install.packages(\"unicefdata\")",
"import": "library(unicefdata)",
"function": "unicefData",
"signature": "unicefData(\n indicator = NULL, # character โ indicator code(s)\n countries = NULL, # character vector โ ISO3 codes, NULL = all\n year = NULL, # numeric, character (\"2015:2023\"), or vector\n sex = \"_T\", # character โ \"_T\", \"M\", \"F\"\n totals = FALSE, # logical โ only return aggregate totals\n tidy = TRUE, # logical โ standardize column names\n country_names = TRUE, # logical โ add country name column\n format = \"long\", # character โ \"long\", \"wide\", \"wide_indicators\"\n latest = FALSE, # logical โ most recent value per country\n circa = FALSE, # logical โ closest available year\n add_metadata = NULL, # character vector โ e.g. c('region', 'income_group')\n dropna = FALSE, # logical โ drop rows with missing values\n simplify = FALSE, # logical โ minimal columns\n mrv = NULL, # integer โ most recent N values per country\n raw = FALSE, # logical โ all disaggregations, no filtering\n)",
"returns": "tibble with columns: indicator_code, iso3, country, period, value, sex, age, wealth_quintile, residence, ...",
"examples": [
{"description": "Under-5 mortality for Brazil, India, Nigeria (2015โ2023)", "code": "df <- unicefData(\"CME_MRY0T4\", countries = c(\"BRA\", \"IND\", \"NGA\"), year = \"2015:2023\")"},
{"description": "Latest stunting data for all countries", "code": "df <- unicefData(\"NT_ANT_HAZ_NE2\", latest = TRUE)"},
{"description": "Wide format with region metadata", "code": "df <- unicefData(\"CME_MRY0T4\", format = \"wide\", add_metadata = c(\"region\", \"income_group\"))"}
]
}
This lets the AI generate correct R/Python/Stata code using the exact parameter names and syntax โ no guessing from training data.
get_data parameters
| Parameter | Type | Default | Description |
|---|
indicator | str | required | Indicator code |
countries | list[str] | required | ISO3 codes (max 30) |
start_year | int | None | Start of year range |
end_year | int | None | End of year range |
sex | str | "_T" | "_T" (total), "M" (male), "F" (female) |
wealth_quintile | str | None | "Q1"โ"Q5", "B20", "B40", "T20" |
residence | str | None | "U" (urban), "R" (rural), "_T" (total) |
format | str | "compact" | "compact" (5 cols) or "full" (all cols) |
limit | int | 200 | Max rows (1โ500) |
Response features
summary: Value range (min/max/mean), year range, country count
disaggregations_in_data: Which dimensions have non-trivial variation
total_rows_available vs rows_returned: Pagination metadata
tip: Contextual guidance for next steps or narrowing results
Prompts
compare_indicators
Pre-built analysis workflow: fetches indicator metadata and data, then produces a structured comparison.
compare_indicators(indicator="CME_MRY0T4", countries="BRA,IND,NGA", start_year="2015", end_year="2023")
write_unicefdata_code
Generate runnable Python, R, or Stata code using the unicefdata package. The AI will call get_api_reference() to get the exact function signatures, then write code matching the user's task.
write_unicefdata_code(
task="Compare under-5 mortality for Brazil and India, 2015-2023, then plot the trends",
language="r"
)
This bridges the gap between conversational exploration (via MCP tools) and reproducible analysis scripts (via unicefdata packages).
Benchmark Results
We benchmarked the MCP against a bare LLM (Claude Sonnet 4, no tools) using the EQA metric from Azevedo (2025). 300 queries across 10 indicators, 20 countries, 2 prompt types, and 2 hallucination test categories.
Two benchmark pillars
The repo maintains two empirical scoreboards:
- 300-query EQA test (10 indicators ร 20 countries ร 2 prompts ร 2 hallucination categories) โ the original Azevedo (2025) benchmark used through v1.0.x / v1.1.x. Numbers below labelled "v0.7.3 + fixes".
- v9 Arm B 1484-cell paired benchmark โ larger, more granular, introduced during the v1.5.x curated-defaults cycle. Compares each MCP vintage cell-by-cell against the v1.2.4 Jun 1 baseline. v1.5.5 measured mean EQA 0.749 (+29.1pp, p = 2ร10โปยนโฐโต).
The two are scored under different rubrics on different samples; they are complementary, not directly comparable.
Current canonical numbers โ 300-query EQA test (v0.7.3 + fixes, May 2026)
The numbers below are the current canonical scoreboard. They reflect (a) four engineering fixes in the v0.7.3 cycle (see CHANGELOG) and (b) a scoring correction from the v1.4 extractor that respects refusal language. Both samples (mcp060: 40 countries; mcp073: disjoint 20-country validation sample) score under the same v1.4 rules.
| Metric | LLM alone (no tools) | LLM + MCP (v0.7.3 + fixes) | Sample |
|---|
| POS EQA mean | 0.121 | 0.891 | mcp060 (40 ctry) |
| POS EQA mean | 0.121 | 0.909 | mcp073 (20 disjoint ctry) |
| hall_b combined (T1+T2) | 2.50% (hall_a) | 1.00% | mcp060 |
| hall_b combined (T1+T2) | 2.50% (hall_a) | 2.25% | mcp073 |
| MCP makes model safer (hall_b < hall_a) | โ | Yes โ both samples | both |
v0.7.3 + fixes is the first version where MCP demonstrably makes the model safer than the no-tools baseline on absent-data queries. Through v0.7.2, hall_b โฅ hall_a โ the safety layer was reducing magnitude but not direction. The four fixes that flipped the property:
server.py:_seed_data_frontier_cache โ monotonic max instead of unconditional overwrite (a probe-induced regression of the cached frontier was telling the LLM that current data was unavailable).
server.py:get_data exception handler โ unicefdata cascade exhaustion reclassified as no_data (not error), so the LLM treats it as authoritative absence rather than tool failure.
benchmark_eqa_batch.py โ persist tool result_str on state.tool_calls so the v1.4 extractor can see refusals.
benchmark_eqa_batch.py โ refusal-respect parity with the sync runner.
Canonical scoreboard: see the [Unreleased] ยงFixed block at the top of CHANGELOG.md for the full cross-version table, including the four post-fix corrections and the mcp073 second-sample validation. Full per-run write-ups live in internal/v0_7_3_validation.md and internal/v0_7_3_second_sample_validation.md in the dev repo (jpazvd/unicefstats-mcp-dev, dev-only and not synced to this public mirror).
Historical: v0.3.0 (n=600, 2025) and v0.7.2 (n=500 same-day, 2026-05-08)
The original v0.3.0 benchmark reported POS EQA 0.147 โ 0.990 (6.7ร) and T2 hallucination 11% โ 37%. Two corrections since:
- The "37%" headline was substantially a v1.3 extractor scoring artefact: the extractor counted any numeric value mentioned in the response as a "claim," including values quoted from
no_data tool results inside an explicit refusal. The v1.4 extractor (_detect_refusal) reclassifies those as appropriate refusals. We did not rescore the v0.3.0 parquets under v1.4 (they're archived); the v1.4-equivalent of "37%" is unknown but substantially lower.
- The v0.7.2 reproduction (n=500, 2026-05-08) under v1.3 scoring reported POS EQA 0.897 and combined T1+T2 hallucination 13%. Under v1.4 scoring the same parquets report POS EQA 0.793 and hall_b 3.75%.
The accuracy headline (~7ร lift) has held up across every rescoring. The hallucination headline required both a scoring fix (v1.4 extractor) and a server fix (v0.7.3 cache + cascade fixes) before MCP actually reduced hallucination below the no-tools baseline.
EQA decomposition (baseline_latest prompt)
| Component | LLM alone | LLM + MCP | Gain |
|---|
| ER (extraction rate) | 0.50 | 1.00 | +0.50 |
| YA (year accuracy) | 0.24 | 0.99 | +0.75 |
| VA (value accuracy) | 0.37 | 1.00 | +0.63 |
| EQA = ER ร YA ร VA | 0.147 | 0.990 | +0.843 |
Key findings
-
All 10 indicators at EQA >= 0.95 with MCP, replicated across 40 countries (R1 + R2 with zero overlap). 7 of 10 achieve perfect EQA = 1.000.
-
Year accuracy is the bare LLM's biggest weakness (YA = 0.24). It cites 2021-2022 as "latest" when IGME 2024 estimates exist. The MCP queries the API and returns the actual latest year.
-
The direct prompt shows larger MCP gain (+0.722 vs +0.613) because it eliminates YA and isolates pure retrieval accuracy.
-
T2 hallucination (~37%) is inflated by ground truth misclassification: the SDMX API has IGME mortality data for micro-states that the ground truth pipeline missed. After correction: MCP ~10%, LLM alone ~5%. The remaining hallucination is driven by the confidence effect โ Claude overrides tool errors when it has strong domain priors.
-
The confidence effect: When the MCP tool returns "no data" but the LLM has strong domain priors (e.g., child mortality for well-known countries), it overrides the tool and fabricates anyway. This is a fundamental LLM behavior, not MCP-specific.
With and without the server
| Metric | LLM alone | unicefstats-mcp (v0.7.3 + fixes) |
|---|
| EQA (POS) | 0.121 | 0.891 |
| hall_b combined (T1+T2) | hall_a 2.50% | 1.00% (mcp060) / 2.25% (mcp073) |
| MCP safer than no-tools? | โ | Yes |
| Cost per query | ~$0.003 | ~$0.04 |
| Avg latency | ~5s | ~10s |
This table used to carry a third column for sdmx-mcp. It was withdrawn on 15 November 2026: see Relationship to sdmx-mcp.
v9 Arm B paired benchmark (1484 cells, Jun 2026)
The v1.5.x cycle introduced a second, larger benchmark โ v9 Arm B, 1484 cells, master_seed=42 โ that compares each MCP vintage cell-by-cell against a fixed v1.2.4 baseline. Unlike the 300-query EQA test (which scores indicator + year + value extraction quality), v9 Arm B measures paired ฮEQA so it can isolate the contribution of each vintage's changes.
| vintage | date | mean EQA | ฮ vs v1.2.4 | p-value |
|---|
| v1.2.4 | Jun 1 2026 | 0.458 | โ | โ |
| v1.4.0 | Jun 4 2026 | 0.481 | +2.3pp | 0.0003 |
| v1.5.1 | Jun 6 2026 | 0.681 | +22.3pp | 7ร10โปโทโถ |
| v1.5.5 | Jun 7 2026 | 0.749 | +29.1pp | 2ร10โปยนโฐโต |
- 4 of 5 decision-rule gates PASS for v1.5.5
- 430 FIRE_RECOVERABLE cells lift at +0.97 EQA each
- Lost:rescued ratio 0.03 (vs v1.4.0's 0.41)
- Cascade-attributable lift on FIRE_ALTERED: +0.80 [+0.77, +0.83]
This benchmark is what validates the +29.1pp claim. The 300-query EQA test above still applies and remains the canonical reference for hallucination-rate measurement.
Full analysis, per-indicator decomposition, and methodology: examples/RESULTS.md
Benchmark data (parquet with full LLM responses): examples/results/
Benchmark design rationale: examples/DESIGN_ISSUES.md
Reproducing the benchmark
python examples/00_build_ground_truth.py
python examples/benchmark_eqa.py
python examples/01_run_direct_supplement.py
Citation
This benchmark uses the EQA metric from:
Azevedo, J.P. (2025). "AI Reliability for Official Statistics: Benchmarking Large Language Models with the UNICEF Data Warehouse." UNICEF Chief Statistician Office. github.com/jpazvd/unicef-sdg-llm-benchmark-dev
Deployment
Local (stdio)
Remote (SSE)
unicefstats-mcp --transport sse --port 8000
Docker
docker build -t unicefstats-mcp .
docker run -p 8000:8000 unicefstats-mcp
Environment variables
The MCP runs with no required configuration. Two opt-in environment variables (v1.5.5+) help large-batch / benchmark use survive UNICEF SDMX's WAF / per-IP rate-limiter:
| variable | default | effect when set |
|---|
UNICEFSTATS_SDMX_THROTTLE_MS | 0 (off) | Minimum inter-call delay (ms) between SDMX unicefData() calls in this process. Recommended 200โ500 for batch use so the request rate stays below the empirically-observed ~50-100 calls/min WAF trigger threshold. |
UNICEFSTATS_SDMX_RETRY_403 | unset | When set to 1 / true / yes / on, retries SDMXForbiddenError (HTTP 403) with exponential backoff (30s, 60s, 120s; max 3 retries) instead of failing immediately. Lets a long-running batch survive transient WAF bursts. |
For normal interactive MCP server use (single-query conversational tasks via Claude / Cursor / etc.), leave both unset โ defaults preserve fast-fail behaviour.
For benchmark / batch use (1000+ sequential queries from one IP):
export UNICEFSTATS_SDMX_THROTTLE_MS=300
export UNICEFSTATS_SDMX_RETRY_403=1
Development
pip install -e ".[dev]"
pytest tests/ -v
ruff check src/ tests/
mypy src/unicefstats_mcp/
Contributing
Contributions are welcome.
Ways to contribute
- Bug reports: Open an issue with steps to reproduce
- Feature requests: Suggest new tools, indicators, or output formats via issues
- Code: Fork, branch, submit a PR โ see development setup below
- Benchmark: Run the EQA benchmark on different models and share results
- Documentation: Improve examples, fix typos, add use cases
Development setup
git clone https://github.com/jpazvd/unicefstats-mcp.git
cd unicefstats-mcp
pip install -e ".[dev,benchmark]"
pytest tests/ -v
ruff check src/ tests/
mypy src/unicefstats_mcp/
Pull request guidelines
- One concern per PR โ keep changes focused and reviewable
- Include tests for new tools or bug fixes
- Run the linter (
ruff check) and type checker (mypy) before submitting
- Update the README if you change tool signatures or add new features
- Do not commit API keys or benchmark result parquets larger than 500KB
Priority areas
See the audit findings for known issues. High-impact areas:
- MNCH dataflow bug:
MNCH_CSEC and MNCH_BIRTH18 return 0 EQA due to a dataflow resolution issue in the unicefdata package
- T2 hallucination reduction: Further reduce fabrication when API returns no results (currently ~10%; see Limitations)
Limitations and Hallucination Risks
Data limitations
- Coverage is uneven across indicators, countries, and years. Survey-based indicators (nutrition, education, protection) have 3-5 year gaps between data points by design.
- Mortality indicators (CME_*) are modeled estimates from the UN Inter-agency Group (IGME), with uncertainty intervals not surfaced in compact output.
- Not all indicators support all disaggregation dimensions;
get_indicator_info() lists what's available per indicator.
get_data() caps at 500 rows per call.
Hallucination risks
Benchmark testing across multiple country samples (v0.7.3 + fixes, May 2026):
| Type | Description | Rate (LLM alone, hall_a) | Rate (LLM + MCP, hall_b) | Notes |
|---|
| T1 (gap-year) | LLM cites a value for a year when the indicator has data but not for that specific year | ~1.0% | 0.00% (mcp060) / 0.50% (mcp073) | Below the no-tools rate |
| T2 (forward-of-frontier) | LLM fabricates a value for a year beyond the data frontier | ~3.5% | 2.00% (mcp060) / 4.00% (mcp073) | Below the no-tools rate |
| Combined | T1 + T2 | hall_a 2.50% | 1.00% (mcp060) / 2.25% (mcp073) | hall_b < hall_a โ MCP makes safer |
v0.7.3 + fixes is the first release where MCP makes the model safer than the no-tools baseline. Through v0.7.2 (v1.4 scoring), hall_b โฅ hall_a โ the safety layer was reducing magnitude relative to a no-safety-layer baseline but never enough to put MCP under the no-tools floor.
What changed:
- Cache contamination in
_seed_data_frontier_cache โ a probe routine was overwriting the cached max-year for an indicator with whatever year happened to come back, sometimes lower than the cached value. The LLM was being told frontiers were 2018 for indicators whose real frontier was 2023, then answering about 2023 from parametric memory. Fix: monotonic max() only.
unicefdata cascade as no_data, not error โ the underlying wrapper's fallback-dataflow exhaustion was leaking as a tool exception, which the LLM read as "tool failed, fall back to my own knowledge." Now classified as a clean no_data signal that the safety layer converts into "do not estimate." Filed unicefdata-dev#74 for an upstream fix.
- v1.4 extractor (
_detect_refusal) โ corrected the long-standing scoring artefact in which any numeric value in a response was counted as a "claim," including values the LLM was quoting from no_data tool results inside an explicit refusal. This dropped hall_b in the v0.7.3 PRE-FIX rescoring from ~37% to 2.25% โ most of the historical "MCP-makes-it-worse" finding was extractor, not behaviour.
- Batch runner parity โ
result_str persistence and refusal-respect in the async batch runner.
This finding still leaves room for the broader tool-augmented LLM literature on the structural cost of giving models an answer-producing pathway:
- The Reasoning Trap: How Enhancing LLM Reasoning Amplifies Tool Hallucination (ICLR 2025) โ shows the relationship is causal: as models get better at tool use, tool hallucination rises proportionally with capability.
- Reducing Tool Hallucination via Reliability Alignment (Cao et al., 2024, arXiv:2412.04141) โ formalises the failure as tool-selection errors (wrong tool, failed refusal) and tool-usage errors (fabricated parameters).
- ReDeEP: Detecting Hallucination in Retrieval-Augmented Generation via Mechanistic Interpretability (Sun et al., 2024) โ shows mechanistically that an LLM's parametric knowledge can override retrieved context inside the residual stream.
These results show the structural tendency is real; the v0.7.3 + fixes result shows it can be reversed for a specific data domain through (a) safety-layer architecture, (b) server-side discipline (no stale frontier cache, no leaking of no-data as tool errors), and (c) honest scoring.
The takeaway for users:
- Load the
unicef://system-prompt and unicef://context resources at session start (handles forward-of-frontier fabrication).
- Treat MCP results as best-effort retrieval, not infallible truth โ verify load-bearing values against the UNICEF Data Warehouse before citing.
- Prefer queries with explicit years ("under-five mortality in Nigeria in 2023") over open-ended ones ("the latest under-five mortality in Nigeria") โ the former triggers refusal more reliably when data is absent.
- The 1.00% / 2.25% residuals were measured on Sonnet 4 only. Cross-model generalisation is the next benchmark.
Full benchmark methodology: examples/RESULTS.md
Provenance and Ownership
All data served by this MCP originates from the UNICEF Data Warehouse, accessed live via the public SDMX REST API. No observation data is stored or cached โ every get_data() call results in a live SDMX request. The indicator and country registries are cached in memory at first access for performance; these are catalogue metadata, not statistical values. The MCP reformats output for LLM consumption but does not alter values.
All releases are published from GitHub Actions using PyPI Trusted Publishing (OIDC). No long-lived API tokens exist. Release provenance is verifiable via PyPI attestations.
For full details on data origin, ownership, distribution pipeline, and interpretation caveats, see PROVENANCE.md.
How to Verify This MCP
| Check | How |
|---|
| Source | Repository is jpazvd/unicefstats-mcp on GitHub |
| Package | pip show unicefstats-mcp โ verify Home-page points to the canonical repo |
| Version | python -c "import unicefstats_mcp; print(unicefstats_mcp.__version__)" โ compare with server.json and PyPI |
| Provenance | PyPI attestations link each release to a GitHub Actions workflow |
| Runtime | Call get_server_metadata() โ returns canonical name, version, publisher, and data source |
License
MIT