SmartCLI
Read this in: English · 简体中文 · 繁體中文 · 日本語 · 한국어
A local Python toolkit for driving, perceiving, and rendering the terminal — three agent skills over one pluggable PTY + pyte core.
Let an AI drive, perceive, and render real terminal programs. SmartCLI reads
the actual screen with a pyte cell model — not a byte pipe — so it knows which
menu row is highlighted, presses the right keys, and waits for the screen to
settle. Below: it drives the real lazygit TUI end-to-end (arrow-key
navigation, opening a commit diff, highlighting a branch) — no script, no mock.
pip install smartcli-toolkit
Requires Python 3.10 or newer. The install includes the shared Python library,
the persistent TUI driver, and the stdio MCP server.
Drive something in 30 seconds
Copy-paste this. It starts a real Python REPL under a PTY, waits for the prompt
(never a blind sleep), types into it, and reads the screen back:
pip install smartcli-toolkit
SID=$(smartcli-tui start --cmd "python3 -i -q" --cols 80 --rows 24 --json | python3 -c "import json,sys;print(json.load(sys.stdin)['sid'])")
smartcli-tui wait-regex --id $SID ">>> " --timeout-ms 15000
smartcli-tui send-line --id $SID "print(6*7)"
smartcli-tui wait-regex --id $SID "42"
smartcli-tui close --id $SID
On Windows use --cmd "py -i -q". Swap the command for vim, htop or
lazygit and the same five verbs drive those too — that is the whole point:
wait-regex and friends react to what the screen actually shows, so an agent
never guesses whether its keystroke landed.
Want the same thing against a real editor, end to end and verifiable?
examples/drive_vim.py drives the actual vim binary —
opens a file, appends a line, saves, and then checks the filesystem, not the
screen:
python examples/drive_vim.py
Note the fourth step. It is there because the example itself once sent five
keystrokes back to back with nothing between them, and under load vim had not
processed G by the time o arrived, so nothing was inserted and the run failed
with no useful diagnosis. Confirming insert mode proves both keys landed — the
same discipline the tool exists to provide, applied to its own demo.
Run the same file against smartcli-toolkit==0.1.8 and two steps fail — and the
file is never saved, because a driver that cannot see the alternate screen
mistimes the :wq. That is why the emulation work below matters: a wrong screen
model does not error, it silently succeeds at nothing.
Already running an MCP client (Claude Code, Cursor, VS Code)? The same verbs are
MCP tools, with the per-session token attached for you:
What & why
SmartCLI is a workspace for terminal work that agents and humans both do: driving
interactive terminal programs, perceiving what a screen actually shows, and
rendering visuals and layouts back out. It is built on one shared, pluggable PTY
backend plus a pyte screen model — chosen over screenshot/vision so a single
structured screen model feeds both perception (read the screen) and rendering
(draw the screen). The PTY layer is intentionally not tmux-bound: local dev runs
on Windows via ConPTY (pywinpty), while target programs can run under POSIX ptys
or tmux elsewhere. Three skills sit on that core, each a self-contained tool you run
in place from the checkout.
Driving a real TUI
The demo above is SmartCLI driving lazygit — a real full-screen curses app —
through its perceive → act → confirm loop: it reads the pyte cell grid (which
row is selected, the alt-screen diff), moves with arrow keys, opens a commit's
diff, and highlights a branch. Captured by driving the actual program in a Linux
container, not scripted or mocked. A byte-stream matcher like pexpect can't
perceive "which row is highlighted"; a screen model can.
How we know the perception is right. A screen model is only useful if it
matches what a real terminal shows, so we measure that instead of asserting it:
identical bytes go to a real tmux pane and to our model, and the two cell
grids are diffed. Three suites do it — 34 curated cases, a three-way check that
only trusts a behaviour when tmux and GNU screen agree, and a generative
fuzz over random VT sequences. That campaign found and fixed 12 emulation bugs,
including the alternate screen buffer (pyte implements none of modes
1049/1047/47, so a full-screen program's output used to be painted over the main
screen and never restored). Scope and remaining edges:
LIMITATIONS.md.
Live effects
Real captures of the cmd-art fx engine — each GIF is the actual effect
rendered frame-by-frame through the project's own pipeline (no screen recorder).
Reproduce any with python -m fx play <name> (see Quickstart).
solarsystem — an orrery: planets on elliptical orbits around a pulsing sun
| | |
|---|
 |  |  |
| donut — the classic ASCII torus | fire — demoscene heat field | rain — Matrix digital rain |
🌐 Explore the live showcase → — play with
the effect engine, drive a menu with arrow keys, and poke the widgets, right in
your browser.
Install
Just want the three Claude Code skills? Download one zip, unzip it, done — no
git, no pip, no marketplace:
curl -LO https://github.com/dwgx/SmartCLI/releases/latest/download/smartcli-skills.zip
unzip smartcli-skills.zip -d ~/.claude/skills/
That gives you cmd-art, drive-tui and tui-ui (309 KiB total). cmd-art and
tui-ui then work with nothing but CPython 3.10+ — verified on a bare virtualenv,
all 30 effects and all 17 widgets load. drive-tui additionally needs pyte, which
the PyPI install below provides. Or install all three via the plugin marketplace:
/plugin marketplace add dwgx/SmartCLI.
Primary — from PyPI (the library, the CLI, and the MCP server):
pip install smartcli-toolkit
Distribution vs import name: the PyPI distribution is smartcli-toolkit
(the names smartcli / smart-cli were taken or blocked), but the importable
package is smartcli_core. So after pip install smartcli-toolkit you still
write from smartcli_core import PtySession.
Alternative — reproduce the full dev environment from a source checkout:
git clone https://github.com/dwgx/SmartCLI SmartCLI
cd SmartCLI
python -m pip install -r requirements.txt
requirements.txt installs pyte, the MCP SDK, and pywinpty on Windows only
(POSIX uses the stdlib pty backend). pip install . installs smartcli_core
plus the smartcli-tui, smartcli-mcp, and smartcli-toolkit commands. The
visual cmd-art and tui-ui skills still run in place from a checkout via
python -m fx and python -m ui.
Optional extras (real FIGlet fonts, raster images, authoritative cell widths — all
degrade gracefully to stdlib fallbacks when absent):
python -m pip install -r requirements-optional.txt
pip install ".[all]"
pip install ".[art]"
pip install ".[image]"
pip install ".[width]"
Windows note: set UTF-8 output before running any skill so box-drawing and CJK
glyphs encode cleanly (the CLIs also auto-reconfigure stdout, but set this to be safe):
set PYTHONIOENCODING=utf-8
Verified dep versions on the dev box (Windows 11, CPython 3.14.6): pyte 0.8.2,
pywinpty 3.0.5, pyfiglet 1.0.4, Pillow 12.2.0, wcwidth 0.8.1.
Diagnostics. python -m smartcli_core prints your OS, Python, terminal, PTY
backend, and dependency versions. smartcli-tui doctor reports where the core
was loaded from and whether drive dependencies are present. Include both outputs
when filing a terminal-sensitive bug.
Quickstart
cmd-art — terminal visual effects
cd skills/cmd-art
python -m fx list
python -m fx play donut --seconds 5
python -m fx gallery
python -m fx show --seq "donut:fire:3,plasma::3"
tui-ui — cell-accurate terminal UI
cd skills/tui-ui
python -m ui widgets
python -m ui gallery --width 100 --height 30
python -m ui demo table --width 80 --height 12 --theme dashboard
drive-tui — perceive & drive interactive programs
Persistent-session CLI (state survives across shell calls):
smartcli-tui start --cmd "python3 -i -q" --cols 80 --rows 24
smartcli-tui wait-regex --id <SID> ">>> " --timeout-ms 15000
smartcli-tui send-line --id <SID> "print(6*7)"
smartcli-tui snapshot --id <SID>
smartcli-tui close --id <SID>
On Windows use py -i -q as the child command. From a source checkout, replace
smartcli-tui with python skills/drive-tui/scripts/tui.py.
Or drive from any MCP client — the same verbs as MCP tools, with the
per-session token attached automatically:
pip install smartcli-toolkit
smartcli-mcp
As a library
The shared core is importable directly:
import sys
from smartcli_core import PtySession
s = PtySession()
s.start([sys.executable, "-q"])
s.wait_for(r">>> ")
print(s.snapshot().to_text())
s.close()
For the full command reference, the screenshot/AGENTCLI harnesses, and the regression
suite, see README-USAGE.md.
Features
cmd-art (skills/cmd-art) — a "living-template" effect engine: an Effect ABC +
@register decorator + auto-discovery. 30 effects (donut, solarsystem, fire, plasma,
rain, starfield, tunnel, text3d, cube, sphere, boids, life, fireworks, sparkle, decrypt,
gradient_text, banner_scroll, image2ascii, typewriter, julia, mandelbrot, perlin, flames, water, nebula, text_flyin, text_converge, text_decrypt, spectrum_bars, cbonsai) across 8 themes (mono, fire,
ocean, synthwave, viridis, pastel, matrix-green, rainbow). Effects are pure frame
producers; play is bounded by default and always restores the terminal.
tui-ui (skills/tui-ui) — a web-like terminal layout engine emitting tmux-safe
ANSI frames (SGR color runs + newlines only; no cursor moves, no alt-screen). 17
widgets (badge, banner, braille_chart, card, fuzzy_filter_list, gradient_rule, kv,
meter, panel, preview_pane, progress, radial_glow, rule, slider_track, table, tabs,
tree) over an engine of two modules the render path actually calls — field.py
(shader compositors) and raster.py (sub-cell half/quad/braille pixels). Two
further helpers, box_junction.py and color_model.py, shipped unwired through
0.3.2 and were removed in 0.3.3: nothing imported them, and tui-ui has no
auto-discovery over ui/, so no caller could ever reach them.
Display-cell accurate for CJK/emoji/ZWJ so columns never desync.
drive-tui (skills/drive-tui) — drives interactive terminal programs (REPLs,
menus, pagers, y/N prompts, wizards) through a PTY via a
perceive → decide → act → wait → confirm loop, never a blind sleep. A thin CLI
(scripts/tui.py) offers a persistent detached session and a one-shot run mode, with
an importable pattern library of 8 recipes (repl, menu_select, pager, search_filter,
confirm, form, progress, wizard) that classify() a screen and drive() it.
Where the wait capabilities live. The wait family can end when the child
process dies rather than sitting out the remaining timeout — a program that
crashes on your first input then costs one poll cycle instead of the whole
ceiling. It is opt-in and off by default: start --detect-child-exit /
run --detect-child-exit, or detect_child_exit on the MCP start tool. Off
is exactly the previous behaviour. On, every wait reply carries exited, and
wait reports reason=EXITED. Three other core capabilities — the
screen-revision wait baseline, the terminal-mode registry, and the session event
log — are library-only: they are importable from smartcli_core and are
deliberately not on the CLI or MCP surface. See
CHANGELOG.md (Unreleased) for the reasoning per capability.
Shared core (smartcli_core) — the pluggable PTY backend + pyte screen model +
semantic snapshot + readiness sync (pty_backend / screen_model / snapshot / readiness / session). The reusable, importable foundation under all three skills. Since 0.3.0 a
budgeted read path and an io block travel with every observation, so a caller can tell
quiet from not read yet (local_cut, byte watermarks, pending facts — unknown values are
null, never 0), STABLE requires a drained observation, and close reports
closed_confirmed or close_unconfirmed with the last progress instead of assuming a
returned native call means the child is gone.
Knowledge graph (knowledge/) — a wiki-link graph (140+ .md files) of exact
rendering formulas, ANSI sequences, and measured constants, each note carrying a
source and cross-links. See knowledge/INDEX.md.
Project layout
SmartCLI/
smartcli_core/ shared PTY + pyte engine (importable package)
skills/cmd-art/ fx effect package and CLI (30 effects, 8 themes)
skills/drive-tui/ TUI pattern library and PTY driver CLI (8 recipes)
skills/tui-ui/ terminal UI layout engine and widgets (17 widgets)
tools/screenshot/ pyte -> PNG smoke-test harness
tools/agentcli/ agent-CLI control validation harness
knowledge/ wiki-link knowledge graph, 140+ .md files (see knowledge/INDEX.md)
showcase/ rendered effect PNGs + demo GIFs (shown above)
tests/ direct script-style regressions
research/ archived first-pass research notes
Documentation
README-USAGE.md — the full usage cheat-sheet: every skill,
the screenshot and AGENTCLI harnesses, and the regression commands.knowledge/INDEX.md — the knowledge graph (140+ .md files).AGENTCLI-VALIDATION.md — agent-CLI control test matrix.CHANGELOG.md — release history.- Verification evidence tree — archived 2026-10-01. The former
verification/
directory (receipts, probes, historical harness revisions) is no longer part
of this repository, and is not distributed with the package. It was moved
byte-for-byte, so nothing was deleted. It was pulled out because it held 8
stale copies of smartcli_core/screen_model.py taken before three bug fixes,
so a repo-wide search returned pre-fix code. The sha256 pins inside it still
resolve — they name files that stayed in the repository.
License
MIT — see LICENSE.
