Modelara CLI
Inspect before you integrate.
Modelara CLI starts an MCP server on your machine, records every tool it exposes with its schema and declared effect, and compares two versions to show what changed: added and removed tools, schema changes and new write access. Static checks need no model and no API key, and never call a tool.
- v0.1.9 for macOS and Linux, arm64 and x86_64
- Free, single binary, no Modelara account
- Text, JSON and SARIF output with stable exit codes
# install (macOS, Linux) $ curl -fsSL https://modelara.ai/install.sh | sh # describe how to start your MCP server $ modelara init # list its tools, effects and fingerprint $ modelara inspect # compare two versions, no model needed $ modelara check --static \ --baseline-command 'npx -y my-mcp@1.0.0' \ --candidate-command 'npx -y my-mcp@1.1.0'
01 The developer problem
A version bump can change what your agent can do.
You can call tools/list yourself and diff the JSON by eye. It works once. Modelara does it the same way every time: it normalizes schemas, classifies each tool's effect, fingerprints the interface and turns each difference into a stable rule with a severity.
| Change | Example | Rule | Result |
|---|---|---|---|
| New read-only tool | A tool that declares it only reads | static.interface.added_read_only | Info |
| New tool that writes, deletes or has an unknown effect | + append_file write not approved | static.authority.new_non_read | FAIL |
| Existing tool gains stronger authority | A tool that declared read-only now declares that it writes | static.authority.stronger_effect | FAIL |
| Tool removed | A tool present in the baseline is gone | static.interface.removed_tool | REVIEW |
| Argument becomes required | A property is added to the input schema's required list | static.interface.schema_changed | REVIEW |
| Argument added | directory_tree: input property added: depth | static.interface.schema_changed | REVIEW |
| Argument type changed | read_file: input property head changed: type "number" → "integer" | static.interface.schema_changed | REVIEW |
| Description changed | edit_file description changed info | static.interface.description_changed | Info |
Code examples are real findings from Filesystem MCP 2.1.0 → 2.2.0; rows in italics describe changes that comparison does not contain. These are signals, not failures. Many changes are harmless to a given agent; the check tells you where to look. FAIL for new authority is a policy decision you can approve, not a claim that the server is unsafe.
02 How the CLI works
Five steps, all on your machine.
- 01
Configure the server
Write a modelara.yaml that says how to start your MCP server and which environment variables it may see. Values are never written to the file.
modelara init - 02
Inspect its interface
Start the server, complete the MCP handshake and list every tool with its schema, declared effect and the interface fingerprint.
modelara inspect - 03
Save a baseline snapshot
Write the full interface to a JSON snapshot. It carries its own fingerprint, so an edited snapshot is refused later.
modelara inspect --json --no-store > base.json - 04
Compare another version
Start the candidate from a command line, or load a second snapshot. No model, API key or tool call is involved.
modelara check --static \ --baseline-interface base.json \ --candidate-command 'npx -y my-mcp@1.1.0' - 05
Review what changed
Read the findings, or emit JSON or SARIF for automation. Approve intended new authority by exact tool name.
modelara check --static … --json modelara check --static … --sarif
03 Real output
What you actually see.
Modelara CLI v0.1.9 against the public Filesystem MCP (opens in a new tab) server, versions 2.1.0 and 2.2.0 from npm. Each server ran in a container with no network access. Nothing below is mocked; excerpts are labelled.
Inspect a server
Every tool with its effect, whether it is retry-safe, its input and output size, and one fingerprint for the whole interface.
$ modelara inspect MODELARA INSPECT Source: secure-filesystem-server 2.2.0 (MCP 2025-11-25) Fingerprint: sha256:28daa4cf473cea0b961f4339affca39603a55e75474254fae65e6ac2431c2020 Discovered 16 tools Read-only: 10 Potential writes: 6 Destructive: 3 Unknown effects: 0 Retry-safe: 12 TOOL EFFECT DESTRUCTIVE RETRY-SAFE INPUTS OUTPUT append_file write no no 2 (2 req) 1 field copy_file write no yes 2 (2 req) 1 field directory_tree read - yes 3 (1 req) 1 field edit_file write yes no 3 (2 req) 1 field read_text_file read - yes 3 (1 req) 1 field write_file write yes unknown 2 (2 req) 1 field
The same check as JSON
Add --json for a versioned document your scripts can read. Verdict, summary and the first finding are shown, with whitespace compacted.
{
"schema_version": 1,
"mode": "static",
"verdict": "FAIL",
"summary": {
"added": 2, "removed": 0, "changed": 11,
"schema_changes": 3, "description_changes": 3,
"new_authority": 2, "approved_authority": 0, "unapproved_authority": 2
},
"findings": [
{
"rule_id": "static.authority.new_non_read",
"severity": "FAIL",
"capability": "append_file",
"change": "added",
"baseline_effect": null,
"candidate_effect": "write",
"approved": false,
"message": "new capability with write effect"
},
…Compare two versions: FAIL
2.2.0 adds two tools that write files, append_file and copy_file. Nobody approved them, so the result is FAIL. Three input schemas also changed; head and tail now require integers.
$ RUN='docker run -i --rm --network none --read-only --user node --cap-drop ALL --security-opt no-new-privileges --memory 256m --pids-limit 64 -v modelara-example-fs:/home/node:ro node:24-bookworm-slim node' $ FS=node_modules/@modelcontextprotocol/server-filesystem/dist/index.js $ modelara check --static \ --baseline-command "$RUN /home/node/2.1.0/$FS /tmp" \ --candidate-command "$RUN /home/node/2.2.0/$FS /tmp" MODELARA STATIC CHECK Baseline: secure-filesystem-server 2.1.0 (14 tools, sha256:bd1fd899684f…) Candidate: secure-filesystem-server 2.2.0 (16 tools, sha256:28daa4cf473c…) Interface changes + append_file write not approved + copy_file write not approved ~ directory_tree input schema changed review input property added: depth ~ read_file input schema changed review input property head changed: +maximum=9007199254740991, +minimum=0, type "number" → "integer" input property tail changed: +maximum=9007199254740991, +minimum=0, type "number" → "integer" ~ read_text_file input schema changed review input property head changed: +maximum=9007199254740991, +minimum=0, type "number" → "integer" input property tail changed: +maximum=9007199254740991, +minimum=0, type "number" → "integer" ~ directory_tree description changed info ~ edit_file description changed info ~ search_files description changed info Summary Added tools 2 Removed tools 0 Schema changes 3 Description changes 3 New authority 2 (0 approved, 2 not approved) RESULT: FAIL 2 new authority change(s) require approval. Next: review the authority changes above if intended, add them to check.security.approved_capabilities in modelara.yaml rerun modelara check --static Static check: interface and declared authority only. It runs no model and calls no tool; modelara test checks agent behavior.
After approving the new tools: REVIEW
List append_file and copy_file under check.security.approved_capabilitiesand rerun. Authority is resolved; the schema changes still need a look.
+ append_file write approved + copy_file write approved Summary Added tools 2 Removed tools 0 Schema changes 3 Description changes 3 New authority 2 (2 approved, 0 not approved) RESULT: REVIEW
SARIF for code scanning tools
Add --sarif (or --sarif-output file.sarif) for SARIF 2.1.0. This comparison produced 18 results:
static.authority.new_non_read2 × errorstatic.interface.schema_changed3 × warningstatic.interface.description_changed3 × notestatic.interface.metadata_changed10 × note
The release defines eight rule IDs in total; see JSON and SARIF.
Run it yourself
Pinned package versions, an isolated container and the exact commands are in Reproduce the Filesystem MCP example. The same comparison is published on Valyd (opens in a new tab), and every tool fingerprint for 2.2.0 matches its record.
04 Reading the result
Four results, four exit codes.
| Result | Exit | Meaning | What to do |
|---|---|---|---|
| PASS | 0 | No policy-significant interface change. Added read-only tools and description edits are reported as information. | Nothing required. Read the informational findings if you want detail. |
| REVIEW | 3 | Something an agent may rely on changed: a schema changed or a tool was removed. | Read the findings. To confirm agent behavior, run a model-based test. Use --review-exit-code 0 if CI should not stop on REVIEW. |
| FAIL | 1 | New or stronger authority that is not approved: a tool that writes, deletes or has an unknown effect. | If the change is intended, approve the exact tool names in modelara.yaml. Otherwise hold the upgrade. |
| INVALID | 2 | The comparison cannot be trusted, for example a server failed to start or a snapshot was edited. | Fix the input. No verdict about the change is given. |
Capability changes
Each tool is classified as read, write, destructive or unknown from the effect hints the server declares. A tool that declares nothing is unknown and treated as possibly destructive, never as safe. Approve intended new authority by exact name under check.security.approved_capabilities; there are no wildcards.
Interface fingerprints
A sha256: fingerprint covers the whole interface. Identical interfaces produce the same fingerprint; any change produces a new one. Snapshots record theirs, and a snapshot whose content no longer matches is refused as INVALID.
Exit codes in CI
0 PASS, 1 FAIL, 2 INVALID, 3 REVIEW. --quiet prints only the result, and--review-exit-code 0 lets a pipeline continue on REVIEW while still recording it.
It compares the declared interface: tool names, schemas, descriptions and the effect hints a server reports about itself. It does not run your agent, so a PASS is not a guarantee that your agent will succeed, and it cannot detect behavior changes behind an unchanged schema. It also does not check that both sides are the same server. For behavior, use modelara test, a model-based regression test described below.
05 What it can do today
Built for the way developers ship.
Check an upgrade before you take it
check --staticstarts each version, lists its tools and stops it. Run it before you bump a pinned server version.Fail CI on unreviewed authority
Stable exit codes and SARIF make new write or destructive tools visible in the same review flow as the rest of your code.
Approve changes in code review
Approvals are exact tool names in
modelara.yaml, so accepting new authority is a reviewed commit, not a click.Test agent behavior, when you need it
modelara testruns read-only tasks through a real model, makes the first run your baseline and compares later runs with it. Results are scored on exact expected facts, with no model grading another. Requires an OpenAI or OpenAI-compatible endpoint and your own API key.Keep a local history
Inspected interfaces are stored locally, so
modelara diffcan compare two stored versions by fingerprint later.
06 Security and privacy
Know exactly what runs and what leaves your machine.
- Local inspection
- The CLI runs on your machine. It sends nothing to Modelara and has no telemetry.
- Server execution
- The CLI starts the MCP server command you configure as a local process with your user's permissions. It is not sandboxed. Inspect servers you trust, or start them in a container, for example with
docker run -i …as the server command. - Tool calls
inspectandcheck --staticcall no tools.inspect --calland model-based tests run only tools declared read-only; anything else is blocked before it reaches the server.- Credentials
modelara.yamlnames environment variables, never values. The server sees only the variables you list underenv_passthrough;inherit_envis off by default.- Network
- The CLI connects to your model provider only during model-based tests, and to Modelara's team service only when you run
modelara sync push. The MCP server you start may use the network on its own. - Evidence
- Interfaces, benchmarks and checks are stored in a local SQLite database at
.modelara/modelara.db. Model-based tests send prompts and tool results to the provider you configure, as inference requires.
07 Installation and first steps
Install in one line.
The script detects your OS and architecture, downloads the latest release from GitHub, verifies its SHA-256 checksum and installs modelara to ~/.local/bin (override withMODELARA_INSTALL_DIR).
$ curl -fsSL https://modelara.ai/install.sh | sh $ export PATH="$HOME/.local/bin:$PATH" $ modelara --version v0.1.9
macOS binaries are currently unsigned. If macOS blocks the binary, runxattr -d com.apple.quarantine ~/.local/bin/modelara.
Release archives (v0.1.9)
| OS | Architecture | Archive |
|---|---|---|
| macOS | Apple silicon (arm64) | modelara_Darwin_arm64.tar.gz |
| macOS | Intel (x86_64) | modelara_Darwin_x86_64.tar.gz |
| Linux | arm64 | modelara_Linux_arm64.tar.gz |
| Linux | x86_64 | modelara_Linux_x86_64.tar.gz |
Windows builds are not available yet. Archives and checksums.txt are on thev0.1.9 release page (opens in a new tab) .
First steps
modelara initdescribe how to start your servermodelara inspectsee its tools, effects and fingerprintmodelara inspect --json --no-store > base.jsonsave a baseline snapshotmodelara check --static --baseline-interface base.json --candidate-command '…'compare the next version
Install Modelara CLI.
Inspect the MCP server you depend on most, and save its snapshot before the next upgrade.