Modelara CLI

Inspect before you integrate.

Modelara CLI starts an MCP server on your machine, records every tool it exposes with its schema and declared effect, and compares two versions to show what changed: added and removed tools, schema changes and new write access. Static checks need no model and no API key, and never call a tool.

  • v0.1.9 for macOS and Linux, arm64 and x86_64
  • Free, single binary, no Modelara account
  • Text, JSON and SARIF output with stable exit codes
terminalQUICK START
# install (macOS, Linux)
$ curl -fsSL https://modelara.ai/install.sh | sh

# describe how to start your MCP server
$ modelara init

# list its tools, effects and fingerprint
$ modelara inspect

# compare two versions, no model needed
$ modelara check --static \
    --baseline-command  'npx -y my-mcp@1.0.0' \
    --candidate-command 'npx -y my-mcp@1.1.0'

01 The developer problem

A version bump can change what your agent can do.

You can call tools/list yourself and diff the JSON by eye. It works once. Modelara does it the same way every time: it normalizes schemas, classifies each tool's effect, fingerprints the interface and turns each difference into a stable rule with a severity.

Interface changes and how modelara check --static reports them
ChangeExampleRuleResult
New read-only toolA tool that declares it only readsstatic.interface.added_read_onlyInfo
New tool that writes, deletes or has an unknown effect+ append_file write not approvedstatic.authority.new_non_readFAIL
Existing tool gains stronger authorityA tool that declared read-only now declares that it writesstatic.authority.stronger_effectFAIL
Tool removedA tool present in the baseline is gonestatic.interface.removed_toolREVIEW
Argument becomes requiredA property is added to the input schema's required liststatic.interface.schema_changedREVIEW
Argument addeddirectory_tree: input property added: depthstatic.interface.schema_changedREVIEW
Argument type changedread_file: input property head changed: type "number" → "integer"static.interface.schema_changedREVIEW
Description changededit_file description changed infostatic.interface.description_changedInfo

Code examples are real findings from Filesystem MCP 2.1.0 → 2.2.0; rows in italics describe changes that comparison does not contain. These are signals, not failures. Many changes are harmless to a given agent; the check tells you where to look. FAIL for new authority is a policy decision you can approve, not a claim that the server is unsafe.

02 How the CLI works

Five steps, all on your machine.

  1. 01

    Configure the server

    Write a modelara.yaml that says how to start your MCP server and which environment variables it may see. Values are never written to the file.

    modelara init
  2. 02

    Inspect its interface

    Start the server, complete the MCP handshake and list every tool with its schema, declared effect and the interface fingerprint.

    modelara inspect
  3. 03

    Save a baseline snapshot

    Write the full interface to a JSON snapshot. It carries its own fingerprint, so an edited snapshot is refused later.

    modelara inspect --json --no-store > base.json
  4. 04

    Compare another version

    Start the candidate from a command line, or load a second snapshot. No model, API key or tool call is involved.

    modelara check --static \
      --baseline-interface base.json \
      --candidate-command 'npx -y my-mcp@1.1.0'
  5. 05

    Review what changed

    Read the findings, or emit JSON or SARIF for automation. Approve intended new authority by exact tool name.

    modelara check --static … --json
    modelara check --static … --sarif

03 Real output

What you actually see.

Modelara CLI v0.1.9 against the public Filesystem MCP (opens in a new tab) server, versions 2.1.0 and 2.2.0 from npm. Each server ran in a container with no network access. Nothing below is mocked; excerpts are labelled.

Inspect a server

Every tool with its effect, whether it is retry-safe, its input and output size, and one fingerprint for the whole interface.

modelara inspectEXIT 0
$ modelara inspect

MODELARA INSPECT

Source:       secure-filesystem-server 2.2.0 (MCP 2025-11-25)
Fingerprint:  sha256:28daa4cf473cea0b961f4339affca39603a55e75474254fae65e6ac2431c2020
Discovered 16 tools

Read-only:         10
Potential writes:  6
Destructive:       3
Unknown effects:   0
Retry-safe:        12

TOOL                       EFFECT  DESTRUCTIVE  RETRY-SAFE  INPUTS     OUTPUT
append_file                write   no           no          2 (2 req)  1 field
copy_file                  write   no           yes         2 (2 req)  1 field
directory_tree             read    -            yes         3 (1 req)  1 field
edit_file                  write   yes          no          3 (2 req)  1 field
read_text_file             read    -            yes         3 (1 req)  1 field
write_file                 write   yes          unknown     2 (2 req)  1 field
Excerpt: 6 of 16 tools shown; the description column is omitted.

The same check as JSON

Add --json for a versioned document your scripts can read. Verdict, summary and the first finding are shown, with whitespace compacted.

modelara check --static … --jsonEXIT 1
{
  "schema_version": 1,
  "mode": "static",
  "verdict": "FAIL",
  "summary": {
    "added": 2, "removed": 0, "changed": 11,
    "schema_changes": 3, "description_changes": 3,
    "new_authority": 2, "approved_authority": 0, "unapproved_authority": 2
  },
  "findings": [
    {
      "rule_id": "static.authority.new_non_read",
      "severity": "FAIL",
      "capability": "append_file",
      "change": "added",
      "baseline_effect": null,
      "candidate_effect": "write",
      "approved": false,
      "message": "new capability with write effect"
    },
    …
Excerpt: 17 more findings follow.

Compare two versions: FAIL

2.2.0 adds two tools that write files, append_file and copy_file. Nobody approved them, so the result is FAIL. Three input schemas also changed; head and tail now require integers.

modelara check --staticEXIT 1
$ RUN='docker run -i --rm --network none --read-only --user node --cap-drop ALL --security-opt no-new-privileges --memory 256m --pids-limit 64 -v modelara-example-fs:/home/node:ro node:24-bookworm-slim node'
$ FS=node_modules/@modelcontextprotocol/server-filesystem/dist/index.js
$ modelara check --static \
    --baseline-command  "$RUN /home/node/2.1.0/$FS /tmp" \
    --candidate-command "$RUN /home/node/2.2.0/$FS /tmp"

MODELARA STATIC CHECK

Baseline:  secure-filesystem-server 2.1.0  (14 tools, sha256:bd1fd899684f…)
Candidate: secure-filesystem-server 2.2.0  (16 tools, sha256:28daa4cf473c…)

Interface changes

  + append_file                 write                  not approved
  + copy_file                   write                  not approved
  ~ directory_tree              input schema changed   review
      input property added: depth
  ~ read_file                   input schema changed   review
      input property head changed: +maximum=9007199254740991, +minimum=0, type "number" → "integer"
      input property tail changed: +maximum=9007199254740991, +minimum=0, type "number" → "integer"
  ~ read_text_file              input schema changed   review
      input property head changed: +maximum=9007199254740991, +minimum=0, type "number" → "integer"
      input property tail changed: +maximum=9007199254740991, +minimum=0, type "number" → "integer"
  ~ directory_tree              description changed    info
  ~ edit_file                   description changed    info
  ~ search_files                description changed    info

Summary

  Added tools          2
  Removed tools        0
  Schema changes       3
  Description changes  3
  New authority        2 (0 approved, 2 not approved)

RESULT: FAIL

2 new authority change(s) require approval.

Next:
  review the authority changes above
  if intended, add them to check.security.approved_capabilities in modelara.yaml
  rerun modelara check --static

Static check: interface and declared authority only. It runs no model and calls no tool;
modelara test checks agent behavior.
Excerpt: ten informational "metadata changed" findings (declared idempotent: false → true) are omitted.

After approving the new tools: REVIEW

List append_file and copy_file under check.security.approved_capabilitiesand rerun. Authority is resolved; the schema changes still need a look.

modelara check --staticEXIT 3
  + append_file                 write                  approved
  + copy_file                   write                  approved

Summary

  Added tools          2
  Removed tools        0
  Schema changes       3
  Description changes  3
  New authority        2 (2 approved, 0 not approved)

RESULT: REVIEW
Excerpt: the approved lines, summary and result.

SARIF for code scanning tools

Add --sarif (or --sarif-output file.sarif) for SARIF 2.1.0. This comparison produced 18 results:

  • static.authority.new_non_read2 × error
  • static.interface.schema_changed3 × warning
  • static.interface.description_changed3 × note
  • static.interface.metadata_changed10 × note

The release defines eight rule IDs in total; see JSON and SARIF.

Run it yourself

Pinned package versions, an isolated container and the exact commands are in Reproduce the Filesystem MCP example. The same comparison is published on Valyd (opens in a new tab), and every tool fingerprint for 2.2.0 matches its record.

04 Reading the result

Four results, four exit codes.

Static check results, exit codes and what to do
ResultExitMeaningWhat to do
PASS0No policy-significant interface change. Added read-only tools and description edits are reported as information.Nothing required. Read the informational findings if you want detail.
REVIEW3Something an agent may rely on changed: a schema changed or a tool was removed.Read the findings. To confirm agent behavior, run a model-based test. Use --review-exit-code 0 if CI should not stop on REVIEW.
FAIL1New or stronger authority that is not approved: a tool that writes, deletes or has an unknown effect.If the change is intended, approve the exact tool names in modelara.yaml. Otherwise hold the upgrade.
INVALID2The comparison cannot be trusted, for example a server failed to start or a snapshot was edited.Fix the input. No verdict about the change is given.

Capability changes

Each tool is classified as read, write, destructive or unknown from the effect hints the server declares. A tool that declares nothing is unknown and treated as possibly destructive, never as safe. Approve intended new authority by exact name under check.security.approved_capabilities; there are no wildcards.

Interface fingerprints

A sha256: fingerprint covers the whole interface. Identical interfaces produce the same fingerprint; any change produces a new one. Snapshots record theirs, and a snapshot whose content no longer matches is refused as INVALID.

Exit codes in CI

0 PASS, 1 FAIL, 2 INVALID, 3 REVIEW. --quiet prints only the result, and--review-exit-code 0 lets a pipeline continue on REVIEW while still recording it.

What a static check does not tell you

It compares the declared interface: tool names, schemas, descriptions and the effect hints a server reports about itself. It does not run your agent, so a PASS is not a guarantee that your agent will succeed, and it cannot detect behavior changes behind an unchanged schema. It also does not check that both sides are the same server. For behavior, use modelara test, a model-based regression test described below.

05 What it can do today

Built for the way developers ship.

  • Check an upgrade before you take it

    check --static starts each version, lists its tools and stops it. Run it before you bump a pinned server version.

  • Fail CI on unreviewed authority

    Stable exit codes and SARIF make new write or destructive tools visible in the same review flow as the rest of your code.

  • Approve changes in code review

    Approvals are exact tool names in modelara.yaml, so accepting new authority is a reviewed commit, not a click.

  • Test agent behavior, when you need it

    modelara test runs read-only tasks through a real model, makes the first run your baseline and compares later runs with it. Results are scored on exact expected facts, with no model grading another. Requires an OpenAI or OpenAI-compatible endpoint and your own API key.

  • Keep a local history

    Inspected interfaces are stored locally, so modelara diff can compare two stored versions by fingerprint later.

06 Security and privacy

Know exactly what runs and what leaves your machine.

Read our security principles
Local inspection
The CLI runs on your machine. It sends nothing to Modelara and has no telemetry.
Server execution
The CLI starts the MCP server command you configure as a local process with your user's permissions. It is not sandboxed. Inspect servers you trust, or start them in a container, for example with docker run -i …as the server command.
Tool calls
inspect and check --static call no tools. inspect --call and model-based tests run only tools declared read-only; anything else is blocked before it reaches the server.
Credentials
modelara.yaml names environment variables, never values. The server sees only the variables you list under env_passthrough; inherit_env is off by default.
Network
The CLI connects to your model provider only during model-based tests, and to Modelara's team service only when you run modelara sync push. The MCP server you start may use the network on its own.
Evidence
Interfaces, benchmarks and checks are stored in a local SQLite database at .modelara/modelara.db. Model-based tests send prompts and tool results to the provider you configure, as inference requires.

07 Installation and first steps

Install in one line.

The script detects your OS and architecture, downloads the latest release from GitHub, verifies its SHA-256 checksum and installs modelara to ~/.local/bin (override withMODELARA_INSTALL_DIR).

install
$ curl -fsSL https://modelara.ai/install.sh | sh
$ export PATH="$HOME/.local/bin:$PATH"
$ modelara --version
v0.1.9

macOS binaries are currently unsigned. If macOS blocks the binary, runxattr -d com.apple.quarantine ~/.local/bin/modelara.

Release archives (v0.1.9)

Supported platforms and release archives
OSArchitectureArchive
macOSApple silicon (arm64)modelara_Darwin_arm64.tar.gz
macOSIntel (x86_64)modelara_Darwin_x86_64.tar.gz
Linuxarm64modelara_Linux_arm64.tar.gz
Linuxx86_64modelara_Linux_x86_64.tar.gz

Windows builds are not available yet. Archives and checksums.txt are on thev0.1.9 release page (opens in a new tab) .

First steps

  1. modelara init describe how to start your server
  2. modelara inspect see its tools, effects and fingerprint
  3. modelara inspect --json --no-store > base.json save a baseline snapshot
  4. modelara check --static --baseline-interface base.json --candidate-command '…' compare the next version

Install Modelara CLI.

Inspect the MCP server you depend on most, and save its snapshot before the next upgrade.