Agent interface intelligence
Methodology
Valyd's methodology is designed for agent-facing interface intelligence. Current public coverage is MCP-first: a reproducible, static view of how an MCP server's discovered interface changes between stable versions.
Server selection
Catalog entries are reviewed and configuration-driven. A server is published as tracked only after its package identity is known, versions can be pinned, historical versions are available, stdio launch and MCP initialization succeed, tools are discovered, and the resulting interface can be compared reproducibly.
Servers that fail eligibility remain unpublished. Modelara Observatory tracks MCP servers from five distribution channels: npm and PyPI packages, Go binaries from GitHub releases, OCI container images, and MCPB bundles. Every server is launched and inspected over stdio. Each channel verifies its release artifact in its own way, so the provenance recorded for a release depends on what that channel publishes.
Version selection
Stable releases are sorted with semantic-version rules; prereleases are excluded by default. Bootstrap scans the most recent configured versions and compares consecutive releases. Update scans skip snapshots already known, fill in missing versions in the configured window, and retain the nearest earlier snapshot as a comparison baseline when needed.
Static analysis
Each version is inspected with the pinned Modelara CLI. Comparisons use modelara check --static --json over saved interface snapshots.
- No model or model API key is used by static analysis.
- No task suite or behavioral benchmark is run.
- Tools are discovered but never called.
- Analysis is based on interface metadata, schemas, and declared MCP annotations.
The analyzer version and interface fingerprints are retained with every comparison.
Each newly captured snapshot also records the release's identity and provenance: registry, package name and version, the upstream version or tag where it differs, publication time, an artifact integrity value or digest, and the repository URL, when the registry exposes them. What the integrity value covers depends on the channel: the npm package integrity, a PyPI wheel's SHA-256, a Go release's checksums file, an OCI image digest, or an MCPB bundle's digest. Fields a channel does not provide are left empty rather than inferred. The Modelara analyzer version is recorded separately; a policy version is left unknown when none is provided. Older schema-v1 records without an explicit schema marker remain readable.
Effects
- Read
- The capability is declared read-only, or Modelara inferred from aligned evidence that it only reads.
- Write
- The capability may change state.
- Destructive
- The capability may destroy or irreversibly change data.
- Unknown
- Valyd could not determine the capability's effect confidently from the declared interface and static evidence.
Declared annotations always take precedence. From Modelara v0.1.8, a capability that declares nothing can receive an inferred effect, but only when its name, description, and input schema agree under Modelara's documented deterministic, static rules. No tool is called to find out. Weak or contradictory evidence stays Unknown, and an inferred Write does not establish that a capability is non-destructive. Release comparisons and authority findings use declared effects only. Snapshots analyzed with earlier Modelara versions keep the effects recorded at the time.
Read, Write, Destructive, and Unknown are descriptive capability assessments. They do not determine whether an MCP is eligible for Valyd coverage. Public coverage means Valyd could safely and reproducibly inspect the interface. It does not mean the interface is read-only or non-destructive, and it does not mean every capability's effect is known.
Destructive and Unknown capabilities stay visible on every server page, with whether each effect was declared by the server or inferred by Modelara. Valyd does not treat Unknown as safe, does not treat it as destructive, and does not convert it into Read or Write to make coverage look more complete.
Capability history and hashes
Capability identity is based on server, MCP capability type, and name. The capability surface is derived from the latest observed snapshot and available earlier snapshots; it reports the analyzed release range, the first version observed in Valyd's coverage, and the last observed interface change. It does not claim a capability was introduced at its first observed version or claim coverage beyond those snapshots.
Observed interface fields, including declared metadata and schemas, remain separate from Modelara-derived effects. Version-2 canonical SHA-256 hashes sort object keys, distinguish absent, null, and empty values, and cover stable agent-facing interface fields rather than analyzer fingerprints or derived metadata. Scanner-known temporary and home paths are redacted before hashing and publication; ordinary interface paths remain intact. Historical version-1 hashes continue to validate using their original scheme.
Output schemas are shown only when exposed by the analyzed interface; Valyd does not infer them from input schemas. Historical comparisons that did not distinguish input and output schema changes continue to display their available finding detail without being rewritten.
Analysis
- PASS
- No policy-significant interface change detected.
- REVIEW
- Interface changes require inspection.
- FAIL
- A policy-significant change was detected, such as a new tool with write, destructive, or unknown declared effect, or an existing tool whose declared effect became stronger.
- INVALID
- The comparison could not be evaluated reliably.
Analysis reports interface changes against Modelara's policy. It is not a security score. A result describes a version comparison, not the overall safety of a server.
FAIL is a static interface validation result: it means the declared interface gained authority that the policy requires a person to review. It is not evidence of a vulnerability, an exploit, or an agent failure, and no tool is called to produce it. Effects follow the MCP tool annotations; as the MCP specification defines, a tool that is not read-only and does not declare destructiveHint is treated as potentially destructive.
Verification and publication
Valyd's data is produced by Modelara Observatory. Discovering or scanning a release does not publish it: new evidence stays private in Observatory until a person reviews and approves it there. Approved records are published to Valyd's public data platform in a reviewed batch, as a new versioned data generation, and the website is rebuilt from one committed generation in a separate, reviewed promotion step. The generation behind the live site is listed in valyd-data-source.json.
Records published before Observatory took over scanning were produced by Valyd's earlier scanner and are kept as they were recorded. New write, destructive, unknown-effect, large, or inconclusive changes remain marked pending review. Automated static-analysis verification is labeled separately from pending human review; no reviewer identity is inferred when none is recorded. Review decisions are recorded in the comparison data, and generated aggregates are rebuilt without rerunning the analyzer. Historical analyzer provenance is not silently rewritten.
Limitations
- The initial scan scope is MCP over stdio; HTTP transport is not included.
- Effect classification depends on annotations reported by the MCP server.
- Static comparison cannot prove whether an agent workflow regressed.
- Behavioral verification requires running
modelara testwith an explicitly configured task suite.