Model and Specification View
Current responsibility and status. This module lists pinned and built models, loads supported VNN-LIB and classifier sources, serves ACT-backed model descriptions and dataset samples, and renders the network and input or output specification. MSV-01 and MSV-04 are implemented. MSV-02 and MSV-03 remain backlog. C owns the views, B integrates the service, and A reviews variable and constraint semantics.
The module owns presentation and content addressed model snapshots. It does not reinterpret ACT semantics, calculate bounds, or replace unsupported specifications with boxes. ACT-side loading and description are engine_bridge/model_spec. Run creation is verification, engine readiness is engine, and selection is workspace. See the parent flow diagram and project development entry point.
Change Guide
| Goal | Required edits | Conditional edits / reused path |
|---|---|---|
| Show a new description field | schemas.py, service.py mapping, consuming view/tests; generate wire.ts | bridge model_spec only if the neutral description lacks the field. Routes/hooks/API only for endpoint/query changes. |
| Support a new source | bridge source builder, schemas.py, service.py, LoadPanel.tsx, source tests | Existing POST /api/models transport can remain if its operation is unchanged. |
| Change graph or input-image presentation | ModelGraph.tsx, SpecPanel.tsx, InputDomain.tsx as relevant | Shared pixels helpers only when reusable rendering changes; no ACT or model-identity change for presentation. |
| Change content identity or persistence | service.py, persistence/hash tests | verification only when its model ID/path contract changes. |
| Add editable specification drafts | MSV-03 backlog: define validation and new immutable model references first | No general draft API exists today. Preserve old runs' pinned snapshots. |
Flow Diagram
Arrows are labeled as calls, results, or events. Neighboring modules are named explicitly and linked in the block tables. HTTP transport may be grouped; views use hooks or props, never direct fetch calls.
List and describe a model
| Block | Code entry | Contract or artifact | Neighbor handoff |
|---|---|---|---|
| List | ModelService.models | Catalog and derived model summaries with methods and sha256 | Verification uses method list |
| Describe | ModelService.describe | ModelDescriptionOut with stable layer IDs, shapes, input bounds, and output constraints | Analysis and workspace use IDs |
| Select | ModelGraph.tsx | Layer selection is presentation state, not ACT computation | Workspace owns global selection |
Load and persist a source
| Block | Code entry | Contract or artifact | Neighbor handoff |
|---|---|---|---|
| Source request | LoadVnnlibRequest, LoadTorchRequest | Relative VNN-LIB paths, classifier source, index or indices, eps | Bridge validates source availability |
| Build | ModelService.load | Content addressed <kind>-<sha12> model and .meta.json | Derived model can be used by verification |
| Repeat load | model_id from sha256 | Same content returns 200 and existing model | No duplicate build is required |
Render an input specification
| Block | Code entry | Contract or artifact | Neighbor handoff |
|---|---|---|---|
| Description | InputSpecOut, OutputSpecOut | ACT shape, bounds, center, dataset, constraint text, and true class; bounds and center are omitted above DISPLAY_VALUES (3x224x224) values per row, and the panel says so | No browser reinterpretation of constraints |
| Image view | InputDomain.tsx | Clean, bounds, heat map, and sampled display perturbations | Samples are visual only, not verification evidence |
| Navigation | ModelGraph.tsx | Stable layer ID | Workspace and analysis consume selection |
Validation
Run the module's real ACT tests:
.venv/bin/python -m pytest visual_bench/model_specAcceptance is observable when catalog models list their pinned sha256 and methods, a valid VNN-LIB or classifier source returns 201 then 200 for the same content, the model survives service restart, descriptions preserve ACT layer IDs and shapes, samples return the requested dataset row, a model the active engine cannot load is reported as load_failed and rebuilds from its recorded parameters, and invalid paths or sources return structured errors.
Run npm run gen:types after changing schemas.py, then npm run check:types. Do not hand edit visual_bench/generated/wire.ts. Do not claim tests were run unless the command and result are recorded.
Reference and Backlog
Current files, routes and contracts
Current Files and Contracts
| Layer | Entry | Contract or artifact | Owner for handoff |
|---|---|---|---|
| Wire | schemas.py | Model summaries, source requests, samples, layers, input and output specs | generated/wire.ts is generated from Pydantic models |
| Service | service.py | models, sources, sample, load, rebuild, describe; content addressed built models with built_with (ACT commit and source) and engine_check | Engine gate is engine.EngineService; ACT description is bridge owned |
| HTTP | routes.py | /api/models, /api/models/{id}, POST /api/models/{id}/rebuild, /api/sources, /api/sources/sample | Verification consumes model IDs and methods |
| Browser data | api.ts, hooks.ts | Queries for models, descriptions, sources, samples; load and rebuild invalidate the model list | Shared query keys own cache identity |
| Views | LoadPanel.tsx, ModelList.tsx, ModelGraph.tsx, SpecPanel.tsx, InputDomain.tsx, RebuildNotice.tsx | Load, graph, spec, image-domain, and sample presentation; rebuild offer for a model the active engine cannot load | Workspace owns selection and navigation |
| Tests | test_model_spec.py | Real ACT loads, idempotency, persistence, descriptions, samples, engine-check and rebuild, structured errors | Bridge loader tests cover ACT source adaptation |
Task acceptance and planned capabilities
Tasks
The task IDs and dependencies remain stable. MSV-04 is the implemented loading slice and appears in the parent task index.
| ID | Deliverable | Depends on | Completion evidence |
|---|---|---|---|
| MSV-01 | Deliver real description list/detail UI | VER-01, WS-02 | A real sample shows correct layer shapes, tensor edges, and specification variables with loading/error states. |
| MSV-02 | Evaluate and wrap the network renderer | MSV-01 | Show stable IDs, external selection, node callbacks, resize, worker/assets, and browser findings; document a fallback if the candidate fails. |
| MSV-03 | Validate specification drafts | MSV-01, MSB-03 | Invalid variable mappings are explained; a valid change creates a new reference and cannot mutate an existing run. |
| MSV-04 | Load models from VNN-LIB and torch sources | MSB-04, MSV-01 | LoadPanel.tsx in the sidebar: VNN-LIB tab (benchmark, instance) and Torch tab (classifier, test-image index with preview and label, eps slider, batch size with a per-image picker: a batch of several test images builds one net with a row per image (fuzzing and Verification; BaB needs one image), and the Specification panel shows it one row at a time). POST /api/models builds through the bridge and returns the content-addressed model (201 new, 200 existing); loaded models appear under "Loaded" with their parameters, and selecting one prefills the form so a new image/eps rebuilds a new snapshot. Flat VNN-LIB image vectors of known datasets are drawn as H x W x C (cora_2024 convention). |
| MSV-05 | Flag and rebuild models built on another ACT engine | MSV-04, VER-08 | ModelService.load records built_with (commit, baseline or remote/branch) in .meta.json. ModelSummary.engine_check is same_commit when the active engine built it, otherwise the outcome of the first describe on the active commit (loads / load_failed) or unchecked; model files are never scanned eagerly (built models can be GBs). ModelList.tsx marks load_failed ("needs rebuild") and untried models from another commit; RebuildNotice.tsx replaces the load error with a button that calls POST /api/models/{id}/rebuild, which rebuilds the recorded source settings through load and selects the new model; the old model and its runs are kept. Evidence: test_model_spec.py::test_model_from_another_engine_is_flagged_when_it_fails_to_load_and_rebuilds. |
| MSV-06 | Load a batch of VNN-LIB properties on one network | MSB-05, MSV-04 | LoadVnnlibRequest.vnnlibs (distinct, starts with vnnlib, at most 64); the VNN-LIB tab has a batch size (capped at the network's property count) and a per-slot property picker restricted to the first slot's ONNX file, with the batch collapsed into chips. hooks.ts batchRows/useBatchRows label batch rows for every view (test image #i or property name): ModelList chip, Verification row verdicts, fuzzing Inputs table. Rebuild keeps the batch. visual_bench/fuzzing/test_fuzzing.py::test_vnnlib_properties_on_one_network_are_one_net_fuzzed_row_by_row. |
MSV-02's renderer evaluation and MSV-03's editable draft validation are not current capabilities. Proposed ModelSpecView.tsx, view_model.ts, and network_renderer.ts are design references only. Keep unsupported specifications visible with their actual representation and reason.