CLI, test, and release conventions
Look up CLI, testing, and release rules with their canonical owners and checks.
9 min read
Updated 2026-10-16
#conventions#cli#testing#release
Use these rows with the Conventions for Cloud applications index and audit procedure. Follow each canonical link for the complete contract.
CLI and Assistant Skills (CLI)
| ID | Rule | Why | How to check | Canonical |
|---|---|---|---|---|
| CLI-01 | Support text, one-value JSON, and compact one-value-per-line JSONL on every command, sending progress and warnings to stderr. | Stdout remains valid for the selected machine output mode. | bun scripts/check.ts boundaries catches built-in JSON branches without JSONL handling only; Manual: run each command in text, --json, and --jsonl modes and parse the structured stdout. |
Output modes |
| CLI-02 | Use the shared primary-resource verbs, plural groups for secondary resources, and domain verbs for operations that fit neither. | People and agents can predict commands across modules. | Manual: compare command declarations and help with the verb table and secondary-resource examples. | Command verbs |
| CLI-03 | Parse resource addresses with parseCliAddress(), resolve them through the application API, and report ambiguous matches rather than guessing. |
IDs remain unambiguous and a person can retry with a listed candidate. | Manual: inspect resource argument parsing; try an ID, container path, and supported local path, then an ambiguous name and verify 409 CONFLICT with candidates. |
Resource addresses |
| CLI-04 | Use the shared output, confirmation, content-input, and download flags, requiring --yes for destructive or irreversible actions and refusing nonterminal prompting. |
Shared flags keep the same meaning across commands. | Manual: compare flags with the shared table; run a destructive command without --yes and without a terminal using demo data. |
Shared flags |
| CLI-05 | Exit with 0 only for complete success and 1 for errors or partial results, reporting errors on stderr. |
Automation can distinguish a completed request from skipped or failed work. | Manual: inspect run() return values and capture exit status and stderr for success, API failure, and a partial result. |
Exit status |
| CLI-06 | Write command summaries, descriptions, and messages in English and German, using the help locale and ctx.options.locale while keeping command syntax and examples unchanged. |
Help and command output use the same locale without changing invocation syntax. | Manual: compare root and command help plus human messages in English and German; inspect the module's locale-aware builder. | Localized help |
| CLI-07 | Add permalink to returned readable resource objects with withCloudResourcePermalink(), using the ID their Type reader accepts. |
Resource links open on the selected profile's server and target readable resources. | Manual: inspect JSON resource output and open each permalink; verify derived entries link to the reader's resource and unreadable or deleted resources receive no link. | Resource permalinks |
| CLI-08 | Keep domain writes and permission checks on the server, with CLI commands parsing input, calling authenticated APIs, and rendering results. | CLI, browser, and integration callers receive the same authorization result. | Manual: trace commands to authenticated routes and compare a denied operation through CLI and browser. | Server authorization |
| CLI-09 | Declare Skills in src/skills.ts, keep each Skill below src/skills/<name>/, and explicitly import every Markdown file with with { type: "text" }. |
The build bundles visible, reviewable content without runtime filesystem scanning. | Manual: compare the Skill folders with the declaration's imports and reference list. | Skill module |
| CLI-10 | Treat a Skill as instructions that grant no permission; every capability it invokes still authorizes the actor. | Loading instructions does not bypass application authorization. | Manual: inspect the named capabilities' authorization and try the workflow with an actor lacking access. | Skill permissions |
| CLI-11 | Test the Skill declaration, verify every named capability ID exists, and check that Assistant can complete the workflow with normal access. | Shipped instructions refer to valid operations and a usable workflow. | Manual: import the Skill declaration in a test, resolve each named capability, and run its workflow with a person's normal access. | Skill tests |
Testing (TEST)
| ID | Rule | Why | How to check | Canonical |
|---|---|---|---|---|
| TEST-01 | Built-in apps: name tests that need the browser build *.behavior.test.ts or *.behavior.test.tsx so the behavior runner discovers them. |
Browser conditions and the Solid DOM preload apply instead of silently skipping the test. | bun scripts/check.ts behavior-tests; Manual: inspect tests using isServer and compare their filenames with the runner's behavior suites. |
Behavior tests |
| TEST-02 | Built-in apps: start Playwright browsers through a direct extensionless import of launchBrowser() from packages/ui/test/browser, except the documented nightly Assistant artifact suites. |
The runner discovers browser tests and can select WebKit as well as Chromium. | bun scripts/check.ts browser-launch; Manual: inspect Playwright imports and launches against the shared-launcher contract and its documented exceptions. |
Shared browser launcher |
| TEST-03 | Built-in apps: check the same browser behavior in Chromium and WebKit, explaining legitimate engine-specific branches or skips. | Browser assertions account for real engine differences. | Manual: compare the browser suites' behavior in both engines and read each browserName branch or skip for its reason. |
Browser engines |
| TEST-04 | Built-in apps: isolate top-level mock.module replacements with --isolate or a child process. |
Module replacements do not leak into tests loaded later in the same process. | bun scripts/check.ts module-mocks; Manual: use rg 'mock.module' packages and compare each top-level replacement with its suite's process isolation. |
Module mocks |
| TEST-05 | Built-in apps: gate integration suites on explicit CLOUD_TEST_* targets through the shared fixture, use databases ending in _test, and fail when configured targets are unreachable. |
Tests cannot fall through to the development database or silently ignore a broken target. | Manual: inspect fixture imports and suite gates in scripts/fixtures/test-infra.ts; verify database-name rejection and missing-variable skips in the fixture tests. |
Integration tests |
| TEST-06 | Built-in apps: take a test-owned Sync instance's namespace from testSyncNamespace() so the fixture deletes its streams. |
Shared test-broker storage does not remain reserved after the suite. | bun scripts/check.ts test-sync-namespaces; Manual: use rg 'createSync|testSyncNamespace' packages tests and trace each test namespace to the fixture helper. |
Test Sync namespaces |
| TEST-07 | Built-in apps: open test connections to the shared NATS broker through connectTestNats(). |
Test credentials and the DEV-account refusal keep streams in the test account. | bun scripts/check.ts test-nats-connections; Manual: inspect test NATS connections for the fixture helper; connections to a test-owned broker carry their own credentials unconditionally. |
Test NATS connections |
| TEST-08 | Record the request count before a browser step, wait for the request it causes, and compare that request rather than the newest one already recorded. | A changed page does not prove that a fake server has recorded the new request. | Manual: read assertions against fake-server request lists after clicks or key presses and trace their wait to the request caused by that step. | Request assertions |
| TEST-09 | Test SSR routes through Hono for identities, permissions, query parsing, service states, links, actions, titles, and essential content, verifying denied records are absent from HTML. | Server-rendered content respects the same access and state boundaries. | Manual: read route tests for anonymous, allowed, denied, invalid, empty, populated, and failed responses and inspect denied HTML. | SSR tests |
| TEST-10 | Test islands from serialized props through interaction, async states, focus, and cleanup, mocking the typed API boundary rather than their own state transitions. | Tests exercise the island's behavior and recovery. | Manual: compare island tests with the DOM and owner-local query case lists, including stale results, aborts, retries, and disposal. | Island tests |
| TEST-11 | Test URL parsing and links together through reload, sharing, Back, Forward, browser-opened links, and enhanced-navigation failure. | Server and browser preserve the same selected resource and filters. | Manual: follow the URL test cases with JavaScript enabled and disabled; inspect rapid navigation and failed popstate coverage. |
URL tests |
| TEST-12 | Test realtime cursor recovery, duplicates, resync, access revocation, and disposal, advancing the cursor only after covering snapshots commit. | Reconnect and in-flight invalidation do not lose affected updates. | Manual: read recovery tests for an event during invalidation, a returning tab, reconnect, and revoked access. | Realtime tests |
| TEST-13 | Built-in apps: keep every repository check green. | Shared dependency, boundary, configuration, CI, formatting, and type contracts remain consistent. | bun run check; Manual: inspect each reported finding against its owning repository rule and verify the complete check result. |
Repository checks |
Release and deployment (REL)
| ID | Rule | Why | How to check | Canonical |
|---|---|---|---|---|
| REL-01 | Built-in apps: use Conventional Commit PR titles in the form type(scope): outcome, with fix for patch, feat for minor, and ! or BREAKING CHANGE: for major only with explicit maintainer approval. |
Squash titles drive release-please and breaking markers move every application image to a new major. | Manual: compare the PR title and body with the release table and verify maintainer approval for any breaking marker. | Release titles |
| REL-02 | Built-in apps: never edit version fields, CHANGELOG.md, or .release-please-manifest.json by hand. |
Release-please owns version and changelog changes. | Manual: inspect the diff for edits to version fields and release-owned files. | Release-owned files |
| REL-03 | Built-in apps: give runtime-contract changes a feat or approved feat! title and a note on the matching Operations page. |
Configuration, route, permission, data-format, and deployment changes land in the next Cloud release with operational guidance. | Manual: compare the runtime diff with the PR title and the matching Operations note. | Runtime contract releases |
| REL-04 | Built-in apps: derive the application set from the workspace through scripts/workspace.ts and keep both Compose files aligned with it. |
Image names, the release set, and the CI matrix share one application source. | bun scripts/check.ts app-set; Manual: compare workspaces.packages with compose.dev.yml and compose.prod.yml and inspect workflow application lists for workspace derivation. |
Application set |
| REL-05 | Built-in apps: show PR screenshots in light mode with demo data and only changed or decision-relevant states, using dark mode only for dark-rendering changes. | Public review exposes no person's data and uses a comparable baseline theme. | Manual: inspect each PR screenshot's theme, data, and relevance to the change or decision. | PR screenshots |
| REL-06 | Built-in apps: answer “How do we prevent this next time?” in a fix PR and add a lesson with what happened, root cause, error class, and guard. | The fix records its prevention, or explicitly names the pending guard or why none fits. | Manual: compare the fix PR with its lesson entry and verify suspected causes and pending guards are labelled as the guide requires. | Add a lesson |