Testing
This page is how tests are written so a signature change edits one file rather than a hundred call sites. AgentUp.Architecture.Tests already enforces the rules.
AgentUp.Verification.Tests was written to this standard deliberately and is the worked
example to copy. The rules below are enforced by AgentUp.Architecture.Tests, so this page
describes what the build already checks rather than a convention to remember.
Test data goes through builders
Production DTOs and models are not constructed in tests. Each test project keeps a builder
per coupling hotspot in its Support/ folder:
var workspace = ServerDomain.Workspace()
.Named("Shop")
.WithApplication(new ApplicationDefinitionBuilder("Api", "dotnet run").WithPort(8080))
.Build();
A builder starts from a sensible default and lets a test state only the attribute it is
actually about. RegisterWorkspaceRequest has five positional fields and five collection
properties; adding one now changes RegisterWorkspaceRequestBuilder rather than the hundred
registrations across the suite.
TestDataBuilders enforces this in two halves:
- A production DTO or model under
Features/*/DTOsorFeatures/*/Models, with more than three constructor parameters, constructed more than fifteen times in one test project, must have aSupport/<Type>Builder.csin that project. A narrower type reads fine positionally and is left alone; the threshold catches a type as it becomes a hotspot. - Once that builder exists, nothing else in the project constructs the type, including
through an implicit
new(...), which is judged by the type it is written into.
Domain vocabulary is shared and named
Each test project has one file naming its world: ServerDomain, CliDomain,
DesktopDomain, DebugDomain, VerificationDomain. One canonical workspace, one
application, one port, one queued commit, named once and reused.
Assert.That(overview.WorktreePath, Is.EqualTo(ServerDomain.WorktreePath));
Tests refer to those names rather than inventing path, branch and commit strings, so a reader can tell at a glance whether two tests are talking about the same workspace or different ones. Where a test needs one value to be specific, it injects that value through a builder instead of restating the whole world.
Variation is a call, not a new fixture method
Canned fixtures fail at the first test that needs "the same thing, but different". A builder returned from the domain gives that test a seam:
DesktopDomain.Workspace().Stopped().Build();
DesktopDomain.WorkspaceServing(port: 10200).Build();
Domain helpers that stand for a shape - WorkspaceServing, WorkspaceWithApplications -
return a builder rather than a finished object, so the next variation does not need another
method on a shared file.
Setup lives in the test
New fixtures arrange their subject inside the test, through a local helper the test calls with explicit parameters:
private static async Task<(AppDriver App, ItemsControl Transcript)> OpenTranscriptWithAThoughtAsync()
A [SetUp] method moves the arrangement out of the test that depends on it, and every test
in the fixture then pays for whatever the slowest one needed. TestFixtureSetup bans
[SetUp] and [OneTimeSetUp] in new fixtures, over a ratchet baseline of the fixtures that
predate the rule: entries come out as the setup moves, and the rule fails on an entry that is
already clean, so the file cannot drift into fiction. [TearDown] is untouched - releasing a
temp directory says nothing about what a test can be read to mean.
Composition helpers compose services, not data
AgentUp.Server.Tests/Fake/ServerTestComposition.cs assembles the object graphs that are too
large to build at every test. It composes services only; test data comes from the builders.
Every parameter that changes behaviour under test - the configuration provider, the identity
provider, the platform branch - is required rather than defaulted behind a ??, so reading a
call still tells you what the subject was given. Where a plain composition is common it gets
a named overload that states what it stands for, rather than a null-coalescing default that
hides the choice.
One behaviour per test
TestAssertionDensity caps a test method at ten assertions. Roughly three quarters of the
suite already carries one to three, which is healthy; the cap exists to catch the tail. A
method asserting twenty or thirty things names one behaviour and covers several, so a failure
does not say what broke and the method can only grow.
Splitting usually means either [TestCase] per instance of the same rule:
[TestCase("DISPLAY", ":123")]
[TestCase("GDK_BACKEND", "x11")]
public async Task Prepare_putsTheHostedDisplayIntoTheApplicationEnvironment(string name, string expected)
or one test per concern over a shared local helper. Assume.That counts towards the cap. Neither
Assume.That nor Assert.Ignore may decide at runtime that platform, privilege, or live system
state makes a test inapplicable: that reports success without having run. Cover those branches
through an injected capability provider instead. The runtime-skip baseline is empty. It rejects
additions and stale entries, so a new Assert.Ignore or Assume.That on live platform state
fails the suite.
Category=LoopInference is the Server loop kernel driven by the locally-inferring
test-agent-infer ACP agent. It is slow and ciOnly. Keep it out of
run-dotnet-tests.sh, the fast server check, and the test-kind watchdog.
Category=OpencodeInference runs the real OpenCode binary through the built-in OpenCode
capability against test-model-server. run-opencode-inference.sh supplies the binary,
endpoint, per-run token, and a published Server. One layer runs the loop kernel in process. The
other starts that Server as a process and drives it only over REST and the agent event stream,
the way a client does, including an Agent-Up MCP tool call and steering sent mid-turn. The write
journey also requires OpenCode to report the written content in a form the transcript can draw as
a diff. It is slow and ciOnly, and kept out of the same suites.
Category=AgentMcpE2E drives a Server from the test output with the sandboxed
test-agent-scripted, whose prompts are MCP directives: it starts a real app through orchestration
and drives it in the real headless browser, its session reaches only its own workspace, and a dev
reviewer is refused orchestration writes, the browser, and the commit queue. run-agent-e2e.sh
runs it beside the sandbox category. The OpenCode journeys hold the live CPU-model counterpart.
Category=AgentSandboxE2E starts a shell through the agent sandbox with real bubblewrap and
requires the kernel to refuse what the member may not do. run-agent-e2e.sh supplies
bubblewrap from Nix. It is slow, needs nix, and is kept out of the same suites. The OpenCode
journeys put bubblewrap on the published Server's PATH, and one requires a dev reviewer's OpenCode
to fail a write a person allowed.
A recorded session is the fast regression for that live path.
AgentUp.Chat/fixtures/opencode-cpu-session.jsonl is a real OpenCode session against the CPU
model. Desktop's AgentGoldenTranscriptTests and Chat's GoldenTranscript.test.ts replay it
through each client's own reducers, so both clients are held to the same transcript.
Concurrency and restart for autonomous workspaces run in the normal E2E
tier. Category=LoopSoak is hundreds of scripted tasks plus bounded-growth
asserts. It is slow and ciOnly. Keep it out of those same suites.
loop-scripted is the fast AgentUp.TestAgents.Tests Scripted slice plus
Server ScriptedLoop Provider tests.
Category=ForgejoE2E hosts a digest-pinned Forgejo container and drives the
first-party module plus the Server Forge slice against it. It is slow and
ciOnly, requires Docker, and stays out of those same suites. Missing Docker
fails the run. CI runs it with ./.github/scripts/run-forgejo-e2e.sh.
Mobile npm test discovers src/features/**/*.test.ts recursively, so a new feature's tests run without editing the script. MobileFeatureTestCoverage fails a src/features/<slice> that has production TypeScript outside components/ with no tests, and a providers/ folder whose production files have no sibling tests. React context controllers and Expo hooks stay thin and delegate to those providers. Component rendering is not gated: do not add react-native-testing-library for product screens. WebAudit npm test runs in the Mobile CI jobs on every pull request, not only the release publish step.
Rules assert properties, not membership
An architecture rule that pins an exact set fails on any legitimate addition while verifying
no behaviour. ReleasePayloadLayout used to assert the payload directories were exactly
cli, desktop, installer, server, tray; it now asserts what CI depends on - that every
payload directory is non-empty and unique - which a new payload satisfies and a colliding one
does not.