Datasetcredit-nov

credit-nov5 findings traces-nov5 findings

Ava L.

Improve

Improve shows what changed in your agents, whether each change made them better or worse, and what Auga proposes to change next.

Waiting for you

1

Review the proposal

Being measured

0

Made it worse

1

Bureau feed 3.3

No clear answer

2

Outcome and every change

Share of runs denied per half hour, 400 runs over 12.0 h

Score model 2.3.1 Bureau feed 3.3 Policy 2026.10.1

Changes

Proposed, live, measured

No	Change	Came from	State	Verdict	Effect on denials	Compared by
IMP-5	Return the bureau feed to 3.2 · Data	Auga, from IMP-2	Proposed	Waiting for a person
IMP-3	Policy 2026.10 → 2026.10.1 · Policy	Seen in telemetry at +6.00 h	Measured	No clear answer	+13 pts	Before and after only
IMP-2	Bureau feed 3.2 → 3.3 · Data	Seen in telemetry at +5.34 h	Measured	Worse	+84 pts	Same hours
IMP-1	Score model 2.3 → 2.3.1 · Model	Seen in telemetry at +3.25 h	Measured	No clear answer	−5 pts	Before and after only
IMP-4	· Data	Seen in telemetry, 11 changes	Measured	Regular updates, not scored

What this page depends on

Open any part to see what it will do and how much is built.

Measure

Know what changed and whether it mattered.

Partly
Partly
Partly
Not built
Partly
In Auga

Propose

Say what to change, and why Auga may say it.

Partly
Not built
Partly
Prototype

Roll out

Bring the change into your code and traffic safely.

Not built
Not built
Not built
Not built
In Auga

Account

Keep a record that a person can be responsible for.

In Auga
In Auga
Not built
Partly

The page is a prototype and a working draft. Auga computes the measured figures from Rootplant credit-4217, a synthetic file of 400 runs. The proposal and the rollout are simulated. Dashed links show planned features and open a note about them.

Repository connection

Not built

What it will do

The repository connection links your organisation's repository and maps each signal in the telemetry to the file and line that sets it.

Why Improve needs it

No proposal can become a pull request without the repository connection. Write access needs far more trust than receiving filtered run data.

Today

The repository connection is not built. The diff on this page is an example, and no repository is connected.

Improve's own mandate

Partly

What it will do

Auga follows the same stages as agents, area by area. It earns the right to suggest in text, then to open configuration PRs, then code PRs.

Why Improve needs it

Self-improvement should not get more rights than the evidence supports, and the stage should fall when a proposal does harm.

Today

Mandates govern agents today (/mandates). Nothing applies them to Auga's own proposals.

Rollout control

Not built

What it will do

Rollout control reads or sets the share of traffic on the new value. It uses your organisation's feature flags or deployment tool.

Why Improve needs it

A comparison with runs on the old version at the same time is possible only while both versions run. The exact term is concurrent control. After a sudden change, only a weaker before-and-after comparison remains.

Today

Rollout control is not built. The rollout on this page is simulated.

Choice of outcome

Partly

What it will do

A person says what counts as a failed run for each agent, and whether each run can be judged. Auga can follow several outcomes for one change.

Why Improve needs it

Here a denial counts as a failed run. A denial can be correct, so this figure only says that denials moved, not that they were wrong.

Today

A person sets the outcome by hand in the health report, with --outcome key=value. In Inbox, the engine's judges decide the outcome.

Change feed

Partly

What it will do

The change feed finds every change of model, prompt, policy and data version in live telemetry by itself. It adds a row here as soon as one appears.

Why Improve needs it

Improve can only measure a change it knows about. Values that update on a regular schedule, such as the hourly sanctions list, are listed but not scored.

Today

Today, auga/health.py on branch feat/agent-health finds changes in one uploaded file. It has no live feed.

Basemath, the measurement service

Partly

What it will do

Basemath runs this comparison as a service on live traffic. It covers every change, each agent and each outcome, with one API for this page.

Why Improve needs it

Basemath produces every result on this page. After a proposal goes live, the same comparison measures what it did.

Today

The comparison exists in auga/health.py on a branch. This page contains a copy of it, checked against that report. There is no endpoint yet.

Rule for when to stop

Not built

What it will do

A test decides when a measurement may stop. It corrects for checking again after each run. The exact term is a sequential test.

Why Improve needs it

This page checks the likely ranges (95% intervals) after every run. Checking that often makes any moment when the ranges stop overlapping look more certain than it is.

Today

The rule for when to stop is not in the product. Mandate Simulator has a Bernoulli CUSUM alarm as a separate prototype.

Run view

Partly

What it will do

The run view opens the single run behind a mark. It shows the steps, the values and the path through the agents, but only data that may pass the data boundary.

Why Improve needs it

A person has to be able to check a result against real runs before believing it.

Today

Evidence has a run space. A full view of one run does not exist yet.

Cause chain in Evidence

In Auga

What it will do

The link opens the graph at this change, with its grade and the path to the outcome. A result can then show how the change had that effect.

Why Improve needs it

Measuring shows that something moved. Evidence can show why, and which setting to change. Proposals need that answer.

Today

Evidence exists. Its screening stops at 15 signals, and in our test suite that kept real links out of the graph.

Independent rule check

Partly

What it will do

Auga also checks every measured improvement against the result of a rule, and the agent cannot learn to satisfy that rule.

Why Improve needs it

If an automatic judge scores success, proposals can start to follow the preference of the judge rather than the outcome.

Today

auga/judgebias.py exists. The fix that lets it run on Inbox data is PR augalabs/augapro#113, which was not merged at the last check.

Proposal writer

Not built

What it will do

The proposal writer turns a finding into a concrete change and a written reason. Statistics choose what to change, and a model only writes it down.

Why Improve needs it

A cause is not an action. Knowing that DTI drives denials does not say what to edit. A person wrote the proposal on this page by hand.

Today

The proposal writer is not built. The rule stays that a model may not create or remove a link in the graph.

Simulation before rollout

Prototype

What it will do

The simulation replays history against the proposal. It shows which runs the proposal would have changed, before any real traffic reaches it.

Why Improve needs it

The simulation shows a reviewer how many runs the change would affect. It cannot show what would have happened afterwards.

Today

Mandate Simulator is a separate prototype for stages, and it is outside the repository.

Draft pull request

Not built

What it will do

Auga opens a draft pull request (PR) with the evidence, the expected effect and how to undo the change. Configuration comes first and code last, and a person always approves.

Why Improve needs it

Here a proposal enters your organisation's own review and release process.

Today

The feature is not built.

Automatic stop

Not built

What it will do

The automatic stop ends a rollout and returns to the old version when the outcome or a guardrail fails. It can also lower the stage of the agent concerned.

Why Improve needs it

When a rollout fails, a person should not have to be watching this page.

Today

The feature is not built.

Data boundary

In Auga

What it will do

The proposal writer and the repository connection run on your organisation's side of the data boundary. This page shows what crossed the boundary.

Why Improve needs it

Writing a good change needs prompts and code, which today's boundary is built to keep out.

Today

Auga edge and its receipt exist (/boundary). A design for content with consent was sketched, but the docs still say something else.

Decision review

In Auga

What it will do

Proposals join the same queue as guarded decisions, with the evidence attached, so a person answers in one place.

Why Improve needs it

Approve and Reject on this page should write to the same review, not to a new inbox.

Today

Decision review exists (/review) for guarded decisions and wording. Proposals are not in it.

The ledger

In Auga

What it will do

Each step is signed into the ledger: finding, proposal, approval, rollout, result. An auditor can follow one change from start to end.

Why Improve needs it

A system that changes itself is acceptable only if every change can be traced afterwards.

Today

The signed ledger exists (/ledger) for guarded decisions. These types of entry do not exist.

Value

Not built

What it will do

Auga turns a measured effect into money, as a range with its grade and the stated assumptions. Observed costs and estimated savings stay separate.

Why Improve needs it

A measured effect of a real change is easier to defend to a controller than a modelled saving.

Today

Only a plan exists. The old Cost Optimization module was a demo.

Agent Health

Partly

What it will do

The Agent Health page of each agent lists the changes that moved it and links here.

Why Improve needs it

Agent Health shows the current state. Improve follows what happens when that state changes, and that is the reason to return.

Today

The code is on branch feat/agent-health. It has no server route, no dashboard view and no link to mandates.