What-If
What-If is a calculator for all the agents in your organisation. Several agents together can harm or help the company, so the page looks at them as a whole. Change the assumptions, see the result, and create reports for top management.
37 of 37 prices are illustrative examples, and none has been researched. Replace them before you quote a figure. The measurements come from two synthetic Rootplant files with 400 applications and 250 shipments over about 12 hours. The figures are an estimate from a model and are not accounting figures.
What if
A bad change is caught and reversed after 3.3 h
Bad changes reach production 4 times a year, in each system
25% of them affect every system at once, through a shared data feed, model or vendor
People check 0% of denials and catch 70% of the wrong ones
Run volume is ×1 today's rate
Model prices change by 0%
0.0% of approved business exists only because of the agents
Making this happen costs €0 a year
Catch it sooner adds €1.12M a year compared with Today, and most of that comes from catching bad changes sooner.
Who helps and who harms
How to read the chart
The hatched area above the line is the work an agent does in place of people. The solid area below the line is what the agent costs to run. The red area is its share of the loss from bad changes. The dashed outline shows Today.
The red line is the fault path. The line joins the agents whose own output changed when the feed changed, measured on runs side by side in the same hours. The changes happened together, and only the causal analysis can show a cause chain.
What if
A bad change is caught and reversed after 6.6 h (as observed)
Bad changes reach production 4 times a year, in each system
25% of them affect every system at once, through a shared data feed, model or vendor
People check 0% of denials and catch 70% of the wrong ones
Run volume is ×1 today's rate
Model prices change by 0%
0.0% of approved business exists only because of the agents
Making this happen costs €0 a year
This model estimates that the agents return €5.89M a year. That is the work they do in place of people, minus what they cost and the loss from bad changes.
Who helps and who harms
How to read the chart
The hatched area above the line is the work an agent does in place of people. The solid area below the line is what the agent costs to run. The red area is its share of the loss from bad changes. The dashed outline shows Today.
The red line is the fault path. The line joins the agents whose own output changed when the feed changed, measured on runs side by side in the same hours. The changes happened together, and only the causal analysis can show a cause chain.
What if
A bad change is caught and reversed after 6.6 h (as observed)
Bad changes reach production 4 times a year, in each system
25% of them affect every system at once, through a shared data feed, model or vendor
People check 100% of denials and catch 70% of the wrong ones
Run volume is ×1 today's rate
Model prices change by 0%
0.0% of approved business exists only because of the agents
Making this happen costs €0 a year
People check denials adds €801k a year compared with Today, and most of that comes from people checking denials.
Who helps and who harms
How to read the chart
The hatched area above the line is the work an agent does in place of people. The solid area below the line is what the agent costs to run. The red area is its share of the loss from bad changes. The dashed outline shows Today.
The red line is the fault path. The line joins the agents whose own output changed when the feed changed, measured on runs side by side in the same hours. The changes happened together, and only the causal analysis can show a cause chain.
What if
A bad change is caught and reversed after 6.6 h (as observed)
Bad changes reach production 4 times a year, in each system
25% of them affect every system at once, through a shared data feed, model or vendor
People check 0% of denials and catch 70% of the wrong ones
Run volume is ×3 today's rate
Model prices change by 0%
0.0% of approved business exists only because of the agents
Making this happen costs €0 a year
Triple the volume adds €11.78M a year compared with Today, and most of that comes from volume.
Who helps and who harms
How to read the chart
The hatched area above the line is the work an agent does in place of people. The solid area below the line is what the agent costs to run. The red area is its share of the loss from bad changes. The dashed outline shows Today.
The red line is the fault path. The line joins the agents whose own output changed when the feed changed, measured on runs side by side in the same hours. The changes happened together, and only the causal analysis can show a cause chain.
The systems in this sum
You can add any Agent Health report, as a .json file or as the report page. If you also add the run file, which holds the recorded runs, the agents are measured exactly, with the fault path. The files are read in this browser and stay there.
| System | From | Agent Health | Runs | Status | |
|---|---|---|---|---|---|
| Loan decisions | built-in demo | UNHEALTHY | 400 | in the sum, fault path placed | |
| Cold-chain dispatch | built-in demo | UNHEALTHY | 250 | in the sum, fault path placed |
What the figures are based on
Prices are in EUR. On the live page, when you enter your own price, its source changes from example to own. Measured figures come from the run files, and you cannot edit them here. How many runs a person reviews depends on each agent's stage: Preview a stage, under Mandates, shows it for a day of traffic.
| Working days a year | example | |
| A wrong outcome becomes a complaint | example | |
| Cost of a complaint | example | |
| How much detection time varies (log σ) | example | |
| How much the cost of a wrong outcome varies (log σ) | example | |
| Share work between agents | convention | |
| Show money in | ||
| 1 EUR = ? EUR | fixed |
| Minutes a person would spend on one application | example | |
| Cost of a person's minute | example | |
| Minutes to review one run | example | |
| Wrong outcome, share of the 277,000 average amount | example | |
| Margin on approved, share of amount a year | example | |
| Runs a day | extrapolated | |
| Compute cost per second of step time | example | |
| model-a-large: per 1k tokens in | example | |
| model-a-large: per 1k tokens out | example | |
| model-b-small: per 1k tokens in | example | |
| model-b-small: per 1k tokens out | example | |
| bureau_fetch: per call | example | |
| document_parse: per call | example | |
| income_verify: per call | example | |
| notify_applicant: per call | example | |
| policy_engine: per call | example | |
| sanctions_screen: per call | example |
| Runs in the period | 400 | measured |
| Window | 11.95 h | measured |
| Outcomes | approve 236, refer 30, deny 134 | measured |
| Strongest change | credit.record_feed_version 3.2 → 3.3 at 5.34 h | measured |
| Runs on the new value | 70 | measured |
| Extra denials compared with the old version | 84% (72% to 89%) | measured |
| Still active when the file ends | 6.6 h | measured |
| Doubling alarm triggers | 3.3 h after the change | measured |
| Other changes | score_model_version: no runs on the old version at the same time; policy_version: no runs on the old version at the same time | measured |
| Health report: report-credit-4217.html | ||
| Minutes a person would spend on one shipment | example | |
| Cost of a person's minute | example | |
| Minutes to review one run | example | |
| Cost of one wrong outcome | example | |
| Margin on one approved run | example | |
| Runs a day | extrapolated | |
| Compute cost per second of step time | example | |
| route-net-fast: per 1k tokens in | example | |
| route-net-fast: per 1k tokens out | example | |
| route-net-large: per 1k tokens in | example | |
| route-net-large: per 1k tokens out | example | |
| corridor_fetch: per call | example | |
| customs_screen: per call | example | |
| manifest_parse: per call | example | |
| notify_shipper: per call | example | |
| reefer_verify: per call | example | |
| sla_rules_engine: per call | example |
| Runs in the period | 250 | measured |
| Window | 11.97 h | measured |
| Outcomes | approve 136, deny 87, refer 27 | measured |
| Strongest change | logistics.record_feed_version 4.0 → 4.1 at 5.34 h | measured |
| Runs on the new value | 43 | measured |
| Extra denials compared with the old version | 81% (64% to 88%) | measured |
| Still active when the file ends | 6.6 h | measured |
| Doubling alarm triggers | 3.2 h after the change | measured |
| Other changes | eta_model_version: no runs on the old version at the same time; policy_version: no runs on the old version at the same time | measured |
| Health report: report-defence-9912.html | ||
Systems without a run file
Enter an agent system by hand to include it in the sum. The system appears as one column, marked as entered by hand. Only the shared sliders affect it.
Not in this number
- Fines: a fine depends on the regulator and the situation, so the sum excludes fines.
- Wrong outcomes when nothing changed: the run files cannot show them, so the normal error rate counts as zero.
- Wrong approvals: the loss here is only the rise in the share of failed runs after a change.
- Customers who leave after a complaint: the sum does not count them.
- Proof of cause for the fault path: the agents changed together while other runs stayed on the old version. Only the causal analysis can show a cause chain.
- A fault rate that changes over time: the model continues the observed rate unchanged after the 6.6 hours that were actually seen.
This page is a prototype. Its values are fixed at the starting values, so the sliders do not move and the columns do not open. The live page calculates again as you drag, simulates the years again and accepts your own reports. Point at a column to see its figures.
Fault path and range of years
The red line shows agents that changed together. Of the simulated years, 1 year in 10 is worse than the low figure and 1 year in 10 is better than the high figure.
More exactly
The fault path is an association measured under a concurrent control, and it is not a causal finding. The low figure is P10 and the high figure is P90 of net value over 3,000 simulated years with one fixed seed.
Measures in the tables
The table compares denials with runs that stayed on the old version at the same time.
More exactly
Extra denials are the share denied beyond the concurrent control, with the likely range in brackets. The spread inputs are log σ, the standard deviation of the logarithm of the value.