Score model 2.3 → 2.3.1 No clear answer
Use this page to check whether the score model change moved denials. Telemetry showed the change in credit.score_model_version at +3.25 h. Nobody at Auga proposed this change, so the measurement starts here.
The measurement
We cannot tell the runs before and after the change apart (−5 percentage points). The old value ended at the change, so no runs on the old value remain to compare with. Next, read What the measurement cannot say.
What moved with the change
| Measure | Before | After |
|---|---|---|
| Denied | 23% | 18% |
| Median DTI | 0.28 | 0.27 |
| Median reported income | 47,500 | 44,500 |
| Median risk score | 737 | 740 |
| Median duration | 3.5 s | 3.6 s |
| Tokens per run | 173 | 179 |
| Failed steps per run | 0.02 | 0.02 |
What the measurement cannot say
We cannot say why denials moved. The table lists what moved at the same time, and that is not a proven cause chain. can show the reason.
We cannot say whether the denials were wrong. We can say only that they moved. The file is synthetic, with a fault added on purpose, and real denials can be right.
We cannot say how sure we are of any moment when the ranges stop overlapping. Checking after every run makes such a moment look more certain than it is.
We cannot say which change moved denials. No runs stayed on the old value, so anything else that changed at that time looks the same.
The page is a prototype and a working draft. Auga computes the measured figures from Rootplant credit-4217, a synthetic file of 400 runs. The proposal and the rollout are simulated. Dashed links show planned features and open a note about them.
How the groups compare
The likely ranges overlap, so we cannot tell the runs before and after the change apart.
More exactly
Denied before the change: 14 of 60 runs, 23%, 95% interval 14% to 35%. Denied after the change: 11 of 60 runs, 18%, 95% interval 11% to 30%. The intervals overlap, so the result is not separated.