Bureau feed 3.2 → 3.3 Worse
Use this page to see what the bureau feed change did to denials. Telemetry showed the change in credit.record_feed_version 5.34 hours after the start of the file. Auga did not propose this change, so this page starts with the measurement.
The measurement
Runs that stayed on 3.2 in the same hours had the same traffic, time of day and other changes. Denials of 100% against 16% suggest that the change itself made the difference. Next, see the proposal to return to the old version.
What moved with the change
| Measure | 3.2 | 3.3 |
|---|---|---|
| Denied | 16% | 100% |
| Median DTI | 0.27 | 222.28 |
| Median reported income | 47,000 | 52 |
| Median risk score | 736 | 742 |
| Median duration | 3.7 s | 3.7 s |
| Tokens per run | 215 | 299 |
| Failed steps per run | 0.03 | 0.01 |
What the measurement cannot say
We cannot say why denials moved. The table lists what moved at the same time, and that is not a proven cause chain. can show the reason.
We cannot say whether the denials were wrong. We can say only that they moved. The file is synthetic, with a fault added on purpose, and real denials can be right.
We cannot say how sure we are of any moment when the ranges stop overlapping. Checking after every run makes such a moment look more certain than it is.
We cannot say whether the split was fair. Nobody assigned runs to 3.2 or 3.3 at random, so the two groups could differ in other ways.
The page is a prototype and a working draft. Auga computes the measured figures from Rootplant credit-4217, a synthetic file of 400 runs. The proposal and the rollout are simulated. Dashed links show planned features and open a note about them.
How the groups compare
The likely ranges do not overlap, so the two groups are clearly different.
More exactly
Denied on 3.2: 24 of 152 runs, 16%, 95% interval 11% to 22%. Denied on 3.3: 70 of 70 runs, 100%, 95% interval 95% to 100%. The intervals separated at +6.45 h, after 10 runs on 3.3, against a concurrent control that was not randomised.