MK-SH-V
A 260M visual decision model: calibrated probabilities, a choice, and a refusal below the gate.
Real cases from the training splits, one click. Or drop your own page or screenshot below and press Decide.
The model answers — confident, and correct (checked against each source's gold label)
On screenshots the claim is end-of-task screen detection. Checking that the goal in the State is the one finished is only partial (measured in E47: paired with an unrelated goal it still says yes about 22 % of the time), so it is not claimed.
Where it fails — a refusal below tau 0.55, and a confident wrong answer above it (real receipts: about 1 in 4 kept answers is wrong)
This demo runs the released policy (E47 checkpoint, seed 43): temperature T = 1.124 and gate tau = 0.55, fitted on calibration splits disjoint from every reported set. Held-out validation, exact two-sided Clopper-Pearson 95 % (budget: 5 % selective risk):
| Set | Coverage | Selective risk | CP95 upper |
|---|---|---|---|
| synthetic test s43 (shipped) | 0.986 | 3.45 % | 4.1 % (held) |
| synthetic test s42 | 1.000 | 4.14 % | 4.8 % (held) |
| real GUI s43/s42 | 0.973 / 1.000 | 16.1 / 20.0 % | above budget |
| real CORD s43/s42 | 0.892 / 1.000 | 25.3 / 26.6 % | above budget |
| 3,000 fresh numeric documents s43 | 0.876 | 22.9 % | above budget |
The budget holds on the synthetic test only. On real images abstention is the campaign's declared open problem, and this demo shows it unfiltered.
Example images: CORD v2 (Park et al. 2019, NAVER Clova) and GUI-Odyssey (Lu et al. 2024, OpenGVLab), both CC BY 4.0. Model: Apache-2.0.