A GUI agent's success report can leak into completion judgment
Submitted October 21 and accepted to the ICSE 2027 research track, this paper finds that exposing a completion judge to the agent's own success report reduces detection of dangerous misreports. It rebuilds the boundary with report isolation, structured trace evidence and deterministic rules.
Why it matters
A generative interface's “completed” label cannot serve as both the claim under review and evidence for that claim. Self-report should be separated from independently recorded execution traces so confident language cannot silently turn failure into a pass.
A closer look
With the task and trace held constant, withholding the agent report improves trace-side dangerous-misreport recall by 43.4 percentage points for Qwen3.5-35B-A3B and 13.3 points for DeepSeek-284B-A13B. The structured auditor reaches F1 0.75–0.78 on 465 audit-target records and 0.77–0.81 on the 130-record held-out Batch 2. When both FLAG and REVIEW block automatic pass, it blocks 74%–90% of dangerous misreports at 45%–56% inspection load within the audit-target set.
Who this is for
Teams evaluating GUI agents, automated testing, generative-UI completion and high-stakes workflows
Original sources
Revisions & corrections (1)
— The original abstract and full PDF provide a direct GUI-agent completion-judging result with held-out evaluation, deployment trade-offs and explicit limitations.