LiveUIBench evaluates Agent-as-App systems as complete workflows
Submitted October 20, LiveUIBench moves generated-app evaluation beyond opening a screen to submission, revision, shared state across applications and return visits.
Why it matters
A generative interface can pass many local checks while leaving the user's goal unfinished. The benchmark requires visible results, service operations, saved state and the intended record to remain correct across one ordered workflow.
A closer look
LiveUIAgent generates an orchestration linking interface elements, action goals and retained information while a shared runtime handles later requests. LiveUIBench assigns 500 tasks and 10,590 assertions across Spotify, Amazon, Gmail, Phone, Venmo, Splitwise, Todoist, Simple Note and File System: 302 single-app, 130 multi-tool and 68 shared-world tasks. The authors report low full-workflow completion for every evaluated model; the best overall FWR is 19.60%, and higher partial assertion coverage does not necessarily mean more completed workflows.
Who this is for
Teams building generative UI, agent-app runtimes, persistent state and end-to-end evaluations
Original sources
Revisions & corrections (1)
— New original research discovered outside the existing source directory; full paper and submission record verified, with attributed results, explicit limitations and counterexamples.