UI‑TAU Bench connects generated interfaces to database tasks, user input and final-state evaluation

Submitted October 8, the GenUI‑Harness paper introduces UI‑TAU Bench: an agent first retrieves read-only evidence, a GUI Coder generates an ephemeral interface for missing information, and a tool agent executes the action, with the complete interaction graded by final database state.

Why it matters

Generative UI should not be judged only by screenshots, executable code or component-schema validity. This work links retrieved evidence, user input, submitted UI payloads, tool actions and terminal state, while directly addressing reward hacking when an LLM judge mistakes superficial polish for usable interaction.

A closer look

UI‑TAU Bench builds domain databases from ten public datasets, with 300 Lite tasks and 1,000 Full tasks. Generated React or TSX interfaces must expose unresolved arguments and support the downstream action. Dynamic UX executes interfaces in isolated sessions, and a Reward Auditor revises the LLM-judge rubric using actual task outcomes. On Lite, GenUI‑Harness improves macro-average Pass@3 over smolagents by 4.48 percentage points across nine shared backbones, with a 95% confidence interval of 2.37–6.59. A role-trained 4B system rises from 9.33% to 58.00%. In a survey restricted to consistently usable interfaces, average dialogue rounds fall from 3.4 to 1.2.

Who this is for

Teams studying end-to-end generative-UI evaluation, GUI-code training, user simulation, reward auditing and database-backed tool agents

Original sources

• When Interfaces Speak paper and GenUI-Agent repository ↗arxiv.org
• Official GenUI-Agent training, evaluation and Dynamic UX code ↗github.com