ResearchResearch paper

InterWebCoder trains multi-step webpages with executable tests

Submitted October 20, InterWebCoder replaces screenshot or GUI-agent rewards with replayable test scripts and state-level visual similarity for interactive webpage generation.

Why it matters

A rendered webpage does not establish that clicks, input, navigation and state transitions work. Executable tests provide faster, inspectable feedback for interaction behavior.

A closer look

Its TEDS pipeline reconstructs self-contained webpages and tests from real interaction trajectories, then attributes failures to either page code or the test. The authors retain 1,736 verified samples from 2,058 tasks and combine test-ID-weighted SFT with GRPO over test pass rate and state-level visual similarity. They report final scores of 48.54 on IWR-Bench and 48.42 on Vision2Web with a 9B model; local tests reduce reward computation from 92.40–130.01 seconds to 2.08 seconds per sample.

Who this is for

Teams working on interactive webpage generation, UI-to-code, test-driven training and generative-UI evaluation

Original sources

Revisions & corrections (1)

— New original research discovered outside the existing source directory; full paper and submission record verified, with attributed results, explicit limitations and counterexamples.

Related events

Research

A second, independent VeriCUA turns GUI-task acceptance into active evidence gathering with abstention

A Google and University of Hong Kong team submitted a separate project also named VeriCUA on October 29. It inspects an environment or recording before constructing a task rubric, then gathers missing evidence through GUI, accessibility-tree, file and database reads, marking requirements met, unmet or unknown. In online experiments it reduces false positives against baselines on AndroidWorld, OSWorld and ScienceBoard.

Research

GUI‑JEPA replaces token-by-token action decoding with candidate-conditioned latent scoring

Submitted October 27, GUI‑JEPA targets single-step mobile GUI action selection. It transforms JEPA representations pretrained on unlabeled interface screenshots for each action and region candidate, then scores accessibility-tree candidates in one batch instead of asking a 7B vision-language model to decode an action token by token.

Research

TAM turns verified task advancement into online state and reusable operations for GUI agents

Submitted October 27, Task Advancement Memory (TAM) uses what has been reliably accomplished at two timescales: it maintains verified progress, remaining goals and visual evidence during a task, then extracts reusable operations from locally successful segments after the task, including segments inside an overall failed workflow.