ResearchResearch paper

TaoD2C-Bench shows visual similarity is not implementation correctness

Researchers from Zhejiang University and Taobao & Tmall introduced TaoD2C-Bench, pairing 2,861 production interfaces with 97,652 expert annotations for component, grouping, alignment and positioning constraints.

Why it matters

Generated UI can resemble a design while selecting the wrong component, breaking container relationships or encoding layouts that fail under changing state. The benchmark separates appearance from implementation requirements.

A closer look

The dataset spans 17 commercial platforms with design images, layer metadata and target component libraries; eight models were evaluated on a reviewed subset of 421 designs. Mean end-to-end F1 was 57.37 for component type, 37.19 for properties, 53.04 for grouping, 23.30 for alignment and 32.22 for positioning. Supplying expert requirements improved every code dimension while average visual scores barely changed, so appearance metrics cannot substitute for implementation checks.

Who this is for

Generative-UI and design-to-code researchers, frontend platforms and design-system teams

Original sources

Revisions & corrections (1)

— Primary arXiv abstract and full HTML paper were read. Submission date, affiliations, dataset scale, evaluated subset, model count, per-dimension results, visual-versus-code finding and licensing limits were verified. This is a distinct industrial UI-code evaluation event not present in the catalog.

Related events

Research

A second, independent VeriCUA turns GUI-task acceptance into active evidence gathering with abstention

A Google and University of Hong Kong team submitted a separate project also named VeriCUA on October 29. It inspects an environment or recording before constructing a task rubric, then gathers missing evidence through GUI, accessibility-tree, file and database reads, marking requirements met, unmet or unknown. In online experiments it reduces false positives against baselines on AndroidWorld, OSWorld and ScienceBoard.

Research

GUI‑JEPA replaces token-by-token action decoding with candidate-conditioned latent scoring

Submitted October 27, GUI‑JEPA targets single-step mobile GUI action selection. It transforms JEPA representations pretrained on unlabeled interface screenshots for each action and region candidate, then scores accessibility-tree candidates in one batch instead of asking a 7B vision-language model to decode an action token by token.

Research

TAM turns verified task advancement into online state and reusable operations for GUI agents

Submitted October 27, Task Advancement Memory (TAM) uses what has been reliably accomplished at two timescales: it maintains verified progress, remaining goals and visual evidence during a task, then extracts reusable operations from locally successful segments after the task, including segments inside an overall failed workflow.