AG-UI.AIGENERATIVE UI INTELLIGENCE
中文Subscribe ↗
ResearchResearch paper

ANIS synthesizes agent-native interfaces from software evolution

Published October 11, ANIS builds DeskIF-Bench from historical software releases and attempts to compress agent-facing surfaces into discoverable, atomic and verifiable command sets.

Event Source published Indexed Verified

Why it matters

Exposing either a full human interface or a low-level SDK can leave agents with a large search space. The study turns interface design itself into an optimization problem measured by task success and artifact state.

A closer look

DeskIF-Bench contains 255 artifact-verified tasks across six applications and compares historical versions with the same scaffold, tasks and verifier. ANIS is then evaluated on 368 tasks across eight applications. The authors report DeepSeek V4 Flash overall pass@3 rising from 68.5% with native SDKs to 72.3% with atomic interfaces and 74.5% with goal-level interfaces, while both model families use roughly 73%–75% fewer tokens. Code and data are public.

Who this is for

Researchers and platform teams designing agent tool surfaces, desktop benchmarks and verifiable generative UI

Original sources

Revisions & corrections (1)

— Historical backfill found outside the directory; paper, repository and dataset read, with results, limits and counterexamples preserved.

Related events

Research

A second, independent VeriCUA turns GUI-task acceptance into active evidence gathering with abstention

A Google and University of Hong Kong team submitted a separate project also named VeriCUA on October 29. It inspects an environment or recording before constructing a task rubric, then gathers missing evidence through GUI, accessibility-tree, file and database reads, marking requirements met, unmet or unknown. In online experiments it reduces false positives against baselines on AndroidWorld, OSWorld and ScienceBoard.

Google and University of Hong Kong VeriCUA arXiv paperResearch paper
Research

GUI‑JEPA replaces token-by-token action decoding with candidate-conditioned latent scoring

Submitted October 27, GUI‑JEPA targets single-step mobile GUI action selection. It transforms JEPA representations pretrained on unlabeled interface screenshots for each action and region candidate, then scores accessibility-tree candidates in one batch instead of asking a 7B vision-language model to decode an action token by token.

GUI-JEPA authors / arXiv and official repositoryResearch paper
Research

TAM turns verified task advancement into online state and reusable operations for GUI agents

Submitted October 27, Task Advancement Memory (TAM) uses what has been reliably accomplished at two timescales: it maintains verified progress, remaining goals and visual evidence during a task, then extracts reusable operations from locally successful segments after the task, including segments inside an overall failed workflow.

arXiv paper and full textResearch paper