# Scry Research

> Benchmarks and cost/performance results for vision-language models and agents doing mobile UI and Figma-to-Storybook diff review.

Scry Design Diff Eval is a 311-pair benchmark of human-reviewed mobile UI defects.
Each pair is a reference screenshot and a generated implementation screenshot with
explicit defect tags and normalized selection boxes. The primary metric is
known-issue recall under strict tag-and-box matching.

## Posts

- [Dark Factory Tiers: How Strong a Model Set Does Each Job Need?](https://blog.scrymore.com/blog/dark-factory-tiers/) (October 19, 2026): We graded 134 shipped features and fixes by the weakest model set that could finish them in an autonomous run. Six fit a Haiku-only build. 87 run longer than any measured model can finish reliably in one attempt, and 12 are beyond one autonomous run today.
- [Dark Factory Patterns: A Developer's Guide](https://blog.scrymore.com/blog/dark-factory-patterns/) (October 19, 2026): A field guide to shipping software with the lights off: 25 patterns in five families, from one human message to a retro that has to pass a check. Each pattern comes with our own run evidence and published work by others.
- [Prodia speed from FLUX.3 on our own GPUs: a generate-instead-of-search demo](https://blog.scrymore.com/blog/prodia-speed-flux3-own-gpus/) (October 17, 2026): We built a Google Images where every picture is generated, ran it on hosted Prodia and on self-hosted FLUX.3 klein, and measured how close our own RTX 5090s got. On 8 RTX 5090s with 384 px tiles we are ahead on the full screen and close on the first picture.
- [10–100× Cheaper Design Diffs: A Judged 60-Screen Scorecard](https://blog.scrymore.com/blog/cheaper-design-diffs-cascade/) (September 25, 2026): On 60 Figma-to-Storybook screens with judged truth, a deterministic pre-filter plus one cheap vision call matches Claude Opus 5.5 xhigh's recall of human-tagged defects for $0.0045 a screen instead of $0.54. Scry's Basic and Plus tiers, built on it, find 89% and 94% for $0.009 and $0.05.
- [Link Figma to any Storybook, then see where they drift](https://blog.scrymore.com/blog/link-figma-to-any-storybook/) (September 25, 2026): Scry Link connects Figma layers to stories in any hosted Storybook, auto-matches a whole page, and shows where the design and the implementation have drifted apart.
- [Component Requests: From a Figma Component to a GitHub Issue in One Click](https://blog.scrymore.com/blog/component-requests-figma-to-github-issue/) (September 25, 2026): Select a component in Figma that has no code yet, press Request this component, and Scry opens a GitHub issue with a preview, the variants and your notes. The Figma node remembers it.
- [The Cost Floor Moved: Five More Figma-to-Storybook Diff Agents](https://blog.scrymore.com/blog/cheaper-diff-agents-five-more-models/) (August 28, 2026): Sixteen model configurations on the same Figma-to-Storybook diff. The cheapest run that still finds the structural problems now costs under a cent.
- [Beyond Screenshots: AI Agents for Figma-to-Storybook Diffs](https://blog.scrymore.com/blog/beyond-screenshots-figma-storybook-diff-agents/) (July 20, 2026): What changes when a UI review agent gets the Figma layer tree, the live DOM with computed styles and the implementation source instead of just two screenshots.
- [Can VLMs Review Mobile UI? Introducing Scry Design Diff Eval](https://blog.scrymore.com/blog/introducing-scry-design-diff-eval/) (July 3, 2026): Scry Design Diff Eval is a 311-pair benchmark of human-reviewed mobile UI defects. The best vision-language models recover roughly two fifths of known issues.

## Research

- [Scry Design Diff Eval benchmark paper](https://blog.scrymore.com/paper/): full methodology, dataset composition, scoring protocol and results for 311 pairs and 557 tagged issues.
- [Design diff leaderboard](https://blog.scrymore.com/leaderboard/): standing leaderboard of Figma-to-Storybook diff detectors: human-tag recall, all-real recall, precision and cost per screen on 60 screens against judged truth (single models, cascades and the Scry Basic/Plus tiers) and the 311-pair screenshot benchmark scores.
- [Pilot dashboard](https://blog.scrymore.com/pilot/): interactive browser over the 60-pair pilot subset.

## Data

- [Dataset on Hugging Face](https://huggingface.co/datasets/Scrymore/scry-design-diff-eval): Scrymore/scry-design-diff-eval, CC-BY-4.0, 311 rows.

## Optional

- [Full text of every post](https://blog.scrymore.com/llms-full.txt)
- Markdown source for any post: append .md to its slug, e.g. https://blog.scrymore.com/blog/dark-factory-tiers.md