Making AI possible for real-world engineering work
Today’s frontier models build web apps in seconds, but struggle in safety-critical embedded environments
We build custom evals, data sets, and refine data for the embedded engineering domain.
Domain engineers registered
We come from the automotive industry, where we have helped the world’s largest carmakers build low-level chip integrations & implement AI in engineering processes.
Data and evaluations for frontier labs & enterprises
Evals and data built to your capability target
- Tasks written by domain experts
- Golden answers and deterministic graders
- Quality review on every task
- Delivered in your format
Ready-to-use data sets and reinforcement learning environments
- Environments with graders built in
- Areas picked from our model failure research
- Licensed and ready to train on
- Improved after delivery
Real engineering work from engineering firms
- Specifications, tickets and source code
- Design reviews and the reasons behind them
- Rebuilt and anonymised
- Work models have never seen
Engineers and agents working in real engineering software
- Screen video at 30–60 fps
- Keys, clicks and window events, on one clock
- Reasoning, spoken or from the agent
- Task files and graded outcomes
Guardrails for the AI inside your engineering toolchain
- Private evals of models in your environment
- Rules and checkers in your editor and pipeline
- Fewer mistakes in AI-written code
- Your own post-trained model
Select a product
Where real-world engineering fails
Today's frontier models are able to code almost any system, but fail on basic real-world engineering tasks, or produce plausible but fake answers.
Timing
Example: The response to an interrupt must land within 10 µs. The model’s code answers late.
Memory
Example: The build must fit 128 KB of RAM. The model’s build runs over.
Budgets
Example: Average current must stay within 2.5 mA. The model’s code keeps the chip awake between wake-ups.
ConstraintBench: A benchmark for real-world constraint-based coding
Measures how AI models handle real engineering work in chip design, RTL, CAD and embedded coding. Domain engineers write every task, and scripts grade each answer against the design’s real constraints. New models are added as they are released.
Chip design, RTL, CAD and embedded coding
Each graded by one family of checker scripts
11 models, 50 tasks, 8 runs per task. Brackets mark the 95% confidence interval.
Latest research and news
Partnering with a small number of frontier labs
Contact us for samples, data cards and pricing.