
Panel
Hongcheng-Gao/EngiWorld
Engiworld: What Can Frontier Agents Deliver in Professional Engineering Environments?

Engiworld: What Can Frontier Agents Deliver in Professional Engineering Environments?
Panel is an open-source printed circuit board by Hongcheng-Gao, from the Hongcheng-Gao/EngiWorld repository on GitHub, published under the MIT licence. It is a 2-layer board of 160 × 70 mm with 30 parts on 3 nets.
What Can Frontier Agents Deliver in Professional Engineering Environments?
🌐 Website · 📄 Paper · 🧩 Tasks & Verifiers · 🤗 Task Dataset · 🤗 Environment Images
Six task types for engineering software use and artifact-based evaluation. Click to enlarge.
EngiWorld evaluates whether agents can operate professional engineering software and deliver the requested engineering artifacts. This repository provides all 1,301 tasks, their evaluators, and the GUI/CLI evaluation runtime. The paper uses a 306-task main-experiment subset, listed in task/splits/main-306.txt.
The full set of 1,301 tasks and their verifiers is also available in the Hugging Face task dataset, with a dataset viewer for browsing task instructions and metadata.
The benchmark covers CAD, CAE, CAM, BIM, EDA, and DCC. Its six task types are Single-Software Execution (931), Software Selection (140), Vision-Guided Modeling (120), Design Optimization (40), Cross-Software Coordination (60), and Open-Environment Engineering (10).
Local evaluation launches Windows or Ubuntu virtual machines through Docker on Linux, calls the model, and saves evaluation results.
Use the cloud evaluation guide for single-task or Redis-scheduled batch evaluation on either cloud. Explicit model presets, API interruption handling and batch state management are shared. Images require compatible engineering applications and desktop services.
Prepare an x86_64 Linux server with Python 3.10, Docker Engine, and zstd. With /dev/kvm, the VM uses hardware acceleration. Without it, the Docker runner attempts software emulation, which starts and runs more slowly. Each environment defaults to 4 CPUs and 16 GB RAM.
Get the code and enter the runtime directory:
git clone --branch main --single-branch --depth 1 https://github.com/Hongcheng-Gao/Engiworld.git &&
cd Engiworld/runtimeInstall the runtime in a Python venv environment:
python3.10 -m venv .venv
source .venv/bin/activate
python -m pip install -e ".[worker-api,download]"Run the following commands from runtime/.
The complete dataset provides all 1,301 task definitions, drawings, evaluators, and required ground-truth files. GitHub includes every drawing and evaluator plus smaller task resources. Download the complete data before starting evaluation.
| Source | Complete task data | Environment images |
|---|---|---|
| Hugging Face | EngiWorld-Tasks | EngiWorld-Images |
| ModelScope | EngiWorld-Tasks | EngiWorld-Images |
Use --source modelscope in the commands below to download from ModelScope.
Hugging Face:
python -m engiworld.prepare_tasks --source hf --task-root ../data/task
python -m engiworld.prepare_images --source hf --image LibreCAD2.2.0.2 --image-root ../environment-imagesModelScope:
python -m engiworld.prepare_tasks --source modelscope --task-root ../data/task
python -m engiworld.prepare_images --source modelscope --image LibreCAD2.2.0.2 --image-root ../environment-imageszstd -d ../environment-images/images/LibreCAD2.2.0.2.qcow2.zst
docker pull happysixd/osworld-docker
python -m engiworld.prepare_tasks --task-root ../data/task --checkDownloads verify checksums and preserve modified local files. ModelScope image downloads use 8 parallel range streams by default; set MODELSCOPE_DOWNLOAD_PARALLEL_WORKERS to adjust this. Omitting --image downloads all 15 public archives (about 98 GB compressed). Choose the image matching the task snapshot from manifest.json. Snapshots outside the public catalog require a matching image prepared by the user. Decompress the selected archive and prepare the Docker runner before evaluation. Task preparation uses the published dataset directly.
The following commands read already downloaded task data and the local image.
python -m engiworld.local_eval \
--task-root ../data/task \
--task single-software-execution/gui/librecad/task-08 \
--image ../environment-images/images/LibreCAD2.2.0.2.qcow2 \
--smoke-testThis starts the VM, loads the task inputs, and saves a smoke.png screenshot in the results directory without calling a model.
Servers without KVM: Check whether the server provides /dev/kvm:
test -e /dev/kvm && echo "KVM available" || echo "Software emulation"If the output says Software emulation, run the same --smoke-test command above. The runtime selects software emulation automatically; no arguments need to change. Startup may take several minutes. When the terminal shows "status": "environment_ready" and smoke.png appears in the results directory, open the screenshot to confirm that LibreCAD and the input drawing are ready. Then follow step 2 to run and score a model.
Configure an OpenAI-compatible Chat Completions endpoint and run the task:
export OPENAI_BASE_URL="https://your-api-endpoint/v1"
export OPENAI_API_KEY="your-api-key"
python -m engiworld.local_eval \
--task-root ../data/task \
--task single-software-execution/gui/librecad/task-08 \
--image ../environment-images/images/LibreCAD2.2.0.2.qcow2 \
--model your-model-name \
--result-dir resultsWhen the run finishes, summary.json summarizes the run, result.txt holds the score, and traj.jsonl records model actions. For another task, change --task and use its matching local image. Complete-data preparation already supplies its resources. See the runtime guide for more options.
task/<category>/<gui|cli>/<software or software1--software2>/<task-number>/
runtime/Task categories are single-software-execution, cross-software-coordination, software-selection,
design-optimization, vision-guided-modeling, and open-environment-engineering.
Open-Environment Engineering tasks use CLI in a blank environment and live under
task/open-environment-engineering/cli/agent-selected/; the agent chooses and installs its tools.
Task JSON IDs follow <category>--<gui|cli>--<software>--<task-number>--<os>, for example
single-software-execution--gui--librecad--task-08--ubuntu. Use either this ID or its task
directory path with --task. See the task guide.
Directory categories and JSON IDs follow the updated paper taxonomy as of 2026-10-09. task/aliases.json preserves historical JSON IDs and directory names; the runtime continues to accept them for task selection. See the full ID migration map and task taxonomy.
The main experiment evaluates seven models on the same 306 tasks: 158 CLI and 148 GUI, including all 10 Open-Environment Engineering tasks. EngiScore is the mean task score on a 0–100 scale. Execution metrics average all 306 tasks, including unsuccessful runs; output tokens are reported per turn, in thousands.
| Model | EngiScore | CLI | GUI | Turns/task | Tokens (K/turn) | Cost ($/task) |
|---|---|---|---|---|---|---|
| Claude Opus 5 | 44.7 | 48.6 | 40.5 | 69.5 | 2.4 | 18.78 |
| GPT-5.6 Sol | 38.6 | 47.9 | 28.6 | 54.2 | 1.0 | 9.57 |
| Qwen3.8 Max | 26.3 | 35.3 | 16.6 | 107.3 | 2.7 | 6.44 |
| Gemini 3.7 Flash | 26.1 | 37.2 | 14.2 | 110.0 | 0.7 | 2.08 |
| DeepSeek V4.1 Flash | 25.9 | 48.3 | 2.0 | 115.3 | 9.8 | 0.80 |
| Kimi K3 | 24.3 | 31.3 | 16.7 | 92.1 | 0.9 | 5.85 |
| Qwen3.8 Flash | 16.7 | 31.5 | 1.0 | 119.6 | 1.3 | 0.44 |
If you use EngiWorld in your research, please cite:
@misc{engiworld2026,
title = {EngiWorld: What Can Frontier Agents Deliver in Professional Engineering Environments?},
author = {Hongcheng Gao and Hailong Qu and Yu Lei and Henghui Sun and Haoyang Li and Yipeng Wei and Naihao Xue and Xiaohan Yu and Zhuo Tao and Yihe Zang and Yajiao Wang and Jingyi Tang and Yi Li and Jingjing Zhou and Jie Luo and Bohan Zeng and Chengyu Shen and Hao Jiang and Chong Chen and Bowen Qu and Olive Huang and Zeqiang Wang},
year = {2026},
eprint = {2609.37686},
archivePrefix = {arXiv},
primaryClass = {cs.AI},
url = {https://arxiv.org/abs/2609.37686}
}EngiWorld's original code is released under the MIT License. See Third-party notices for component licenses.
| Qty | Value | Footprint | References |
|---|---|---|---|
| 12 | Fiducial | Fiducial:Fiducial_1mm_Mask2mm | FID1, FID2, FID3, FID4, FID5, FID6, FID7, FID8, FID9, FID10, FID11, FID12 |
| 6 | 100nF | Capacitor_SMD:C_0402 | A1_C1, A2_C1, A3_C1, A4_C1, A5_C1, A6_C1 |
| 6 | 10k | Resistor_SMD:R_0402 | A1_R1, A2_R1, A3_R1, A4_R1, A5_R1, A6_R1 |
| 6 | LED_G | LED_SMD:LED_0402 | A1_D1, A2_D1, A3_D1, A4_D1, A5_D1, A6_D1 |

Hongcheng-Gao/EngiWorld
Engiworld: What Can Frontier Agents Deliver in Professional Engineering Environments?

Hongcheng-Gao/EngiWorld
Engiworld: What Can Frontier Agents Deliver in Professional Engineering Environments?

Hongcheng-Gao/EngiWorld
Engiworld: What Can Frontier Agents Deliver in Professional Engineering Environments?

Hongcheng-Gao/EngiWorld
Engiworld: What Can Frontier Agents Deliver in Professional Engineering Environments?

Hongcheng-Gao/EngiWorld
Engiworld: What Can Frontier Agents Deliver in Professional Engineering Environments?

Hongcheng-Gao/EngiWorld
Engiworld: What Can Frontier Agents Deliver in Professional Engineering Environments?

Hongcheng-Gao/EngiWorld
Engiworld: What Can Frontier Agents Deliver in Professional Engineering Environments?

Hongcheng-Gao/EngiWorld
Engiworld: What Can Frontier Agents Deliver in Professional Engineering Environments?

Hongcheng-Gao/EngiWorld
Engiworld: What Can Frontier Agents Deliver in Professional Engineering Environments?

Hongcheng-Gao/EngiWorld
Engiworld: What Can Frontier Agents Deliver in Professional Engineering Environments?

Hongcheng-Gao/EngiWorld
Engiworld: What Can Frontier Agents Deliver in Professional Engineering Environments?

Hongcheng-Gao/EngiWorld
Engiworld: What Can Frontier Agents Deliver in Professional Engineering Environments?

google/mozc-devices
Circuit diagrams and firmware source code for Gboard DIY keyboards


VoronDesign/Voron-Hardware
Cool Hardware designed by the Voron Design group

hoglet67/RGBtoHDMI
Bare-metal Raspberry Pi project that provides pixel-perfect sampling of Retro Computer RGB/YUV video and conversion to HDMI

tenderlove/initial-v
It's a BMW shifter converted to a Bluetooth Keyboard that you use with Vim

hackclub/OnBoard
💡 Join 1,000 teenagers and make your first PCB with a $100 grant!


It measures 160 × 70 mm and has 2 copper layers, 1.6 mm thick.
30 parts. The full list of its parts is on this page, with links to each part's symbol and footprint where PCBKit's library has them.
From the Hongcheng-Gao/EngiWorld repository on GitHub, at the commit this page was made from: the board is task/single-software-execution/cli/kicad/task-19/ground_truth/panel.kicad_pcb. PCBKit opens the board in the browser, with nothing to install.
Its repository publishes it under the MIT licence: you may reuse it on that licence's terms (keep the author's notice, and for a copyleft licence share your changes alike).