Proposal
Qwen2.5-Coder 3B mutates or crosses over selected PyTorch CNNs. Random search samples a hand-designed architecture space.
A local Qwen2.5-Coder 3B proposed PyTorch architectures; random sampling was the baseline. With 30 proxy-training attempts per strategy, the best random finalist scored 92.72% on the test set versus 89.34% for the best LLM finalist.
One run per strategy · CIFAR-10 · 20-epoch finalist training · not evidence that random search always wins.
Left: best proxy validation accuracy after each of 30 proxy-training attempts per strategy. The random run has 2 timed-out attempts, shown as flat steps. Right: the ten report-listed top-five candidates, plotted against parameter count. Neither plot shows test accuracy; finalist test scores appear below.
runs/llm_qwen3b/archive.jsonl, runs/random/archive.jsonl and analysis/RESULTS.md. Hand-designed LLM seeds are included in the trajectory.Download 2400 × 1350 PNG ↗+3.38 percentage points for the random finalist. Search used validation accuracy. Test accuracy was evaluated once after training for each of the four finalists. These are different-sized models; the result does not isolate search strategy from model size.
All ten top-five candidates named in the original report. Sort the table to compare proxy validation accuracy and size; only four finalists have reported test results. Proxy and test accuracy are different evaluations.
| Origin / idea |
|---|
The left plot shows all 60 proxy-training attempts, but the table and right-hand plot show only the ten published top-five entries. Source: analysis/RESULTS.md ↗. Original lower-resolution report plot ↗
The search changes only model architecture, not the training recipe. The LLM runs on CPU while the laptop GPU trains candidates.
Qwen2.5-Coder 3B mutates or crosses over selected PyTorch CNNs. Random search samples a hand-designed architecture space.
Static checks and a CPU dry run enforce shape and compute limits (≤1M parameters and ≤600 MFLOPs). The AST checks are not a security boundary.
A 5-epoch proxy trains on 10k images and scores on held-out validation data. The finalists retrain for 20 epochs on 45k images before the test evaluation.
The LLM's median non-seed proxy candidate reached 59.78% against 56.75% for random; the best proxy candidates reached 66.68% and 69.08%, respectively. Each strategy had 30 proxy-training attempts: 30 completed for the LLM and 28 for random search. Proposal counts exclude the 3 LLM seeds (46 LLM, 33 random). The two proposal spaces differ, and wall-clock time differed.
The LLM repeatedly proposed depthwise-separable convolutions close to its seeds; its run logged 9 duplicates and 8 errors. Random explored larger inverted-residual networks. A noisy proxy, unequal model sizes and no repeated seeds limit any broader conclusion. Larger LLMs and ablations remain future work, not reported experiments.
See the source code, training recipe, published figures and report in the public repository. The run archives and checkpoints are not public. This page is a static account, not a live training demo.
Browse the source ↗Setup and limitations ↗