First author · manuscript in preparation (2026)
Benchmark Gains Without Reasoning Gains: Filtering, Labelling and Weighting Verified Traces for Distillation
Abstract: We compare methods of leveraging a dataset of chain-of-thought (CoT) traces, where an LLM is prompted to come up with novel and/or groundbreaking heuristics for NP-hard problems. Each of the traces was prompted to end with an executable algorithm that is scored by performance and checked for feasibility by a deterministic verifier. We try three ways of distilling knowledge: gate, grade, and weight. The gate keeps only CoTs that lead to valid algorithms, the grade fine-tunes on all the traces but assigns a quality label to each, and the weight scales the training loss of each trace based on the performance of the algorithm.
We find that the gate is the most promising statistically significant recipe among those tested, scoring a 3.0pp accuracy gain on GSM8K against the unlabelled pool and beating the base by 8.0pp. However, when we investigate further, we find that benchmark improvements over the base come from an increased answer emission rate. When comparing the models' performance on answered-only tests, we find that the base beats all fine-tuned models and beats the gate by 8.6pp. This is a signal that problem-specific fine-tuning of already heavily post-trained 4B–8B-parameter models may get an enhanced emission rate at most.
-
teacher
Kimi-Dev-72B
island-evolutionary search over 4 NP-hard families
-
deterministic verifier
13,621 (reasoning, code) traces
score on 11,731 of them
pooled dataset
-
-
gate
filter
(strict / top-tertile)
-
grade
quality prefix
(binary / 4-class / decile / continuous)
-
weight
score-shaped loss
(AWR/
CRR-like)
-
-
DoRA students
Qwen3-8B
Qwen3-4B
Gemma-3-4Bcontrols: base, generic-SFT, unlabelled pool
-
truncation-honest eval
in-domain, 3 unseen families