MILES: Modular Instruction Memory with Learnable
Selection for Self-Improving LLM Reasoning

Abstract

Large language models increasingly improve their reasoning at test time via additional computation, yet most procedures treat each problem in isolation. When problems arrive sequentially, accumulating reusable experience across them can further improve performance. Existing memory-based methods either store whole-solution templates that generalize poorly to novel problems or use heuristic step-level selection that is not optimized for final-answer correctness.

We propose MILES (Modular Instruction Memory with LEarnable Selection for self-improving LLM reasoning), a framework that dynamically expands step-wise memory and applies correctness-optimized memory composition under realistic test-time constraints. MILES maintains modular memory units consisting of asymmetric pairs of sub-goal embeddings and sub-instructions, each associated with a learnable selection head.

This memory structure enables a coarse-to-fine retrieval mechanism: the coarse level enables memory expansion and collects supervision for training selection heads from confident samples, while the fine stage applies learned selection heads to rerank coarse-level candidates and guide reasoning for uncertain samples. Experiments across six reasoning benchmarks demonstrate the effectiveness, robustness, and transferability of MILES.

Why MILES?

Existing test-time memory methods often retrieve whole solutions or rely on heuristic step-level selection. MILES turns memory reuse into learnable, step-wise composition.

Method Overview

A short animation illustrating how MILES incrementally constructs memory and applies learned memory selection during reasoning.

Method

MILES organizes reusable reasoning experience as modular instruction memory and applies a two-layer selection mechanism to guide uncertain reasoning cases.

Main Results

MILES consistently matches or outperforms prior methods across six reasoning benchmarks and four frozen backbones.

Final-answer accuracy (%) on six reasoning benchmarks. Per-row best results are bolded.

Accuracy–Efficiency Tradeoff

Place accuracy-token figure here:
assets/accuracy_tokens.png

MILES achieves a strong accuracy-token tradeoff compared with Self-Consistency, Tree-of-Thoughts, rStar, and DORA.

MILES dominates the frontier across evaluated token budgets while baselines flatten at higher budgets.

Transferability and Reuse

MILES memory can be reused across models and can also be pre-built from separate labeled training sets, showing that the learned memory captures transferable reasoning knowledge.

Ablation and Selection Quality

Table 4 verifies the contribution of the two-layer memory selection mechanism, while Figure 3 further analyzes the relation between sub-instruction correctness and final-answer accuracy.

Table 4. Ablation Study

Incorporating the learned Layer 2 selection head consistently improves performance over retrieval-only or alternative reranking strategies.

Dataset	Exp.	Layer 1	Layer 2	Accuracy
AIME 2024	1	✗	✗	60.00
	2	✓	✗	63.33
	3	✗	✓	63.33
	4	✓	✓	66.67
	5	✓	State sim.	63.33
	6	✓	LLM rerank	60.00
AIME 2025	1	✗	✗	53.33
	2	✓	✗	53.33
	3	✗	✓	56.67
	4	✓	✓	60.00
	5	✓	State sim.	56.67
	6	✓	LLM rerank	53.33

Layer 1 retrieval and Layer 2 learned selection jointly provide the best performance.

Figure 3. Selection Quality Analysis

Higher sub-instruction correctness aligns with improved final-answer performance on uncertain samples.

Place Figure 3 here:
assets/selection_quality_v1.png

Citation

@misc{tong2026milesmodularinstructionmemory,
  title={MILES: Modular Instruction Memory with Learnable Selection for Self-Improving LLM Reasoning},
  author={Ruilin Tong and Dong Gong},
  year={2026},
  eprint={2607.06974},
  archivePrefix={arXiv},
  primaryClass={cs.CL},
  url={https://arxiv.org/abs/2607.06974},
}