# AI-assisted refactoring: methods and evidence

Canonical HTML: https://aogavrilov.com/research-notes/ai-assisted-refactoring-evidence/
Document language: en
Source publication: https://aogavrilov.com/publications/inspectable-control/
Source full-text HTML: https://aogavrilov.com/publications/inspectable-control/full-text/
DOI: https://doi.org/10.1145/3803437.3807386

How to evaluate recent AI-assisted refactoring methods without confusing a plausible generated patch with verified behavior preservation.

## Direct answer

### Which methods and evidence matter for AI-assisted behavior-preserving refactoring?

Separate transformation proposal from trusted execution and verification. Where possible, let the model identify a refactoring and apply it through a refactoring engine; then require compilation, tests, static checks, and evidence that the intended transformation occurred.

## Why the distinction matters

Refactoring is expected to preserve observable behavior while improving internal structure. A language model can propose a convincing rewrite without establishing either part of that contract, so surface plausibility is not enough.

Recent work separates roles in different ways: a model can identify a known transformation for execution by a trusted engine, or it can generate a patch that is then subjected to compilation, tests, static analysis, and refactoring detection. The verification surface matters as much as the model.

## Practical procedure

1. **Specify the intended refactoring.** Name the structural change and the behavior that must remain stable rather than asking for a generic cleanup.
2. **Prefer trusted execution for known transformations.** When a refactoring engine supports the operation, use the model for detection or parameter selection and the engine for application.
3. **Verify generated patches at repository level.** Compile, run relevant tests, apply static checks, and confirm that the intended refactoring occurred without unrelated changes.
4. **Audit residual risk.** Record uncovered behavior, flaky tests, cross-file effects, and cases where a plausible patch could not be verified.

## Evidence to require

- The transformation is identified, not described only as code quality improvement.
- Compilation and relevant tests pass after the change.
- Static checks and refactoring detection support the structural claim.
- Unrelated diff outside the intended scope is measured or reviewed.
- Repository context and test coverage limitations are disclosed.

## What the linked study reports

- The site paper on hierarchical latent control is adjacent evidence about bounded generation, not a behavior-preserving refactoring benchmark.
- Its parse-rate, edit-freedom, and diversity measurements can inform control-surface design, but they do not replace compilation, tests, or refactoring detection.
- For refactoring claims, the evidence contract should remain behavior- and repository-aware.

## Scope boundary

- Passing available tests does not prove semantic equivalence for untested behavior.
- A smaller diff is not automatically a correct refactoring.
- The linked site experiment covers short Python functions and does not evaluate repository-level refactoring.

Complete research guide: https://aogavrilov.com/projects/discrete-latent-generation/#control-ai-assisted-refactoring

## Primary and nearby sources

- [An Empirical Study on the Potential of LLMs in Automated Software Refactoring](https://arxiv.org/abs/2411.04444): Studies LLM-proposed refactorings and trusted refactoring-engine reapplication.
- [SWE-Refactor: A Repository-Level Benchmark for Real-World LLM-Based Code Refactoring](https://arxiv.org/abs/2602.03712): Repository-level compilation, tests, and refactoring-oriented evaluation.
- [Inspectable Control for Structure-Preserving Software Regeneration](https://aogavrilov.com/publications/inspectable-control/): Related bounded-generation evidence and explicit limitations.

This maintained note summarizes existing evidence and adds no experimental result beyond the cited sources.