# Predictable code generation needs a preservation contract

Canonical HTML: https://aogavrilov.com/research-notes/predictable-code-generation-preservation-contract/
Document language: en
Source publication: https://aogavrilov.com/publications/inspectable-control/
Source full-text HTML: https://aogavrilov.com/publications/inspectable-control/full-text/
DOI: https://doi.org/10.1145/3803437.3807386

Why deterministic sampling is not enough, and how observable protected properties and acceptance checks make code-generation behavior testable.

## Direct answer

### What makes code generation predictable rather than merely controllable?

Predictability requires an observable contract stated before generation: what may change, what must remain stable, how each property will be measured, and when an output will be rejected. A fixed prompt, seed, mask, grammar, or latent code is only a mechanism.

## Why the distinction matters

Deterministic generation repeats an output under fixed conditions. Predictable generation makes a declared property testable: for example, an API signature remains unchanged, edits stay inside a region, syntax remains valid, or a test suite continues to pass.

The distinction matters because a control input does not define its effect. Locking a latent position, fixing a seed, or constraining a grammar can make generation easier to inspect, but the protected property must still be measured after decoding.

## Practical procedure

1. **Declare protected properties.** State the source regions, structures, interfaces, behaviors, or distributions that must remain within a defined tolerance.
2. **Choose a matching control mechanism.** Use source reuse, edit masks, formal constraints, latent locking, or proposal-plus-validation according to the property being protected.
3. **Define acceptance before sampling.** Specify parsing, compilation, tests, structural checks, locality thresholds, and task-success criteria before inspecting outputs.
4. **Measure repeated behavior.** Report acceptance, rejection, stability, diversity, and uncertainty across repeated runs instead of relying on one illustrative result.

## Evidence to require

- Protected and editable properties are stated separately.
- Every claimed guarantee has an observable test or metric.
- Task success is evaluated alongside stability.
- Repeated-run variability and rejection rates are visible.
- Unmeasured properties are explicitly excluded from the claim.

## What the linked study reports

- The linked experiment exposes hierarchical discrete positions as one inspectable control surface for short Python functions.
- Locking four top-level positions improved parse rate while leaving high change in unlocked positions and high sample uniqueness.
- The paper reports this as early probabilistic evidence; it does not claim exact AST preservation, semantic equivalence, functional correctness, or repository-scale predictability.

## Scope boundary

- Predictability is property-specific: a system can be predictable about syntax and unpredictable about behavior.
- Deterministic decoding can repeat the same wrong or over-broad edit.
- The available site experiment does not establish guarantees beyond its measured short-function setting.

Complete research guide: https://aogavrilov.com/projects/discrete-latent-generation/#core-idea

## Primary and nearby sources

- [Inspectable Control for Structure-Preserving Software Regeneration](https://aogavrilov.com/publications/inspectable-control/): Primary bounded experiment and stated evidence limits.
- [EfficientEdit: Accelerating Code Editing via Edit-Oriented Speculative Decoding](https://arxiv.org/abs/2506.02780): An edit-oriented mechanism that reuses unchanged source segments.
- [Constrained Decoding of Diffusion LLMs with Context-Free Grammars](https://arxiv.org/abs/2508.10111): An example of a mechanism with an explicit formal constraint.

This maintained note summarizes existing evidence and adds no experimental result beyond the cited sources.