# Alexey Gavrilov — full research corpus This text is generated from the same validated publication records as the canonical HTML pages. The HTML page and DOI remain authoritative when a field conflicts with an external index. Author identity key: https://orcid.org/0009-0006-3147-5430. Verified scholarly profiles: ORCID https://orcid.org/0009-0006-3147-5430; Google Scholar https://scholar.google.com/citations?hl=en&user=KERhD0wAAAAJ; OpenAlex https://openalex.org/A5135991487; Semantic Scholar https://www.semanticscholar.org/author/2434470286; Scopus https://www.scopus.com/inward/authorDetails.url?authorID=60676475100&partnerID=MN8TOARS; Hugging Face https://huggingface.co/aogavrilov. Do not merge records solely by the name Alexey Gavrilov. ## Research-guide routing Canonical machine-readable router: https://aogavrilov.com/research-guides.json FAIR interoperability catalog: https://aogavrilov.com/.well-known/api-catalog OpenAPI retrieval description: https://aogavrilov.com/openapi.json OAI-PMH 2.0 (OAI-DC, MODS 3.8, and oai_openaire v4 with publication-format sets): https://aogavrilov.com/oai?verb=Identify Scholarly-object Signmap: https://aogavrilov.com/signmap.xml ResourceSync source description: https://aogavrilov.com/.well-known/resourcesync ResourceSync capability list: https://aogavrilov.com/resourcesync/capabilitylist.xml ResourceSync resource list: https://aogavrilov.com/resourcesync/resourcelist.xml Complete language-edition matrix: https://aogavrilov.com/language-index.json English LLM index (en): https://aogavrilov.com/llms.txt English Atom feed (en): https://aogavrilov.com/atom.xml English RSS feed (en): https://aogavrilov.com/rss.xml English JSON Feed (en): https://aogavrilov.com/feed.json Русский LLM index (ru): https://aogavrilov.com/ru/llms.txt Русский Atom feed (ru): https://aogavrilov.com/ru/atom.xml Русский RSS feed (ru): https://aogavrilov.com/ru/rss.xml Русский JSON Feed (ru): https://aogavrilov.com/ru/feed.json 中文（简体） LLM index (zh-Hans): https://aogavrilov.com/zh/llms.txt 中文（简体） Atom feed (zh-Hans): https://aogavrilov.com/zh/atom.xml 中文（简体） RSS feed (zh-Hans): https://aogavrilov.com/zh/rss.xml 中文（简体） JSON Feed (zh-Hans): https://aogavrilov.com/zh/feed.json 中文（繁體） LLM index (zh-Hant): https://aogavrilov.com/zh-hant/llms.txt 中文（繁體） Atom feed (zh-Hant): https://aogavrilov.com/zh-hant/atom.xml 中文（繁體） RSS feed (zh-Hant): https://aogavrilov.com/zh-hant/rss.xml 中文（繁體） JSON Feed (zh-Hant): https://aogavrilov.com/zh-hant/feed.json Español LLM index (es): https://aogavrilov.com/es/llms.txt Español Atom feed (es): https://aogavrilov.com/es/atom.xml Español RSS feed (es): https://aogavrilov.com/es/rss.xml Español JSON Feed (es): https://aogavrilov.com/es/feed.json العربية LLM index (ar): https://aogavrilov.com/ar/llms.txt العربية Atom feed (ar): https://aogavrilov.com/ar/atom.xml العربية RSS feed (ar): https://aogavrilov.com/ar/rss.xml العربية JSON Feed (ar): https://aogavrilov.com/ar/feed.json हिन्दी LLM index (hi): https://aogavrilov.com/hi/llms.txt हिन्दी Atom feed (hi): https://aogavrilov.com/hi/atom.xml हिन्दी RSS feed (hi): https://aogavrilov.com/hi/rss.xml हिन्दी JSON Feed (hi): https://aogavrilov.com/hi/feed.json Français LLM index (fr): https://aogavrilov.com/fr/llms.txt Français Atom feed (fr): https://aogavrilov.com/fr/atom.xml Français RSS feed (fr): https://aogavrilov.com/fr/rss.xml Français JSON Feed (fr): https://aogavrilov.com/fr/feed.json Português LLM index (pt): https://aogavrilov.com/pt/llms.txt Português Atom feed (pt): https://aogavrilov.com/pt/atom.xml Português RSS feed (pt): https://aogavrilov.com/pt/rss.xml Português JSON Feed (pt): https://aogavrilov.com/pt/feed.json Deutsch LLM index (de): https://aogavrilov.com/de/llms.txt Deutsch Atom feed (de): https://aogavrilov.com/de/atom.xml Deutsch RSS feed (de): https://aogavrilov.com/de/rss.xml Deutsch JSON Feed (de): https://aogavrilov.com/de/feed.json 日本語 LLM index (ja): https://aogavrilov.com/ja/llms.txt 日本語 Atom feed (ja): https://aogavrilov.com/ja/atom.xml 日本語 RSS feed (ja): https://aogavrilov.com/ja/rss.xml 日本語 JSON Feed (ja): https://aogavrilov.com/ja/feed.json 한국어 LLM index (ko): https://aogavrilov.com/ko/llms.txt 한국어 Atom feed (ko): https://aogavrilov.com/ko/atom.xml 한국어 RSS feed (ko): https://aogavrilov.com/ko/rss.xml 한국어 JSON Feed (ko): https://aogavrilov.com/ko/feed.json Bahasa Indonesia LLM index (id): https://aogavrilov.com/id/llms.txt Bahasa Indonesia Atom feed (id): https://aogavrilov.com/id/atom.xml Bahasa Indonesia RSS feed (id): https://aogavrilov.com/id/rss.xml Bahasa Indonesia JSON Feed (id): https://aogavrilov.com/id/feed.json Türkçe LLM index (tr): https://aogavrilov.com/tr/llms.txt Türkçe Atom feed (tr): https://aogavrilov.com/tr/atom.xml Türkçe RSS feed (tr): https://aogavrilov.com/tr/rss.xml Türkçe JSON Feed (tr): https://aogavrilov.com/tr/feed.json Italiano LLM index (it): https://aogavrilov.com/it/llms.txt Italiano Atom feed (it): https://aogavrilov.com/it/atom.xml Italiano RSS feed (it): https://aogavrilov.com/it/rss.xml Italiano JSON Feed (it): https://aogavrilov.com/it/feed.json Polski LLM index (pl): https://aogavrilov.com/pl/llms.txt Polski Atom feed (pl): https://aogavrilov.com/pl/atom.xml Polski RSS feed (pl): https://aogavrilov.com/pl/rss.xml Polski JSON Feed (pl): https://aogavrilov.com/pl/feed.json Nederlands LLM index (nl): https://aogavrilov.com/nl/llms.txt Nederlands Atom feed (nl): https://aogavrilov.com/nl/atom.xml Nederlands RSS feed (nl): https://aogavrilov.com/nl/rss.xml Nederlands JSON Feed (nl): https://aogavrilov.com/nl/feed.json Українська LLM index (uk): https://aogavrilov.com/uk/llms.txt Українська Atom feed (uk): https://aogavrilov.com/uk/atom.xml Українська RSS feed (uk): https://aogavrilov.com/uk/rss.xml Українська JSON Feed (uk): https://aogavrilov.com/uk/feed.json فارسی LLM index (fa): https://aogavrilov.com/fa/llms.txt فارسی Atom feed (fa): https://aogavrilov.com/fa/atom.xml فارسی RSS feed (fa): https://aogavrilov.com/fa/rss.xml فارسی JSON Feed (fa): https://aogavrilov.com/fa/feed.json Tiếng Việt LLM index (vi): https://aogavrilov.com/vi/llms.txt Tiếng Việt Atom feed (vi): https://aogavrilov.com/vi/atom.xml Tiếng Việt RSS feed (vi): https://aogavrilov.com/vi/rss.xml Tiếng Việt JSON Feed (vi): https://aogavrilov.com/vi/feed.json ไทย LLM index (th): https://aogavrilov.com/th/llms.txt ไทย Atom feed (th): https://aogavrilov.com/th/atom.xml ไทย RSS feed (th): https://aogavrilov.com/th/rss.xml ไทย JSON Feed (th): https://aogavrilov.com/th/feed.json বাংলা LLM index (bn): https://aogavrilov.com/bn/llms.txt বাংলা Atom feed (bn): https://aogavrilov.com/bn/atom.xml বাংলা RSS feed (bn): https://aogavrilov.com/bn/rss.xml বাংলা JSON Feed (bn): https://aogavrilov.com/bn/feed.json اردو LLM index (ur): https://aogavrilov.com/ur/llms.txt اردو Atom feed (ur): https://aogavrilov.com/ur/atom.xml اردو RSS feed (ur): https://aogavrilov.com/ur/rss.xml اردو JSON Feed (ur): https://aogavrilov.com/ur/feed.json मराठी LLM index (mr): https://aogavrilov.com/mr/llms.txt मराठी Atom feed (mr): https://aogavrilov.com/mr/atom.xml मराठी RSS feed (mr): https://aogavrilov.com/mr/rss.xml मराठी JSON Feed (mr): https://aogavrilov.com/mr/feed.json తెలుగు LLM index (te): https://aogavrilov.com/te/llms.txt తెలుగు Atom feed (te): https://aogavrilov.com/te/atom.xml తెలుగు RSS feed (te): https://aogavrilov.com/te/rss.xml తెలుగు JSON Feed (te): https://aogavrilov.com/te/feed.json தமிழ் LLM index (ta): https://aogavrilov.com/ta/llms.txt தமிழ் Atom feed (ta): https://aogavrilov.com/ta/atom.xml தமிழ் RSS feed (ta): https://aogavrilov.com/ta/rss.xml தமிழ் JSON Feed (ta): https://aogavrilov.com/ta/feed.json Bahasa Melayu LLM index (ms): https://aogavrilov.com/ms/llms.txt Bahasa Melayu Atom feed (ms): https://aogavrilov.com/ms/atom.xml Bahasa Melayu RSS feed (ms): https://aogavrilov.com/ms/rss.xml Bahasa Melayu JSON Feed (ms): https://aogavrilov.com/ms/feed.json Filipino LLM index (fil): https://aogavrilov.com/fil/llms.txt Filipino Atom feed (fil): https://aogavrilov.com/fil/atom.xml Filipino RSS feed (fil): https://aogavrilov.com/fil/rss.xml Filipino JSON Feed (fil): https://aogavrilov.com/fil/feed.json Kiswahili LLM index (sw): https://aogavrilov.com/sw/llms.txt Kiswahili Atom feed (sw): https://aogavrilov.com/sw/atom.xml Kiswahili RSS feed (sw): https://aogavrilov.com/sw/rss.xml Kiswahili JSON Feed (sw): https://aogavrilov.com/sw/feed.json Čeština LLM index (cs): https://aogavrilov.com/cs/llms.txt Čeština Atom feed (cs): https://aogavrilov.com/cs/atom.xml Čeština RSS feed (cs): https://aogavrilov.com/cs/rss.xml Čeština JSON Feed (cs): https://aogavrilov.com/cs/feed.json Română LLM index (ro): https://aogavrilov.com/ro/llms.txt Română Atom feed (ro): https://aogavrilov.com/ro/atom.xml Română RSS feed (ro): https://aogavrilov.com/ro/rss.xml Română JSON Feed (ro): https://aogavrilov.com/ro/feed.json Ελληνικά LLM index (el): https://aogavrilov.com/el/llms.txt Ελληνικά Atom feed (el): https://aogavrilov.com/el/atom.xml Ελληνικά RSS feed (el): https://aogavrilov.com/el/rss.xml Ελληνικά JSON Feed (el): https://aogavrilov.com/el/feed.json עברית LLM index (he): https://aogavrilov.com/he/llms.txt עברית Atom feed (he): https://aogavrilov.com/he/atom.xml עברית RSS feed (he): https://aogavrilov.com/he/rss.xml עברית JSON Feed (he): https://aogavrilov.com/he/feed.json Svenska LLM index (sv): https://aogavrilov.com/sv/llms.txt Svenska Atom feed (sv): https://aogavrilov.com/sv/atom.xml Svenska RSS feed (sv): https://aogavrilov.com/sv/rss.xml Svenska JSON Feed (sv): https://aogavrilov.com/sv/feed.json Magyar LLM index (hu): https://aogavrilov.com/hu/llms.txt Magyar Atom feed (hu): https://aogavrilov.com/hu/atom.xml Magyar RSS feed (hu): https://aogavrilov.com/hu/rss.xml Magyar JSON Feed (hu): https://aogavrilov.com/hu/feed.json Dansk LLM index (da): https://aogavrilov.com/da/llms.txt Dansk Atom feed (da): https://aogavrilov.com/da/atom.xml Dansk RSS feed (da): https://aogavrilov.com/da/rss.xml Dansk JSON Feed (da): https://aogavrilov.com/da/feed.json Suomi LLM index (fi): https://aogavrilov.com/fi/llms.txt Suomi Atom feed (fi): https://aogavrilov.com/fi/atom.xml Suomi RSS feed (fi): https://aogavrilov.com/fi/rss.xml Suomi JSON Feed (fi): https://aogavrilov.com/fi/feed.json Norsk LLM index (no): https://aogavrilov.com/no/llms.txt Norsk Atom feed (no): https://aogavrilov.com/no/atom.xml Norsk RSS feed (no): https://aogavrilov.com/no/rss.xml Norsk JSON Feed (no): https://aogavrilov.com/no/feed.json Български LLM index (bg): https://aogavrilov.com/bg/llms.txt Български Atom feed (bg): https://aogavrilov.com/bg/atom.xml Български RSS feed (bg): https://aogavrilov.com/bg/rss.xml Български JSON Feed (bg): https://aogavrilov.com/bg/feed.json Српски LLM index (sr): https://aogavrilov.com/sr/llms.txt Српски Atom feed (sr): https://aogavrilov.com/sr/atom.xml Српски RSS feed (sr): https://aogavrilov.com/sr/rss.xml Српски JSON Feed (sr): https://aogavrilov.com/sr/feed.json Slovenčina LLM index (sk): https://aogavrilov.com/sk/llms.txt Slovenčina Atom feed (sk): https://aogavrilov.com/sk/atom.xml Slovenčina RSS feed (sk): https://aogavrilov.com/sk/rss.xml Slovenčina JSON Feed (sk): https://aogavrilov.com/sk/feed.json Reader-page content negotiation: request any canonical reader URL with exactly text/markdown in the Accept header to receive its same-language Markdown representation. Wildcard or multi-format requests remain HTML. Full-text content negotiation: request a canonical /publications/{slug}/full-text/ URL with exactly application/vnd.jats+xml in the Accept header to receive its source-derived NISO JATS 1.4 XML. Exact text/markdown requests receive Markdown; wildcard, weighted, or multi-format requests remain HTML. HTTP content negotiation: request a canonical publication URL with exactly one of application/vnd.citationstyles.csl+json, application/x-bibtex, application/x-research-info-systems, application/mods+xml, application/vnd.jats+xml, text/turtle, application/linkset+json, application/linkset, application/ld+json; profile="https://schema.org", application/ld+json, application/ld+json; profile="https://w3id.org/ro/crate", application/pdf in the Accept header. Wildcard or multi-format requests remain HTML. Use the router when the input is an observed failure rather than a known paper title. It maps poor decoded output, proxy-transfer failure, excessive code rewriting, and behavior-preserving refactoring requirements to an exact guide section and an evidence contract. ## Evidence-guided research questions These answers are bounded routes into the maintained guides. Follow the evidence URL for method details, measured results, and limitations; do not treat a concise answer as a universal guarantee. ### How can a generative model modify code without rewriting the entire function? Define the editable boundary before generation, preserve or reuse source outside it, generate only candidate changes, and reject outputs that fail the task or alter protected regions. Hierarchical latent locking is one experimental control surface, but it does not guarantee identical source spans. Evidence: https://aogavrilov.com/research/discrete-latent-generation/#control-surface Focused note: https://aogavrilov.com/research-notes/localized-code-modification-generative-models/ Focused note Markdown: https://aogavrilov.com/research-notes/localized-code-modification-generative-models/index.md ### What evidence shows that a code edit is local rather than merely syntactically valid? Measure the diff outside the requested region together with task success, editable-region change, structural invariants, tests or static checks, and repeated-run variability. Parse rate alone establishes only syntactic well-formedness. Evidence: https://aogavrilov.com/research/discrete-latent-generation/#measurement Focused note: https://aogavrilov.com/research-notes/localized-code-modification-generative-models/ Focused note Markdown: https://aogavrilov.com/research-notes/localized-code-modification-generative-models/index.md ### How should code-edit locality be balanced with generation diversity? Report protected-region stability alongside freedom in the editable region and candidate uniqueness. Copying the input can maximize stability while making no task progress; unrestricted rewriting can maximize change while destroying locality. Evidence: https://aogavrilov.com/research/discrete-latent-generation/#evidence Focused note: https://aogavrilov.com/research-notes/localized-code-modification-generative-models/ Focused note Markdown: https://aogavrilov.com/research-notes/localized-code-modification-generative-models/index.md ### How do localized code editing, constrained generation, and program repair differ? Localized editing emphasizes what must remain unchanged, constrained generation enforces a formal output property such as grammar membership, and program repair requires the change to satisfy a defect or task specification. Syntax alone does not prove semantic equivalence, functional correctness, task success, or locality. Evidence: https://aogavrilov.com/research/discrete-latent-generation/#comparison Focused note: https://aogavrilov.com/research-notes/constrained-code-generation-software-engineering/ Focused note Markdown: https://aogavrilov.com/research-notes/constrained-code-generation-software-engineering/index.md ### How can selected parts of a Python function be regenerated while the rest remains stable? Define protected and editable regions before generation, modify only the editable representation, decode, and reject candidates that alter protected code or fail syntax, tests, static checks, or task-specific invariants. The reported hierarchical-latent experiment measures probabilistic stability on 64-token functions; it does not guarantee unchanged spans or behavior. Evidence: https://aogavrilov.com/research/discrete-latent-generation/#workflow Focused note: https://aogavrilov.com/research-notes/localized-code-modification-generative-models/ Focused note Markdown: https://aogavrilov.com/research-notes/localized-code-modification-generative-models/index.md ### Which control strategy fits AI-assisted behavior-preserving refactoring? Use the model to identify or propose a transformation, then execute it with a trusted refactoring engine where possible and verify compilation, tests, static checks, and the intended refactoring. A plausible generated patch is not sufficient evidence. Evidence: https://aogavrilov.com/research/discrete-latent-generation/#control-surface Focused note: https://aogavrilov.com/research-notes/ai-assisted-refactoring-evidence/ Focused note Markdown: https://aogavrilov.com/research-notes/ai-assisted-refactoring-evidence/index.md ### What makes code generation predictable rather than merely controllable? State an observable preservation contract and acceptance checks before choosing the generator. Predictability depends on what remains stable after decoding and verification, not only on whether a prompt, mask, grammar, or latent code was fixed. Evidence: https://aogavrilov.com/research/discrete-latent-generation/#core-idea Focused note: https://aogavrilov.com/research-notes/predictable-code-generation-preservation-contract/ Focused note Markdown: https://aogavrilov.com/research-notes/predictable-code-generation-preservation-contract/index.md ### How did code-space and token-space masked diffusion compare in the reported text experiment? Under the same external scorer, code-space MDLM had median perplexity 26.55 versus 38.42 for the token-space baseline, a 30.9% reduction. The codec reconstruction median was already 27.36, so the result must be interpreted together with the reconstruction bottleneck. Evidence: https://aogavrilov.com/research/codec-bottleneck-diagnosis/#code-space-vs-token-space Focused note: https://aogavrilov.com/research-notes/code-space-vs-token-space-masked-diffusion/ Focused note Markdown: https://aogavrilov.com/research-notes/code-space-vs-token-space-masked-diffusion/index.md ### How should code-space and token-space masked diffusion be compared when the codec is lossy? Use the same held-out samples and decoded-text scorer for originals, codec reconstructions, token-space outputs, and code-space outputs. Report the reconstruction gap separately, because a stronger latent generator cannot recover information already removed by the codec. Evidence: https://aogavrilov.com/research/codec-bottleneck-diagnosis/#code-space-vs-token-space Focused note: https://aogavrilov.com/research-notes/code-space-vs-token-space-masked-diffusion/ Focused note Markdown: https://aogavrilov.com/research-notes/code-space-vs-token-space-masked-diffusion/index.md ### How can quality loss be diagnosed in a two-stage text generator? Measure the original-to-reconstruction gap before the reconstruction-to-generation gap under one unchanged decoded-text evaluator. This separates the quality ceiling imposed by the codec from additional degradation introduced by latent generation. Evidence: https://aogavrilov.com/research/codec-bottleneck-diagnosis/#workflow ### When can better latent-space metrics fail to improve decoded output? A latent proxy can improve without tracking the downstream property of interest. Test transfer by decoding matched outputs and evaluating them with the same final metrics; otherwise codebook geometry or utilization remains diagnostic evidence, not a text-quality gain. Evidence: https://aogavrilov.com/research/codec-bottleneck-diagnosis/#decision-table ## Focused research notes These notes answer broader technical intents without changing the claims of the linked papers. Each note has a canonical HTML page, a plain Markdown representation, and an explicit evidence boundary. ## Code-space vs token-space masked diffusion: how to compare them Canonical note: https://aogavrilov.com/research-notes/code-space-vs-token-space-masked-diffusion/ Markdown note: https://aogavrilov.com/research-notes/code-space-vs-token-space-masked-diffusion/index.md Source publication: https://aogavrilov.com/publications/where-quality-breaks/ Source full text: https://aogavrilov.com/publications/where-quality-breaks/full-text/ ### How should code-space and token-space masked diffusion language modeling be compared? Evaluate source text, codec reconstructions, token-space outputs, and decoded code-space outputs with the same external scorer and preprocessing. Report the codec reconstruction gap separately: code-space generation cannot recover information that the codec already removed. ### Practical procedure 1. **Establish the source baseline.** Score held-out source texts with the evaluator and preprocessing that will be used for every later stage. 2. **Measure reconstruction before generation.** Encode and decode the same examples without sampling new codes. This isolates the ceiling imposed by the codec. 3. **Evaluate both generation spaces after decoding.** Score token-space samples directly and decode code-space samples before scoring them. Do not compare a latent proxy with a decoded-text metric. 4. **Report distributions and uncertainty.** Show median and tail behavior, sample counts, preprocessing, and repeated-run uncertainty. One average can hide severe reconstruction failures. ### Evidence to require - The same external decoded-text evaluator and preprocessing are used at every checkpoint. - Source, reconstruction, token-space, and decoded code-space results are reported separately. - The codec reconstruction gap is not attributed to the latent generator. - Median and tail behavior accompany any mean comparison. - Seeds or uncertainty estimates are reported before small differences are generalized. ### Published evidence - In the reported 64-to-16 TinyStories setting, codec reconstruction alone raised median external perplexity from 15.17 to 27.36. - Under the same scorer, code-space MDLM reached median perplexity 26.55, while the token-space baseline reached 38.42, a 30.9% reduction for code-space generation in that experiment. - Geometry-aware regularization improved local latent diagnostics in the available matched runs but did not improve decoded-text metrics. ### Scope boundary - These numbers describe one TinyStories compression regime, one hierarchical codec, and the reported evaluation setup; they do not establish a universal ordering of code-space and token-space models. - Perplexity is informative but not a complete measure of semantic quality, diversity, factuality, or usefulness. - The available comparisons should be read with their stated sample sizes and uncertainty limitations. ## Localized code modification with generative models Canonical note: https://aogavrilov.com/research-notes/localized-code-modification-generative-models/ Markdown note: https://aogavrilov.com/research-notes/localized-code-modification-generative-models/index.md Source publication: https://aogavrilov.com/publications/inspectable-control/ Source full text: https://aogavrilov.com/publications/inspectable-control/full-text/ ### How can a generative model modify code without rewriting the entire function? Define protected and editable regions before generation, reuse or constrain the protected representation, generate only candidate changes, and reject outputs that alter protected code or fail task-specific checks. Locality and task success must be measured together. ### Practical procedure 1. **State the preservation contract.** Identify the editable region and the exact text, structure, interface, or behavior that must remain unchanged. 2. **Choose the narrowest useful control surface.** Reuse unchanged source spans, use infilling or edit-oriented decoding, apply formal constraints, or lock selected latent positions according to the required property. 3. **Generate only where change is allowed.** Keep enough freedom inside the editable region to solve the task; copying the entire input is local but makes no progress. 4. **Verify locality and success together.** Reject candidates that alter protected regions, fail parsing or compilation, violate structural invariants, or do not satisfy the requested change. ### Evidence to require - Outside-region diff or another direct protected-region stability measure. - Task success inside the editable region. - Parsing, compilation, tests, static checks, or task-specific invariants as applicable. - Editable-region change rate so copying is not mistaken for control. - Candidate uniqueness and repeated-run variability so locality is not confused with mode collapse. ### Published evidence - The linked experiment compresses 64-token Python functions into hierarchical discrete positions and regenerates selected latent positions under partial constraints. - Locking four top-level codes raised parse rate from 0.453 to 0.591 while unlocked positions changed at rate 0.936 and conditional samples remained 0.998 unique. - Those measurements expose a stability-freedom trade-off above the token level; they do not prove exact source-span, AST, semantic, or behavioral preservation. ### Scope boundary - A locked latent code is not automatically an AST node, protected source span, or formal invariant. - Parse rate establishes syntactic well-formedness, not functional correctness or successful repair. - The reported evidence comes from short preprocessed Python functions and does not establish repository-scale behavior. ## Constrained code generation for software engineering Canonical note: https://aogavrilov.com/research-notes/constrained-code-generation-software-engineering/ Markdown note: https://aogavrilov.com/research-notes/constrained-code-generation-software-engineering/index.md Source publication: https://aogavrilov.com/publications/inspectable-control/ Source full text: https://aogavrilov.com/publications/inspectable-control/full-text/ ### What does constrained code generation guarantee in a software-engineering workflow? Only the property explicitly enforced by the constraint. Grammar-constrained decoding can guarantee membership in a grammar; type-aware methods can target type validity; neither alone proves task correctness, semantic equivalence, behavior preservation, or edit locality. ### Practical procedure 1. **Name the required property.** Decide whether the requirement concerns grammar, types, APIs, source locality, structural invariants, tests, or another observable contract. 2. **Choose an enforcement point.** Apply constraints during decoding when possible, or use proposal-plus-validation when the property can only be checked after generation. 3. **Keep separate acceptance checks.** Test task success and protected properties even when the decoder already guarantees syntax or types. 4. **Report rejection and failure behavior.** A constrained method should disclose how often candidates are rejected, whether valid solutions remain reachable, and what is still unchecked. ### Evidence to require - The constrained property is stated in observable terms. - The enforcement mechanism and post-generation checks are distinguished. - Syntax or type validity is not presented as functional correctness. - Locality is measured directly when unchanged code is part of the claim. - Constraint failures, rejection rates, and task-success rates are reported. ### Published evidence - The linked hierarchical-latent study locks selected learned codes and measures decoded parse rate, edit freedom, and diversity. - This is an inspectable partial-control experiment, not a formal grammar, type, semantic, or behavior guarantee. - Its value for constrained workflows is the explicit control surface and measurement discipline, not a claim that latent locking replaces formal validation. ### Scope boundary - Different constraints can conflict; stronger restriction can remove valid solutions or reduce generation diversity. - Post-generation tests provide evidence only for the behaviors they cover. - The linked paper does not evaluate formal constrained decoding or repository-scale software repair. ## AI-assisted refactoring: methods and evidence Canonical note: https://aogavrilov.com/research-notes/ai-assisted-refactoring-evidence/ Markdown note: https://aogavrilov.com/research-notes/ai-assisted-refactoring-evidence/index.md Source publication: https://aogavrilov.com/publications/inspectable-control/ Source full text: https://aogavrilov.com/publications/inspectable-control/full-text/ ### Which methods and evidence matter for AI-assisted behavior-preserving refactoring? Separate transformation proposal from trusted execution and verification. Where possible, let the model identify a refactoring and apply it through a refactoring engine; then require compilation, tests, static checks, and evidence that the intended transformation occurred. ### Practical procedure 1. **Specify the intended refactoring.** Name the structural change and the behavior that must remain stable rather than asking for a generic cleanup. 2. **Prefer trusted execution for known transformations.** When a refactoring engine supports the operation, use the model for detection or parameter selection and the engine for application. 3. **Verify generated patches at repository level.** Compile, run relevant tests, apply static checks, and confirm that the intended refactoring occurred without unrelated changes. 4. **Audit residual risk.** Record uncovered behavior, flaky tests, cross-file effects, and cases where a plausible patch could not be verified. ### Evidence to require - The transformation is identified, not described only as code quality improvement. - Compilation and relevant tests pass after the change. - Static checks and refactoring detection support the structural claim. - Unrelated diff outside the intended scope is measured or reviewed. - Repository context and test coverage limitations are disclosed. ### Published evidence - The site paper on hierarchical latent control is adjacent evidence about bounded generation, not a behavior-preserving refactoring benchmark. - Its parse-rate, edit-freedom, and diversity measurements can inform control-surface design, but they do not replace compilation, tests, or refactoring detection. - For refactoring claims, the evidence contract should remain behavior- and repository-aware. ### Scope boundary - Passing available tests does not prove semantic equivalence for untested behavior. - A smaller diff is not automatically a correct refactoring. - The linked site experiment covers short Python functions and does not evaluate repository-level refactoring. ## Predictable code generation needs a preservation contract Canonical note: https://aogavrilov.com/research-notes/predictable-code-generation-preservation-contract/ Markdown note: https://aogavrilov.com/research-notes/predictable-code-generation-preservation-contract/index.md Source publication: https://aogavrilov.com/publications/inspectable-control/ Source full text: https://aogavrilov.com/publications/inspectable-control/full-text/ ### What makes code generation predictable rather than merely controllable? Predictability requires an observable contract stated before generation: what may change, what must remain stable, how each property will be measured, and when an output will be rejected. A fixed prompt, seed, mask, grammar, or latent code is only a mechanism. ### Practical procedure 1. **Declare protected properties.** State the source regions, structures, interfaces, behaviors, or distributions that must remain within a defined tolerance. 2. **Choose a matching control mechanism.** Use source reuse, edit masks, formal constraints, latent locking, or proposal-plus-validation according to the property being protected. 3. **Define acceptance before sampling.** Specify parsing, compilation, tests, structural checks, locality thresholds, and task-success criteria before inspecting outputs. 4. **Measure repeated behavior.** Report acceptance, rejection, stability, diversity, and uncertainty across repeated runs instead of relying on one illustrative result. ### Evidence to require - Protected and editable properties are stated separately. - Every claimed guarantee has an observable test or metric. - Task success is evaluated alongside stability. - Repeated-run variability and rejection rates are visible. - Unmeasured properties are explicitly excluded from the claim. ### Published evidence - The linked experiment exposes hierarchical discrete positions as one inspectable control surface for short Python functions. - Locking four top-level positions improved parse rate while leaving high change in unlocked positions and high sample uniqueness. - The paper reports this as early probabilistic evidence; it does not claim exact AST preservation, semantic equivalence, functional correctness, or repository-scale predictability. ### Scope boundary - Predictability is property-specific: a system can be predictable about syntax and unpredictable about behavior. - Deterministic decoding can repeat the same wrong or over-broad edit. - The available site experiment does not establish guarantees beyond its measured short-function setting. ## Русские сфокусированные исследовательские заметки Русский индекс: https://aogavrilov.com/ru/research-notes/ ## Маскированная диффузия в пространстве кодов и токенов: как их сравнивать Каноническая русская страница: https://aogavrilov.com/ru/research-notes/code-space-vs-token-space-masked-diffusion/ Русская версия Markdown: https://aogavrilov.com/ru/research-notes/code-space-vs-token-space-masked-diffusion/index.md Исходная английская страница: https://aogavrilov.com/research-notes/code-space-vs-token-space-masked-diffusion/ Исходная статья: https://aogavrilov.com/publications/where-quality-breaks/ ### Как следует сравнивать маскированное диффузионное языковое моделирование в пространстве кодов и токенов? Оценивайте исходный текст, реконструкции кодека, результаты в пространстве токенов и декодированные результаты в пространстве кодов одним внешним оценщиком и с одинаковой предобработкой. Разрыв реконструкции кодека нужно показывать отдельно: генератор кодов не может восстановить информацию, которую кодек уже удалил. ### Практическая процедура 1. **Задать базовый уровень исходного текста.** Оцените отложенные исходные тексты тем же оценщиком и с той же предобработкой, которые будут использоваться на всех последующих этапах. 2. **Измерить реконструкцию до генерации.** Закодируйте и декодируйте те же примеры без выборки новых кодов. Это изолирует потолок качества, заданный кодеком. 3. **Сравнить оба пространства после декодирования.** Оценивайте результаты в пространстве токенов напрямую, а результаты в пространстве кодов — только после декодирования. Не сравнивайте латентную прокси-метрику с метрикой декодированного текста. 4. **Показать распределения и неопределённость.** Сообщайте медиану, поведение хвостов, число примеров, предобработку и неопределённость повторных запусков. Одно среднее значение может скрыть тяжёлые ошибки реконструкции. ### Требуемые доказательства - Во всех контрольных точках используется один внешний оценщик декодированного текста и одна предобработка. - Результаты для исходного текста, реконструкции, пространства токенов и декодированного пространства кодов показаны отдельно. - Разрыв реконструкции кодека не приписывается латентному генератору. - Средние значения сопровождаются медианой и поведением хвостов. - До обобщения небольших различий сообщаются случайные зерна или оценки неопределённости. ### Опубликованные свидетельства - В описанном режиме TinyStories со сжатием 64→16 одна только реконструкция кодека повысила медианную внешнюю перплексию с 15,17 до 27,36. - При том же оценщике медианная перплексия MDLM в пространстве кодов составила 26,55, а базовой модели в пространстве токенов — 38,42; в этом эксперименте снижение для кодовой генерации составило 30,9%. - В доступных сопоставимых запусках геометрически осведомлённая регуляризация улучшила локальные латентные диагностические показатели, но не улучшила метрики декодированного текста. ### Границы применимости - Эти числа относятся к одному режиму сжатия TinyStories, одному иерархическому кодеку и описанной схеме оценки; они не задают универсальный порядок моделей в пространстве кодов и токенов. - Перплексия полезна, но не измеряет полностью семантическое качество, разнообразие, фактическую точность или практическую полезность. - Сравнения нужно интерпретировать вместе с указанными размерами выборок и ограничениями неопределённости. ## Локальное изменение кода генеративными моделями Каноническая русская страница: https://aogavrilov.com/ru/research-notes/localized-code-modification-generative-models/ Русская версия Markdown: https://aogavrilov.com/ru/research-notes/localized-code-modification-generative-models/index.md Исходная английская страница: https://aogavrilov.com/research-notes/localized-code-modification-generative-models/ Исходная статья: https://aogavrilov.com/publications/inspectable-control/ ### Как генеративная модель может изменять код, не переписывая функцию целиком? До генерации определите защищённые и редактируемые области, повторно используйте или ограничьте защищённое представление, генерируйте только варианты изменения и отклоняйте результаты, которые затрагивают защищённый код или не проходят проверки задачи. Локальность и успешность нужно измерять вместе. ### Практическая процедура 1. **Сформулировать контракт сохранения.** Укажите редактируемую область и точный текст, структуру, интерфейс или поведение, которые должны остаться неизменными. 2. **Выбрать самый узкий достаточный механизм управления.** Повторно используйте неизменённые фрагменты, применяйте заполнение пропусков или декодирование для редактирования, формальные ограничения либо фиксацию выбранных латентных позиций — в зависимости от защищаемого свойства. 3. **Генерировать только там, где изменение разрешено.** Оставьте внутри редактируемой области достаточно свободы для решения задачи: копирование всего входа локально, но не выполняет требуемое изменение. 4. **Проверять локальность и успех одновременно.** Отклоняйте варианты, которые меняют защищённые области, не разбираются или не компилируются, нарушают структурные инварианты либо не выполняют запрос. ### Требуемые доказательства - Diff вне редактируемой области или другая прямая мера стабильности защищённой области. - Успешность задачи внутри редактируемой области. - Разбор, компиляция, тесты, статические проверки или инварианты задачи — в зависимости от сценария. - Доля изменений в редактируемой области, чтобы копирование не считалось управлением. - Уникальность вариантов и изменчивость повторных запусков, чтобы локальность не путалась с коллапсом мод. ### Опубликованные свидетельства - Связанный эксперимент сжимает Python-функции длиной 64 токена в иерархические дискретные позиции и регенерирует выбранные латентные позиции при частичных ограничениях. - Фиксация четырёх кодов верхнего уровня повысила долю синтаксически разбираемых результатов с 0,453 до 0,591; незаблокированные позиции менялись с частотой 0,936, а условные выборки оставались уникальными с долей 0,998. - Эти измерения показывают компромисс между стабильностью и свободой выше уровня токенов, но не доказывают точное сохранение фрагментов, AST, семантики или поведения. ### Границы применимости - Зафиксированный латентный код не становится автоматически узлом AST, защищённым фрагментом или формальным инвариантом. - Доля разбираемых результатов подтверждает синтаксическую корректность, но не функциональную правильность или успешность исправления. - Доказательства получены на коротких предобработанных Python-функциях и не устанавливают поведение на уровне репозитория. ## Ограниченная генерация кода для программной инженерии Каноническая русская страница: https://aogavrilov.com/ru/research-notes/constrained-code-generation-software-engineering/ Русская версия Markdown: https://aogavrilov.com/ru/research-notes/constrained-code-generation-software-engineering/index.md Исходная английская страница: https://aogavrilov.com/research-notes/constrained-code-generation-software-engineering/ Исходная статья: https://aogavrilov.com/publications/inspectable-control/ ### Что гарантирует ограниченная генерация кода в процессе разработки программного обеспечения? Только то свойство, которое явно обеспечивается ограничением. Декодирование по грамматике может гарантировать принадлежность грамматике, а методы с учётом типов — стремиться к корректности типов; ни то ни другое само по себе не доказывает правильность задачи, семантическую эквивалентность, сохранение поведения или локальность изменений. ### Практическая процедура 1. **Назвать требуемое свойство.** Определите, относится ли требование к грамматике, типам, API, локальности исходного кода, структурным инвариантам, тестам или другому наблюдаемому контракту. 2. **Выбрать точку применения ограничения.** По возможности применяйте ограничение при декодировании; если свойство проверяется только после генерации, используйте схему «предложение плюс проверка». 3. **Сохранить независимые проверки приёмки.** Проверяйте успешность задачи и защищённые свойства, даже если декодер уже гарантирует синтаксис или типы. 4. **Сообщать о блокировках и отказах.** Показывайте, как часто варианты отклоняются, остаются ли достижимыми корректные решения и какие свойства всё ещё не проверены. ### Требуемые доказательства - Ограничиваемое свойство сформулировано в наблюдаемых терминах. - Механизм применения ограничения отделён от проверок после генерации. - Корректность синтаксиса или типов не выдаётся за функциональную правильность. - Локальность измеряется напрямую, если заявляется сохранение неизменённого кода. - Сообщаются отказы ограничений, доля отклонений и успешность задачи. ### Опубликованные свидетельства - Связанное исследование иерархических латентных переменных фиксирует выбранные обученные коды и измеряет разбор, свободу редактирования и разнообразие после декодирования. - Это эксперимент с проверяемым частичным управлением, а не формальная гарантия грамматики, типов, семантики или поведения. - Для ограниченных процессов важны явный механизм управления и дисциплина измерений, а не предположение, что фиксация латентных кодов заменяет формальную проверку. ### Границы применимости - Разные ограничения могут конфликтовать; более сильное ограничение способно исключить корректные решения или снизить разнообразие. - Проверки после генерации дают свидетельства только о покрываемом ими поведении. - Связанная статья не оценивает формально ограниченное декодирование или исправление на уровне репозитория. ## Рефакторинг с помощью ИИ: методы и доказательства Каноническая русская страница: https://aogavrilov.com/ru/research-notes/ai-assisted-refactoring-evidence/ Русская версия Markdown: https://aogavrilov.com/ru/research-notes/ai-assisted-refactoring-evidence/index.md Исходная английская страница: https://aogavrilov.com/research-notes/ai-assisted-refactoring-evidence/ Исходная статья: https://aogavrilov.com/publications/inspectable-control/ ### Какие методы и доказательства важны для рефакторинга с помощью ИИ и сохранением поведения? Отделяйте предложение преобразования от доверенного исполнения и проверки. Если возможно, поручайте модели распознать рефакторинг, а применять его — движку рефакторинга; затем требуйте компиляцию, тесты, статические проверки и подтверждение того, что нужное преобразование действительно выполнено. ### Практическая процедура 1. **Указать ожидаемый рефакторинг.** Назовите структурное преобразование и поведение, которое должно остаться стабильным, вместо общего запроса «почистить код». 2. **Для известных преобразований предпочесть доверенное исполнение.** Если операция поддерживается движком рефакторинга, используйте модель для обнаружения или выбора параметров, а движок — для применения. 3. **Проверить сгенерированный патч на уровне репозитория.** Соберите проект, запустите релевантные тесты и статические проверки, убедитесь в наличии ожидаемого рефакторинга и отсутствии посторонних изменений. 4. **Проверить остаточный риск.** Зафиксируйте непокрытое поведение, нестабильные тесты, межфайловые эффекты и правдоподобные патчи, которые не удалось подтвердить. ### Требуемые доказательства - Преобразование названо явно, а не описано только как улучшение качества кода. - После изменения проект компилируется и проходит релевантные тесты. - Статические проверки и обнаружение рефакторинга поддерживают структурное утверждение. - Изменения вне ожидаемой области измерены или проверены. - Ограничения контекста репозитория и покрытия тестами раскрыты. ### Опубликованные свидетельства - Статья сайта об иерархическом латентном управлении даёт смежные данные об ограниченной генерации, но не является тестом рефакторинга с сохранением поведения. - Её измерения разбираемости, свободы редактирования и разнообразия полезны для проектирования управления, но не заменяют компиляцию, тесты или обнаружение рефакторинга. - Для утверждений о рефакторинге контракт доказательств должен учитывать поведение и контекст репозитория. ### Границы применимости - Прохождение имеющихся тестов не доказывает семантическую эквивалентность непроверенного поведения. - Меньший diff не обязательно означает корректный рефакторинг. - Связанный эксперимент сайта охватывает короткие Python-функции, а не рефакторинг на уровне репозитория. ## Предсказуемой генерации кода нужен контракт сохранения Каноническая русская страница: https://aogavrilov.com/ru/research-notes/predictable-code-generation-preservation-contract/ Русская версия Markdown: https://aogavrilov.com/ru/research-notes/predictable-code-generation-preservation-contract/index.md Исходная английская страница: https://aogavrilov.com/research-notes/predictable-code-generation-preservation-contract/ Исходная статья: https://aogavrilov.com/publications/inspectable-control/ ### Что делает генерацию кода предсказуемой, а не просто управляемой? Предсказуемость требует наблюдаемого контракта, заданного до генерации: что может измениться, что должно остаться стабильным, как измеряется каждое свойство и когда результат отклоняется. Фиксированный запрос, зерно, маска, грамматика или латентный код — лишь механизмы. ### Практическая процедура 1. **Объявить защищённые свойства.** Укажите области исходного кода, структуры, интерфейсы, поведение или распределения, которые должны оставаться в заданном допуске. 2. **Выбрать подходящий механизм управления.** Используйте повторное применение исходного кода, маски редактирования, формальные ограничения, фиксацию латентных переменных или схему «предложение плюс проверка» в соответствии с защищаемым свойством. 3. **Задать приёмку до выборки.** Определите разбор, компиляцию, тесты, структурные проверки, пороги локальности и критерии успешности до просмотра результатов. 4. **Измерить повторяемое поведение.** Показывайте принятие, отклонение, стабильность, разнообразие и неопределённость на повторных запусках, а не на одном удачном примере. ### Требуемые доказательства - Защищённые и редактируемые свойства сформулированы отдельно. - Для каждой заявленной гарантии есть наблюдаемая проверка или метрика. - Успешность задачи оценивается вместе со стабильностью. - Показаны изменчивость повторных запусков и доля отклонений. - Неизмеренные свойства явно исключены из утверждения. ### Опубликованные свидетельства - Связанный эксперимент использует иерархические дискретные позиции как один проверяемый механизм управления короткими Python-функциями. - Фиксация четырёх позиций верхнего уровня повысила разбираемость, сохранив высокую долю изменений в незаблокированных позициях и высокую уникальность выборок. - Работа описывает это как раннее вероятностное свидетельство и не заявляет точное сохранение AST, семантическую эквивалентность, функциональную корректность или предсказуемость на уровне репозитория. ### Границы применимости - Предсказуемость зависит от свойства: система может быть предсказуемой по синтаксису и непредсказуемой по поведению. - Детерминированное декодирование способно повторять одну и ту же ошибочную или чрезмерно широкую правку. - Имеющийся эксперимент сайта не устанавливает гарантий за пределами измеренного режима коротких функций. ## 简体中文聚焦研究笔记 简体中文索引：https://aogavrilov.com/zh/research-notes/ ## 码空间与 token 空间掩码扩散：如何公平比较 简体中文规范页面：https://aogavrilov.com/zh/research-notes/code-space-vs-token-space-masked-diffusion/ 简体中文 Markdown：https://aogavrilov.com/zh/research-notes/code-space-vs-token-space-masked-diffusion/index.md 英文原页：https://aogavrilov.com/research-notes/code-space-vs-token-space-masked-diffusion/ 来源论文：https://aogavrilov.com/publications/where-quality-breaks/ ### 应如何比较码空间与 token 空间的掩码扩散语言建模？ 对源文本、编解码器重建、token 空间输出和解码后的码空间输出采用同一个外部评分器与同一套预处理。必须单独报告编解码器重建差距：码空间生成无法恢复编解码器已经丢失的信息。 ### 实用步骤 1. **建立源文本基线。** 使用后续所有阶段都会采用的评分器和预处理，对留出的源文本进行评分。 2. **在生成之前测量重建。** 对同一批样本只执行编码和解码，不采样新码，从而隔离编解码器施加的质量上限。 3. **解码后再比较两个生成空间。** 直接评分 token 空间样本；对码空间样本先解码再评分。不要把潜在空间代理指标与解码文本指标直接比较。 4. **报告分布与不确定性。** 同时给出中位数、尾部表现、样本数量、预处理方式和重复运行的不确定性；单一平均值可能掩盖严重的重建失败。 ### 应要求哪些证据 - 每个检查点使用同一个外部解码文本评分器和同一套预处理。 - 分别报告源文本、重建文本、token 空间输出和解码后的码空间输出。 - 不把编解码器重建差距归因于潜在生成器。 - 除平均值外，同时报告中位数和尾部表现。 - 在推广微小差异之前，报告随机种子或不确定性估计。 ### 已发表证据 - 在论文报告的 TinyStories 64→16 压缩设置中，仅编解码器重建就使外部困惑度中位数从 15.17 上升到 27.36。 - 在同一评分器下，码空间 MDLM 的困惑度中位数为 26.55，token 空间基线为 38.42；在该实验中，码空间生成的困惑度降低了 30.9%。 - 在可比运行中，几何感知正则化改善了局部潜在空间诊断指标，但没有改善解码文本指标。 ### 适用边界 - 这些数值只描述一种 TinyStories 压缩配置、一个分层编解码器和论文报告的评估设置，不能证明码空间模型普遍优于 token 空间模型。 - 困惑度有参考价值，但不能完整衡量语义质量、多样性、事实性或实际用途。 - 解释这些比较时，必须同时考虑论文给出的样本量与不确定性限制。 ## 使用生成模型进行局部代码修改 简体中文规范页面：https://aogavrilov.com/zh/research-notes/localized-code-modification-generative-models/ 简体中文 Markdown：https://aogavrilov.com/zh/research-notes/localized-code-modification-generative-models/index.md 英文原页：https://aogavrilov.com/research-notes/localized-code-modification-generative-models/ 来源论文：https://aogavrilov.com/publications/inspectable-control/ ### 如何让生成模型修改代码而不重写整个函数？ 在生成之前定义受保护区域与可编辑区域，复用或约束受保护表示，只生成候选变更，并拒绝任何修改受保护代码或未通过任务检查的输出。必须同时衡量局部性和任务成功。 ### 实用步骤 1. **声明保持契约。** 明确可编辑区域，以及必须保持不变的文本、结构、接口或行为。 2. **选择最窄但足够的控制界面。** 根据需要保护的属性，复用未变源代码、采用填空式或编辑式解码、施加形式约束，或锁定选定的潜在位置。 3. **只在允许变化的位置生成。** 在可编辑区域内保留足够自由度以完成任务；复制全部输入虽然局部，但没有完成任何修改。 4. **同时验证局部性与成功。** 拒绝修改受保护区域、无法解析或编译、违反结构不变量，或者没有完成所请求变更的候选结果。 ### 应要求哪些证据 - 直接测量可编辑区域之外的 diff，或采用另一种受保护区域稳定性指标。 - 测量可编辑区域内的任务成功。 - 根据场景执行解析、编译、测试、静态检查或任务专用不变量验证。 - 报告可编辑区域的变化率，避免把原样复制误认为控制能力。 - 报告候选唯一性和重复运行差异，避免把局部性与模式坍塌混为一谈。 ### 已发表证据 - 相关实验把 64-token 的 Python 函数压缩为分层离散位置，并在部分约束下重新生成选定的潜在位置。 - 锁定四个顶层码后，解析率从 0.453 提高到 0.591；未锁定位置的变化率为 0.936，条件样本仍有 0.998 的唯一率。 - 这些测量揭示了 token 之上的稳定性—自由度权衡，但不能证明源代码片段、AST、语义或行为被精确保留。 ### 适用边界 - 被锁定的潜在码不会自动对应 AST 节点、受保护源代码片段或形式不变量。 - 解析率只说明语法可解析，不能证明功能正确或修复成功。 - 报告的证据来自经过预处理的短 Python 函数，不能推导出仓库级行为。 ## 面向软件工程的受约束代码生成 简体中文规范页面：https://aogavrilov.com/zh/research-notes/constrained-code-generation-software-engineering/ 简体中文 Markdown：https://aogavrilov.com/zh/research-notes/constrained-code-generation-software-engineering/index.md 英文原页：https://aogavrilov.com/research-notes/constrained-code-generation-software-engineering/ 来源论文：https://aogavrilov.com/publications/inspectable-control/ ### 受约束代码生成在软件工程流程中能够保证什么？ 它只能保证被显式执行的属性。语法约束解码可以保证输出属于某个语法；类型感知方法可以以类型有效性为目标；但两者都不能单独证明任务正确、语义等价、行为保持或编辑局部性。 ### 实用步骤 1. **命名所需属性。** 明确要求涉及语法、类型、API、源代码局部性、结构不变量、测试行为，还是其他可观察契约。 2. **选择约束执行位置。** 在可能时于解码期间施加约束；如果属性只能在生成后检查，则采用“提出候选—验证候选”的流程。 3. **保留独立验收检查。** 即使解码器已经保证语法或类型，也必须另行检查任务成功和受保护属性。 4. **报告拒绝与失败行为。** 说明候选被拒绝的频率、有效解是否仍然可达，以及哪些属性仍未被检查。 ### 应要求哪些证据 - 用可观察的术语声明被约束的属性。 - 区分约束执行机制与生成后的验收检查。 - 不把语法或类型有效性描述成功能正确性。 - 如果声称未变代码得到保持，则直接测量局部性。 - 报告约束失败、拒绝率和任务成功率。 ### 已发表证据 - 相关分层潜在研究锁定选定的学习式码，并测量解码后的解析率、编辑自由度和多样性。 - 这是一个可检查的部分控制实验，不是形式语法、类型、语义或行为保证。 - 它对受约束流程的价值在于显式控制界面和测量纪律，而不是声称潜在码锁定可以替代形式验证。 ### 适用边界 - 不同约束可能相互冲突；更强的限制可能排除有效解或降低生成多样性。 - 生成后测试只能为其覆盖到的行为提供证据。 - 相关论文没有评估形式约束解码或仓库级软件修复。 ## AI 辅助重构：方法与证据 简体中文规范页面：https://aogavrilov.com/zh/research-notes/ai-assisted-refactoring-evidence/ 简体中文 Markdown：https://aogavrilov.com/zh/research-notes/ai-assisted-refactoring-evidence/index.md 英文原页：https://aogavrilov.com/research-notes/ai-assisted-refactoring-evidence/ 来源论文：https://aogavrilov.com/publications/inspectable-control/ ### 评估 AI 辅助、行为保持的重构时，哪些方法和证据最重要？ 把变换建议与可信执行和验证分开。条件允许时，让模型识别重构，再由重构引擎执行；随后要求编译、测试和静态检查通过，并确认预期变换确实发生。 ### 实用步骤 1. **明确预期重构。** 命名结构变换和必须保持稳定的行为，而不是只提出笼统的“清理代码”要求。 2. **已知变换优先采用可信执行。** 如果重构引擎支持该操作，让模型负责检测或参数选择，让引擎负责应用变换。 3. **在仓库级验证生成补丁。** 执行编译和相关测试，应用静态检查，并确认预期重构发生且没有引入无关变更。 4. **审计剩余风险。** 记录未覆盖行为、不稳定测试、跨文件影响，以及无法验证的可疑补丁。 ### 应要求哪些证据 - 明确识别具体变换，而不是只称其为代码质量改进。 - 变更后能够编译并通过相关测试。 - 静态检查和重构检测支持所声称的结构变化。 - 测量或审查预期范围之外的无关 diff。 - 披露仓库上下文和测试覆盖的限制。 ### 已发表证据 - 站内分层潜在控制论文提供的是有边界生成的邻近证据，而不是行为保持重构基准。 - 其中的解析率、编辑自由度和多样性测量可帮助设计控制界面，但不能替代编译、测试或重构检测。 - 对于重构主张，证据契约仍应面向行为和仓库上下文。 ### 适用边界 - 通过现有测试不能证明未测试行为也语义等价。 - 更小的 diff 不一定就是正确重构。 - 相关站内实验只覆盖短 Python 函数，没有评估仓库级重构。 ## 可预测代码生成需要保持契约 简体中文规范页面：https://aogavrilov.com/zh/research-notes/predictable-code-generation-preservation-contract/ 简体中文 Markdown：https://aogavrilov.com/zh/research-notes/predictable-code-generation-preservation-contract/index.md 英文原页：https://aogavrilov.com/research-notes/predictable-code-generation-preservation-contract/ 来源论文：https://aogavrilov.com/publications/inspectable-control/ ### 什么使代码生成具有可预测性，而不只是可控制？ 可预测性需要在生成之前声明可观察契约：什么可以变化，什么必须稳定，如何测量每个属性，以及何时拒绝输出。固定提示、随机种子、掩码、语法或潜在码都只是机制。 ### 实用步骤 1. **声明受保护属性。** 明确必须保持在给定容差内的源代码区域、结构、接口、行为或分布。 2. **选择匹配的控制机制。** 根据要保护的属性，采用源代码复用、编辑掩码、形式约束、潜在码锁定或“提出候选—验证候选”流程。 3. **在采样前定义验收条件。** 在查看输出之前，规定解析、编译、测试、结构检查、局部性阈值和任务成功标准。 4. **测量重复运行行为。** 跨重复运行报告接受、拒绝、稳定性、多样性和不确定性，而不是依赖一个示例。 ### 应要求哪些证据 - 分别声明受保护属性和可编辑属性。 - 每个所声称的保证都有可观察测试或指标。 - 在衡量稳定性的同时评估任务成功。 - 展示重复运行差异与拒绝率。 - 明确排除尚未测量的属性。 ### 已发表证据 - 相关实验把分层离散位置作为短 Python 函数的一种可检查控制界面。 - 锁定四个顶层位置后，解析率提高，同时未锁定位置保持较高变化率，样本唯一率也保持较高。 - 论文把这描述为早期概率性证据，并未声称精确 AST 保持、语义等价、功能正确或仓库级可预测性。 ### 适用边界 - 可预测性取决于具体属性：系统可能在语法上可预测，却在行为上不可预测。 - 确定性解码也可能重复同一个错误或过度宽泛的编辑。 - 现有站内实验不能建立超出短函数测量范围的保证。 ## 한국어 문제 중심 연구 노트 한국어 목록: https://aogavrilov.com/ko/research-notes/ ## 코드 공간과 토큰 공간 마스크 확산: 공정한 비교 방법 표준 한국어 페이지: https://aogavrilov.com/ko/research-notes/code-space-vs-token-space-masked-diffusion/ 한국어 Markdown: https://aogavrilov.com/ko/research-notes/code-space-vs-token-space-masked-diffusion/index.md 영어 원문: https://aogavrilov.com/research-notes/code-space-vs-token-space-masked-diffusion/ 출처 논문: https://aogavrilov.com/publications/where-quality-breaks/ ### 코드 공간과 토큰 공간 마스크 확산 언어 모델은 어떻게 비교해야 하는가? 원문, 코덱 재구성, 토큰 공간 출력, 디코딩한 코드 공간 출력을 같은 외부 평가기와 전처리로 측정한다. 코덱 재구성 격차는 따로 보고해야 한다. 코드 공간 생성기는 코덱이 이미 제거한 정보를 복구할 수 없기 때문이다. ### 실행 절차 1. **원문 기준선 구축.** 이후 모든 단계에 사용할 평가기와 전처리로 보류한 원문 텍스트를 점수화한다. 2. **생성 전에 재구성 측정.** 새 코드를 샘플링하지 않고 같은 예제를 인코딩하고 디코딩한다. 이 과정은 코덱이 정한 품질 상한을 분리한다. 3. **디코딩 뒤 두 생성 공간 평가.** 토큰 공간 샘플은 직접 평가하고 코드 공간 샘플은 먼저 디코딩한 뒤 평가한다. 잠재 대리 지표와 디코딩 텍스트 지표를 직접 비교하지 않는다. 4. **분포와 불확실성 보고.** 중앙값과 꼬리 구간, 샘플 수, 전처리, 반복 실행의 불확실성을 함께 제시한다. 평균 하나만으로는 심각한 재구성 실패를 가릴 수 있다. ### 요구해야 할 근거 - 모든 체크포인트에서 같은 외부 디코딩 텍스트 평가기와 전처리를 사용한다. - 원문, 재구성, 토큰 공간, 디코딩한 코드 공간 결과를 따로 보고한다. - 코덱 재구성 격차를 잠재 생성기의 손실로 돌리지 않는다. - 평균을 제시할 때 중앙값과 꼬리 구간도 함께 제시한다. - 작은 차이를 일반화하기 전에 시드 또는 불확실성 추정치를 보고한다. ### 공개된 근거 - 보고된 TinyStories 64→16 설정에서 코덱 재구성만으로 외부 perplexity 중앙값이 15.17에서 27.36으로 증가했다. - 같은 평가기에서 코드 공간 MDLM의 중앙값 perplexity는 26.55, 토큰 공간 기준선은 38.42였으며, 해당 실험에서 코드 공간 생성은 30.9% 낮았다. - 비교 가능한 실행에서 기하 인식 정규화는 국소 잠재 진단 지표를 개선했지만 디코딩 텍스트 지표는 개선하지 못했다. ### 적용 범위와 한계 - 이 수치는 하나의 TinyStories 압축 체계, 하나의 계층적 코덱, 보고된 평가 설정을 설명하며 코드 공간 모델이 항상 우수하다는 뜻이 아니다. - Perplexity는 유용하지만 의미 품질, 다양성, 사실성, 실용성을 모두 측정하지는 않는다. - 비교 결과는 논문에 명시된 샘플 수와 불확실성 한계와 함께 해석해야 한다. ## 생성 모델을 이용한 국소 코드 수정 표준 한국어 페이지: https://aogavrilov.com/ko/research-notes/localized-code-modification-generative-models/ 한국어 Markdown: https://aogavrilov.com/ko/research-notes/localized-code-modification-generative-models/index.md 영어 원문: https://aogavrilov.com/research-notes/localized-code-modification-generative-models/ 출처 논문: https://aogavrilov.com/publications/inspectable-control/ ### 생성 모델이 함수 전체를 다시 쓰지 않고 코드를 수정하게 하려면 어떻게 해야 하는가? 생성 전에 보호 영역과 편집 가능 영역을 정하고, 보호된 표현을 재사용하거나 제약하며, 변경 후보만 생성한다. 보호 코드를 바꾸거나 작업별 검사를 통과하지 못한 출력은 거부한다. 국소성과 작업 성공을 함께 측정해야 한다. ### 실행 절차 1. **보존 계약 명시.** 편집 가능 영역과 그대로 유지해야 하는 텍스트, 구조, 인터페이스 또는 동작을 정확히 정한다. 2. **가장 좁고 유용한 제어 수단 선택.** 필요한 속성에 따라 변경하지 않은 소스를 재사용하고, 인필링·편집 지향 디코딩을 사용하거나, 형식 제약을 적용하거나, 선택한 잠재 위치를 잠근다. 3. **허용된 위치에서만 생성.** 편집 영역 안에는 작업을 해결할 충분한 자유를 남긴다. 입력 전체를 복사하는 것은 국소적이지만 작업을 진전시키지 못한다. 4. **국소성과 성공을 함께 검증.** 보호 영역을 변경하거나, 파싱·컴파일에 실패하거나, 구조 불변량을 어기거나, 요청한 변경을 수행하지 못한 후보를 거부한다. ### 요구해야 할 근거 - 영역 밖 diff 또는 보호 영역 안정성을 직접 측정하는 다른 지표. - 편집 가능 영역 안에서의 작업 성공. - 상황에 맞는 파싱, 컴파일, 테스트, 정적 검사 또는 작업별 불변량. - 복사를 제어 능력으로 오인하지 않도록 편집 영역의 변화율. - 국소성을 모드 붕괴와 혼동하지 않도록 후보 고유성과 반복 실행 변동성. ### 공개된 근거 - 연결된 실험은 64토큰 Python 함수를 계층적 이산 위치로 압축하고 부분 제약 아래에서 선택한 잠재 위치를 다시 생성한다. - 상위 코드 네 개를 잠그면 파싱 성공률이 0.453에서 0.591로 증가했고, 잠그지 않은 위치의 변화율은 0.936, 조건부 샘플 고유성은 0.998이었다. - 이 측정은 토큰보다 높은 수준에서 안정성-자유도 절충을 보여 주지만 정확한 소스 범위, AST, 의미 또는 동작 보존을 증명하지 않는다. ### 적용 범위와 한계 - 잠근 잠재 코드가 자동으로 AST 노드, 보호된 소스 범위 또는 형식 불변량에 대응하는 것은 아니다. - 파싱 성공률은 문법적 형식만 확인하며 기능 정확성이나 수리 성공을 보장하지 않는다. - 보고된 근거는 전처리된 짧은 Python 함수에서 나온 것이며 저장소 규모 동작을 입증하지 않는다. ## 소프트웨어 공학을 위한 제약 코드 생성 표준 한국어 페이지: https://aogavrilov.com/ko/research-notes/constrained-code-generation-software-engineering/ 한국어 Markdown: https://aogavrilov.com/ko/research-notes/constrained-code-generation-software-engineering/index.md 영어 원문: https://aogavrilov.com/research-notes/constrained-code-generation-software-engineering/ 출처 논문: https://aogavrilov.com/publications/inspectable-control/ ### 소프트웨어 공학 워크플로에서 제약 코드 생성은 무엇을 보장하는가? 제약이 명시적으로 강제하는 속성만 보장한다. 문법 제약 디코딩은 특정 문법에 속함을 보장할 수 있고 타입 인식 방법은 타입 유효성을 목표로 할 수 있다. 그러나 어느 쪽도 그 자체로 작업 정확성, 의미 동등성, 동작 보존 또는 편집 국소성을 증명하지 않는다. ### 실행 절차 1. **필요한 속성 명명.** 요구사항이 문법, 타입, API, 소스 국소성, 구조 불변량, 테스트 또는 다른 관찰 가능 계약 중 무엇인지 정한다. 2. **강제 지점 선택.** 가능하면 디코딩 중에 제약을 적용하고, 생성 뒤에만 검사할 수 있는 속성은 후보 제안 후 검증 절차를 사용한다. 3. **별도의 합격 검사 유지.** 디코더가 문법이나 타입을 이미 보장하더라도 작업 성공과 보호 속성을 별도로 검사한다. 4. **거부와 실패 동작 보고.** 후보 거부 빈도, 유효한 해가 여전히 도달 가능한지, 아직 검사하지 않은 속성을 공개한다. ### 요구해야 할 근거 - 제약되는 속성을 관찰 가능한 용어로 명시한다. - 강제 메커니즘과 생성 후 검사를 구분한다. - 문법 또는 타입 유효성을 기능 정확성으로 제시하지 않는다. - 변경하지 않은 코드가 주장의 일부라면 국소성을 직접 측정한다. - 제약 실패, 거부율, 작업 성공률을 보고한다. ### 공개된 근거 - 연결된 계층적 잠재 연구는 선택한 학습 코드 위치를 잠그고 디코딩 뒤 파싱 성공률, 편집 자유도, 다양성을 측정한다. - 이는 점검 가능한 부분 제어 실험이며 형식 문법, 타입, 의미 또는 동작 보장이 아니다. - 제약 워크플로에 주는 가치는 명시적인 제어 표면과 측정 규율이지 잠재 잠금이 형식 검증을 대체한다는 주장이 아니다. ### 적용 범위와 한계 - 서로 다른 제약은 충돌할 수 있으며 더 강한 제한은 유효한 해를 제거하거나 생성 다양성을 줄일 수 있다. - 생성 후 테스트는 해당 테스트가 다루는 동작에 대해서만 근거를 제공한다. - 연결된 논문은 형식 제약 디코딩이나 저장소 규모 소프트웨어 수리를 평가하지 않는다. ## AI 보조 리팩터링: 방법과 근거 표준 한국어 페이지: https://aogavrilov.com/ko/research-notes/ai-assisted-refactoring-evidence/ 한국어 Markdown: https://aogavrilov.com/ko/research-notes/ai-assisted-refactoring-evidence/index.md 영어 원문: https://aogavrilov.com/research-notes/ai-assisted-refactoring-evidence/ 출처 논문: https://aogavrilov.com/publications/inspectable-control/ ### AI 보조 동작 보존 리팩터링에는 어떤 방법과 근거가 중요한가? 변환 제안과 신뢰할 수 있는 실행·검증을 분리한다. 가능하면 모델은 리팩터링을 식별하고 리팩터링 엔진이 적용하게 한다. 이후 컴파일, 테스트, 정적 검사, 의도한 변환이 실제로 일어났다는 근거를 요구한다. ### 실행 절차 1. **의도한 리팩터링 명시.** 일반적인 정리를 요청하지 말고 구조 변화와 안정적으로 유지해야 할 동작을 이름 붙인다. 2. **알려진 변환에는 신뢰 가능한 실행 우선.** 리팩터링 엔진이 지원하는 작업은 모델을 탐지나 매개변수 선택에 사용하고 실제 적용은 엔진에 맡긴다. 3. **저장소 수준에서 생성 패치 검증.** 컴파일하고 관련 테스트와 정적 검사를 실행하며, 의도한 리팩터링이 관련 없는 변경 없이 수행됐는지 확인한다. 4. **잔여 위험 감사.** 테스트되지 않은 동작, 불안정한 테스트, 파일 간 영향, 그럴듯하지만 검증할 수 없는 패치를 기록한다. ### 요구해야 할 근거 - 변환을 단순한 코드 품질 향상이 아니라 구체적인 작업으로 식별한다. - 변경 뒤 컴파일과 관련 테스트가 통과한다. - 정적 검사와 리팩터링 탐지가 구조적 주장을 뒷받침한다. - 의도한 범위 밖의 관련 없는 diff를 측정하거나 검토한다. - 저장소 문맥과 테스트 커버리지 한계를 공개한다. ### 공개된 근거 - 계층적 잠재 제어에 관한 사이트 논문은 제한된 생성에 대한 인접 근거이지 동작 보존 리팩터링 벤치마크가 아니다. - 파싱 성공률, 편집 자유도, 다양성 측정은 제어 수단 설계에 참고할 수 있지만 컴파일, 테스트 또는 리팩터링 탐지를 대체하지 않는다. - 리팩터링 주장의 근거 계약은 동작과 저장소 문맥을 계속 포함해야 한다. ### 적용 범위와 한계 - 사용 가능한 테스트를 통과해도 테스트되지 않은 동작의 의미 동등성을 증명하지는 않는다. - 더 작은 diff가 자동으로 올바른 리팩터링인 것은 아니다. - 연결된 사이트 실험은 짧은 Python 함수를 다루며 저장소 수준 리팩터링을 평가하지 않는다. ## 예측 가능한 코드 생성에는 보존 계약이 필요하다 표준 한국어 페이지: https://aogavrilov.com/ko/research-notes/predictable-code-generation-preservation-contract/ 한국어 Markdown: https://aogavrilov.com/ko/research-notes/predictable-code-generation-preservation-contract/index.md 영어 원문: https://aogavrilov.com/research-notes/predictable-code-generation-preservation-contract/ 출처 논문: https://aogavrilov.com/publications/inspectable-control/ ### 코드 생성을 단순히 제어 가능하게가 아니라 예측 가능하게 만드는 것은 무엇인가? 생성 전에 관찰 가능한 계약을 정해야 한다. 무엇을 바꿀 수 있는지, 무엇이 안정적으로 유지돼야 하는지, 각 속성을 어떻게 측정할지, 언제 출력을 거부할지를 명시한다. 고정 프롬프트, 시드, 마스크, 문법 또는 잠재 코드는 메커니즘일 뿐이다. ### 실행 절차 1. **보호 속성 선언.** 허용 범위 안에서 유지돼야 할 소스 영역, 구조, 인터페이스, 동작 또는 분포를 명시한다. 2. **속성에 맞는 제어 메커니즘 선택.** 보호할 속성에 따라 소스 재사용, 편집 마스크, 형식 제약, 잠재 잠금 또는 후보 제안 후 검증을 사용한다. 3. **샘플링 전에 합격 기준 정의.** 출력을 보기 전에 파싱, 컴파일, 테스트, 구조 검사, 국소성 임계값, 작업 성공 기준을 정한다. 4. **반복 동작 측정.** 한 가지 예시가 아니라 반복 실행의 합격률, 거부율, 안정성, 다양성, 불확실성을 보고한다. ### 요구해야 할 근거 - 보호 속성과 편집 가능 속성을 따로 명시한다. - 주장하는 모든 보장에 관찰 가능한 검사 또는 지표가 있다. - 안정성과 함께 작업 성공을 평가한다. - 반복 실행 변동성과 거부율을 공개한다. - 측정하지 않은 속성을 주장 범위에서 명시적으로 제외한다. ### 공개된 근거 - 연결된 실험은 짧은 Python 함수의 계층적 이산 위치를 점검 가능한 한 가지 제어 수단으로 노출한다. - 상위 위치 네 개를 잠그면 파싱 성공률이 향상됐고, 잠그지 않은 위치에서는 높은 변화율과 높은 샘플 고유성이 유지됐다. - 논문은 이를 초기 확률적 근거로 보고하며 정확한 AST 보존, 의미 동등성, 기능 정확성 또는 저장소 규모 예측 가능성을 주장하지 않는다. ### 적용 범위와 한계 - 예측 가능성은 속성별이다. 문법은 예측 가능하면서 동작은 예측 불가능할 수 있다. - 결정적 디코딩은 잘못되거나 지나치게 넓은 편집도 똑같이 반복할 수 있다. - 현재 사이트 실험은 측정된 짧은 함수 설정을 넘어서는 보장을 확립하지 않는다. ## Where Quality Breaks in Compressed Short-Text Generation: Staged Bottleneck Localization Canonical page: https://aogavrilov.com/publications/where-quality-breaks/ Full-text HTML: https://aogavrilov.com/publications/where-quality-breaks/full-text/ Full-text Markdown: https://aogavrilov.com/publications/where-quality-breaks/full-text/index.md Full-text section/block index JSON: https://aogavrilov.com/publications/where-quality-breaks/full-text/sections.json W3C Web Annotation block collection: https://aogavrilov.com/publications/where-quality-breaks/full-text/annotations.jsonld Source-derived citation-context index: https://aogavrilov.com/publications/where-quality-breaks/full-text/citations.json Full-text NISO JATS 1.4 XML: https://aogavrilov.com/publications/where-quality-breaks/full-text/article.jats.xml Markdown reader representation: https://aogavrilov.com/publications/where-quality-breaks/index.md DOI: https://doi.org/10.23919/FRUCT70069.2026.11506553 PDF: https://aogavrilov.com/publications/where-quality-breaks/paper.pdf Scholarly link set (JSON): https://aogavrilov.com/publications/where-quality-breaks/linkset.json Scholarly link set (HTTP Link format): https://aogavrilov.com/publications/where-quality-breaks/linkset RO-Crate research object: https://aogavrilov.com/publications/where-quality-breaks/ro-crate-metadata.json Authors: Alexey Gavrilov; Alan-Barsag Gazzaev; Sergey Muravyov Venue: 2026 39th Conference of Open Innovations Association (FRUCT) Published: 2026-04-28 Page last modified: 2026-07-31 OAI record datestamp: 2026-08-01 OAI identifier: oai:aogavrilov.com:where-quality-breaks OAI publication-format set: main-conference OAI set memberships: main-conference; openaire Research guide: https://aogavrilov.com/research/codec-bottleneck-diagnosis/ ### Paper in 30 seconds Problem: In compressed short-text generation, poor decoded text may come from information lost by the codec or weak latent-space generation. End-to-end scores blur those failure modes and can send optimization effort to the wrong component. Approach: The staged protocol scores originals, paired codec reconstructions, token-space MDLM outputs, and code-space diffusion outputs with one external GPT-2 evaluator. Codebook and geometry measures remain diagnostics rather than substitutes for decoded-text quality. Main result: Codec reconstruction raises median external perplexity from 15.17 to 27.36 and p95 from 25.10 to 98.91. Code-space MDLM still reduces median perplexity by 30.9% versus token-space MDLM. Why it matters: The result turns codec bottleneck diagnosis into an actionable order of work: prioritize codec improvements, then compare token-space and code-space generation, and trust latent proxy gains only when decoded text improves too. ### Abstract Compressed short-text generators can fail in two different places: the codec may discard information before generation starts, or the latent generator may produce weak codes. Without separating these failure modes, researchers can spend compute improving the wrong component. We study this problem in a controlled 64-to-16 TinyStories case study built from a hierarchical VQ-VAE-2 codec and a masked discrete diffusion generator (MDLM). We use a staged validation protocol that separates codec reconstruction fidelity, latent generation quality, and auxiliary latent diagnostics under one shared external GPT-2 scorer, while reporting complementary semantic metrics for the geometry study. In the tested configuration, codec reconstruction alone raises median external perplexity from 15.17 to 27.36 (+80.4%) and p95 from 25.10 to 98.91 (+294.1%), showing that the dominant quality loss appears before latent generation begins. Under the same scorer, code-space MDLM remains materially stronger than token-space diffusion, reducing mean, median, and p95 by 32.9%, 30.9%, and 36.6%, respectively. Geometry-aware regularization improves local latent proxies but does not improve decoded-text metrics in the available runs. The contribution is methodological rather than algorithmic: the paper presents a reusable staged diagnosis for one concrete pipeline and shows that, in this setting, codec fidelity rather than latent denoising sets the practical quality ceiling. ### Questions this paper helps answer ### Where does quality break in compressed short-text generation? In the tested TinyStories 64-to-16 pipeline, the dominant degradation appears at codec reconstruction, before latent generation begins. Median external GPT-2 perplexity increases from 15.17 for original text to 27.36 after reconstruction, while p95 increases from 25.10 to 98.91. This is a result for one configuration rather than a universal codec ranking. ### How can codec loss be separated from latent-generation loss? The staged protocol evaluates originals, paired codec reconstructions, and final generated text with one shared external scorer. The original-to-reconstruction gap estimates the codec contribution, while the reconstruction-to-generated-output comparison helps localize the additional generation-stage loss. Latent health metrics are reported separately from decoded-text evidence. ### How should discrete-latent text generation be evaluated? The paper recommends checking reconstruction fidelity before comparing generators, applying the same decoded-text scorer at every stage, and reporting central and tail statistics. It also treats codebook usage, geometry, and other latent proxies as diagnostics rather than substitutes for the quality of decoded text. ### Does code-space diffusion outperform token-space diffusion? Under the shared scorer and tested setup, code-space MDLM reduces mean, median, and p95 perplexity by 32.9%, 30.9%, and 36.6% relative to token-space MDLM. The comparison is descriptive and configuration-specific: it does not establish a universal ranking across datasets, compression ratios, codecs, or diffusion architectures. ### Do better latent geometry metrics guarantee better generated text? No. In the available matched runs, geometry-aware regularization improved local latent-space proxies but did not improve decoded-text metrics. The result supports auditing every proxy gain at the final decoded output and treating the absence of transfer as a useful negative result rather than evidence of generation improvement. ### Key results - evaluation: Original texts; n: 256; mean: 16.24; median: 15.17; p95: 25.1 - evaluation: Codec reconstructions; n: 256; mean: 37.26; median: 27.36; p95: 98.91 - evaluation: AR baseline; n: 251; mean: 30.98; median: 23.27; p95: 56.11 - evaluation: Token-space MDLM; n: 256; mean: 44.74; median: 38.42; p95: 93.6 - evaluation: Code-space MDLM; n: 256; mean: 30.01; median: 26.55; p95: 59.36 Takeaway: Most of the observed quality loss is introduced before generation; code-space diffusion still reduces median perplexity by 30.9% versus token-space diffusion. Dataset: TinyStories External result-summary mirror: https://huggingface.co/datasets/aogavrilov/where-quality-breaks-results External result-summary collection: https://huggingface.co/collections/aogavrilov/publication-result-summaries-6a6b733b3116a87befb37ec2 Sample size: 256 paired reconstruction samples; 251–256 generated samples per mode; four matched geometry settings. Uncertainty: The reported comparisons are descriptive single runs; confidence intervals and multi-seed significance estimates were not computed. Conditions: GPT-2 token sequences of length 64 compressed to 16 top-level codes with a hierarchical VQ-VAE-2; all generation modes use the shared external scorer. ### Limitations - The empirical study uses TinyStories only. - The main analysis covers one aggressive 64-to-16 compression regime. - The evaluated system combines one hierarchical VQ-VAE-2 codec family with one MDLM generator. - Comparisons are based on single runs and are descriptive rather than multi-seed statistical estimates. - External GPT-2 perplexity is a shared diagnostic, not a universal semantic-quality metric. - The conclusions should not be transferred directly to all datasets, codec architectures, or compression ratios. ### References cited by the paper - [1] Subham Sekhar Sahoo, Marianne Arriola, Yair Schiff, Aaron Gokaslan, Edgar Marroquin, Justin T Chiu, Alexander Rush, Volodymyr Kuleshov. 2024. Simple and Effective Masked Diffusion Language Models. Advances in Neural Information Processing Systems. Canonical reference: https://arxiv.org/abs/2406.07524 arXiv: https://arxiv.org/abs/2406.07524 - [2] Aaron van den Oord, Oriol Vinyals, Koray Kavukcuoglu. 2017. Neural Discrete Representation Learning. Advances in Neural Information Processing Systems. Canonical reference: https://arxiv.org/abs/1711.00937 arXiv: https://arxiv.org/abs/1711.00937 - [3] Ali Razavi, Aaron van den Oord, Oriol Vinyals. 2019. Generating Diverse High-Fidelity Images with VQ-VAE-2. Advances in Neural Information Processing Systems. Canonical reference: https://arxiv.org/abs/1906.00446 arXiv: https://arxiv.org/abs/1906.00446 - [4] Jacob Austin, Daniel D. Johnson, Jonathan Ho, Daniel Tarlow, Rianne van den Berg. 2021. Structured Denoising Diffusion Models in Discrete State-Spaces. Advances in Neural Information Processing Systems. Canonical reference: https://arxiv.org/abs/2107.03006 arXiv: https://arxiv.org/abs/2107.03006 - [5] Aaron Lou, Chenlin Meng, Stefano Ermon. 2024. Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution. Proceedings of the 41st International Conference on Machine Learning. Canonical reference: https://arxiv.org/abs/2310.16834 arXiv: https://arxiv.org/abs/2310.16834 - [6] Xiaochuang Han, Sachin Kumar, Yulia Tsvetkov. 2023. SSD-LM: Semi-autoregressive Simplex-based Diffusion Language Model for Text Generation and Modular Control. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics. Canonical reference: https://doi.org/10.18653/v1/2023.acl-long.647 DOI: https://doi.org/10.18653/v1/2023.acl-long.647 arXiv: https://arxiv.org/abs/2210.17432 - [7] Xiang Lisa Li, John Thickstun, Ishaan Gulrajani, Percy Liang, Tatsunori B. Hashimoto. 2022. Diffusion-LM Improves Controllable Text Generation. Advances in Neural Information Processing Systems. Canonical reference: https://arxiv.org/abs/2205.14217 arXiv: https://arxiv.org/abs/2205.14217 - [8] Huiwen Chang, Han Zhang, Lu Jiang, Ce Liu, William T. Freeman. 2022. MaskGIT: Masked Generative Image Transformer. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Canonical reference: https://openaccess.thecvf.com/content/CVPR2022/html/Chang_MaskGIT_Masked_Generative_Image_Transformer_CVPR_2022_paper.html arXiv: https://arxiv.org/abs/2202.04200 - [9] Marjan Ghazvininejad, Omer Levy, Yinhan Liu, Luke Zettlemoyer. 2019. Mask-Predict: Parallel Decoding of Conditional Masked Language Models. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. Canonical reference: https://doi.org/10.18653/v1/D19-1633 DOI: https://doi.org/10.18653/v1/D19-1633 arXiv: https://arxiv.org/abs/1904.09324 - [10] Emiel Hoogeboom, Didrik Nielsen, Priyank Jaini, Patrick Forré, Max Welling. 2021. Argmax Flows and Multinomial Diffusion: Learning Categorical Distributions. Advances in Neural Information Processing Systems. Canonical reference: https://arxiv.org/abs/2102.05379 arXiv: https://arxiv.org/abs/2102.05379 - [11] Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever. 2019. Language Models are Unsupervised Multitask Learners. OpenAI Technical Report. Canonical reference: https://cdn.openai.com/better-language-models/language-models.pdf - [12] Nils Reimers, Iryna Gurevych. 2019. Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. Canonical reference: https://doi.org/10.18653/v1/D19-1410 DOI: https://doi.org/10.18653/v1/D19-1410 arXiv: https://arxiv.org/abs/1908.10084 - [13] Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, Yoav Artzi. 2020. BERTScore: Evaluating Text Generation with BERT. International Conference on Learning Representations. Canonical reference: https://arxiv.org/abs/1904.09675 arXiv: https://arxiv.org/abs/1904.09675 - [14] Krishna Pillutla, Swabha Swayamdipta, Rowan Zellers, John Thickstun, Sean Welleck, Yejin Choi, Zaid Harchaoui. 2021. MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontiers. Advances in Neural Information Processing Systems. Canonical reference: https://arxiv.org/abs/2102.01454 arXiv: https://arxiv.org/abs/2102.01454 - [15] Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, Ion Stoica. 2023. Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena. Advances in Neural Information Processing Systems, Datasets and Benchmarks Track. Canonical reference: https://arxiv.org/abs/2306.05685 arXiv: https://arxiv.org/abs/2306.05685 ### Availability Public implementation: none is currently linked or claimed. Public data package: none is currently linked or claimed. BibTeX: https://aogavrilov.com/publications/where-quality-breaks/citation.bib APA formatted citation: https://aogavrilov.com/publications/where-quality-breaks/citation-apa.txt IEEE formatted citation: https://aogavrilov.com/publications/where-quality-breaks/citation-ieee.txt CSL-JSON: https://aogavrilov.com/publications/where-quality-breaks/citation.json OAI Dublin Core XML: https://aogavrilov.com/publications/where-quality-breaks/oai-dc.xml OpenAIRE Guidelines v4 XML: https://aogavrilov.com/publications/where-quality-breaks/openaire.xml MODS 3.8 XML: https://aogavrilov.com/publications/where-quality-breaks/mods.xml NISO JATS 1.4 XML metadata: https://aogavrilov.com/publications/where-quality-breaks/metadata.jats.xml RDF Turtle: https://aogavrilov.com/publications/where-quality-breaks/metadata.ttl Schema.org JSON-LD: https://aogavrilov.com/publications/where-quality-breaks/metadata.jsonld ## Inspectable Control for Structure-Preserving Software Regeneration Canonical page: https://aogavrilov.com/publications/inspectable-control/ Full-text HTML: https://aogavrilov.com/publications/inspectable-control/full-text/ Full-text Markdown: https://aogavrilov.com/publications/inspectable-control/full-text/index.md Full-text section/block index JSON: https://aogavrilov.com/publications/inspectable-control/full-text/sections.json W3C Web Annotation block collection: https://aogavrilov.com/publications/inspectable-control/full-text/annotations.jsonld Source-derived citation-context index: https://aogavrilov.com/publications/inspectable-control/full-text/citations.json Full-text NISO JATS 1.4 XML: https://aogavrilov.com/publications/inspectable-control/full-text/article.jats.xml Markdown reader representation: https://aogavrilov.com/publications/inspectable-control/index.md DOI: https://doi.org/10.1145/3803437.3807386 PDF: https://aogavrilov.com/publications/inspectable-control/paper.pdf Scholarly link set (JSON): https://aogavrilov.com/publications/inspectable-control/linkset.json Scholarly link set (HTTP Link format): https://aogavrilov.com/publications/inspectable-control/linkset RO-Crate research object: https://aogavrilov.com/publications/inspectable-control/ro-crate-metadata.json Authors: Alexey Gavrilov; Alan-Barsag Gazzaev; Mikhail Mozikov; Ilya Makarov; Sergey Muravyov Venue: Proceedings of the 34th ACM International Conference on the Foundations of Software Engineering Controlled subject: ACM CCS 2012 — Automatic programming [10011007.10011074.10011092.10011782] Controlled subject: ACM CCS 2012 — Software maintenance tools [10011007.10011006.10011073] Published: 2026-07-05 Page last modified: 2026-08-01 OAI record datestamp: 2026-08-01 OAI identifier: oai:aogavrilov.com:inspectable-control OAI publication-format set: companion-poster OAI set memberships: companion-poster; openaire Research guide: https://aogavrilov.com/research/discrete-latent-generation/ ### Paper in 30 seconds Problem: AI-assisted code modification often needs one bounded change while selected program structure remains fixed. Regenerating the entire program can disturb unrelated regions, and token-level constraints do not expose a coarse control surface. Approach: The study encodes 64-token Python functions with a hierarchical VQ-VAE, locks selected coarse discrete codes, and uses masked discrete generation for localized code regeneration in the remaining latent positions. Main result: Locking four top-level codes raises parse rate from 0.453 to 0.591 while unlocked positions still change at rate 0.936 and conditional samples remain 0.998 unique. Why it matters: The results show a measurable stability–freedom trade-off for controllable code editing and partial program regeneration. They provide early evidence for an inspectable latent control layer, not proof of semantic equivalence or functional correctness. ### Abstract Software-engineering workflows such as constrained repair, staged refinement, and structure-preserving modification require control over what changes and what remains fixed. Token-level generation is a weak control surface for these operations because it constrains local surface text rather than the coarse structural invariants that software engineering often aims to preserve. We study hierarchical discrete latents as an inspectable intermediate representation for software artifacts: a hierarchical VQ-VAE compresses a 64-token Python function into coarse and fine discrete codes, and masked discrete generation regenerates only selected positions under partial constraints. On 2,000 preprocessed Python functions, locking four top-level codes improves parse rate from 0.453 to 0.591 while preserving substantial change in unlocked positions (edit freedom, 0.936) and near-maximal sample uniqueness (diversity, 0.998). Under fixed coarse context, lower-level refinement is weaker but remains monotonic, supporting a coarse-to-fine reading of the hierarchy. Overall, these results provide early evidence for a practical control layer that supports bounded, structure-preserving software-artifact regeneration above the token level. ### Questions this paper helps answer ### How can AI edit code without rewriting everything? The paper studies partial code regeneration above the token level. A hierarchical VQ-VAE maps a short Python function to coarse and fine discrete codes; selected coarse positions are locked, and masked discrete generation changes only the remaining latent positions before decoding. This provides an explicit preservation boundary instead of regenerating the whole function. ### What methods preserve program structure during code generation? This work tests hierarchical discrete latent control. Coarse latent positions can be fixed while unlocked positions are regenerated, after which parse rate and structural proxies are measured. The evidence concerns probabilistic structural stability on short Python functions; it does not establish exact AST preservation, semantic equivalence, or functional correctness. ### Can hierarchical discrete latents provide localized control over code? In the reported 2,000-function experiment, locking four top-level codes increased parse rate from 0.453 to 0.591. At the same time, 0.936 of unlocked positions changed and conditional samples were 0.998 unique. These results are early evidence that coarse latent constraints can preserve some structure without eliminating local edit freedom or sample diversity. ### How can code generation balance structural stability and diversity? The paper evaluates stability and freedom together rather than optimizing only validity. Coarse-code locking raises syntactic validity while unlocked-position change remains high and conditional samples remain almost entirely unique. The result demonstrates a measurable stability-freedom trade-off under the tested configuration, not a universal optimum. ### How does this work relate to LLM-assisted code editing? The tested model is a hierarchical VQ-VAE with masked discrete generation, not a large language model. The control problem is nevertheless relevant to LLM-assisted editing because unnecessary changes outside a requested region are a practical concern. The paper contributes a complementary latent-space mechanism and evaluation framing, not an LLM editing benchmark. ### Key results - setting: Input (truncated); parse: 0.994; skeleton: 0.994; signature: 0.994; change: — - setting: Codec reconstruction; parse: 0.857; skeleton: 0.848; signature: 0.493; change: 0 - setting: Unconditional generation; parse: 0.453; skeleton: 0.08; signature: 0; change: 0.995 - setting: Conditional, prefix k=4; parse: 0.591; skeleton: 0.295; signature: 0.061; change: 0.936 - setting: Conditional, signature span; parse: 0.6; skeleton: 0.302; signature: 0.063; change: not reported Takeaway: Coarse latent locking improves syntactic stability without collapsing change in the editable region; the result demonstrates structural control, not guaranteed functional equivalence. Dataset: 2,000 preprocessed Python functions from a CodeParrot Clean subset External result-summary mirror: https://huggingface.co/datasets/aogavrilov/inspectable-control-results External result-summary collection: https://huggingface.co/collections/aogavrilov/publication-result-summaries-6a6b733b3116a87befb37ec2 Sample size: 2,000 preprocessed Python functions; conditional sample uniqueness is 0.998. Uncertainty: The two-page study reports point estimates without confidence intervals or multi-seed statistical analysis. Conditions: 64-token functions, argmax decoding, 16 top-level codes and 32 lower-level codes; full locking exactly recovers the codec reconstruction. ### Limitations - The study is limited to short Python functions truncated to 64 tokens. - Evaluation uses argmax decoding and syntactic or structural proxies rather than tests of functional equivalence. - Exact signature preservation remains weak. - Lower-level control is weaker than top-level control. - Latent positions are not yet aligned to semantic regions such as AST spans, signatures, or control-flow structure. - The results do not establish correctness for practical repair, refactoring, or repository-level changes. ### References cited by the paper - [1] Ali Razavi, Aaron van den Oord, Oriol Vinyals. 2019. Generating Diverse High-Fidelity Images with VQ-VAE-2. Advances in Neural Information Processing Systems. Canonical reference: https://arxiv.org/abs/1906.00446 arXiv: https://arxiv.org/abs/1906.00446 - [2] Jacob Austin, Daniel D. Johnson, Jonathan Ho, Daniel Tarlow, Rianne van den Berg. 2021. Structured Denoising Diffusion Models in Discrete State-Spaces. Advances in Neural Information Processing Systems. Canonical reference: https://arxiv.org/abs/2107.03006 arXiv: https://arxiv.org/abs/2107.03006 - [3] Subham Sekhar Sahoo, Marianne Arriola, Yair Schiff, Aaron Gokaslan, Edgar Marroquin, Justin T Chiu, Alexander Rush, Volodymyr Kuleshov. 2024. Simple and Effective Masked Diffusion Language Models. Advances in Neural Information Processing Systems. Canonical reference: https://arxiv.org/abs/2406.07524 arXiv: https://arxiv.org/abs/2406.07524 - [4] Shraddha Barke, Michael B. James, Nadia Polikarpova. 2023. Grounded Copilot: How Programmers Interact with Code-Generating Models. Proceedings of the ACM on Programming Languages. Canonical reference: https://doi.org/10.1145/3586030 DOI: https://doi.org/10.1145/3586030 arXiv: https://arxiv.org/abs/2206.15000 - [5] Fengji Zhang, Bei Chen, Yue Zhang, Jacky Keung, Jin Liu, Daoguang Zan, Yi Mao, Jian-Guang Lou, Weizhu Chen. 2023. RepoCoder: Repository-Level Code Completion Through Iterative Retrieval and Generation. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. Canonical reference: https://doi.org/10.18653/v1/2023.emnlp-main.151 DOI: https://doi.org/10.18653/v1/2023.emnlp-main.151 arXiv: https://arxiv.org/abs/2303.12570 ### Availability Public implementation: none is currently linked or claimed. Public data package: none is currently linked or claimed. BibTeX: https://aogavrilov.com/publications/inspectable-control/citation.bib APA formatted citation: https://aogavrilov.com/publications/inspectable-control/citation-apa.txt IEEE formatted citation: https://aogavrilov.com/publications/inspectable-control/citation-ieee.txt CSL-JSON: https://aogavrilov.com/publications/inspectable-control/citation.json OAI Dublin Core XML: https://aogavrilov.com/publications/inspectable-control/oai-dc.xml OpenAIRE Guidelines v4 XML: https://aogavrilov.com/publications/inspectable-control/openaire.xml MODS 3.8 XML: https://aogavrilov.com/publications/inspectable-control/mods.xml NISO JATS 1.4 XML metadata: https://aogavrilov.com/publications/inspectable-control/metadata.jats.xml RDF Turtle: https://aogavrilov.com/publications/inspectable-control/metadata.ttl Schema.org JSON-LD: https://aogavrilov.com/publications/inspectable-control/metadata.jsonld