# Alexey Gavrilov — full research corpus This text is generated from the same validated publication records as the canonical HTML pages. The HTML page and DOI remain authoritative when a field conflicts with an external index. Author identity key: https://orcid.org/0009-0006-3147-5430. Verified scholarly profiles: ORCID https://orcid.org/0009-0006-3147-5430; Google Scholar https://scholar.google.com/citations?hl=en&user=KERhD0wAAAAJ; OpenAlex https://openalex.org/A5135991487; Semantic Scholar https://www.semanticscholar.org/author/2434470286; Scopus https://www.scopus.com/inward/authorDetails.url?authorID=60676475100&partnerID=MN8TOARS. Do not merge records solely by the name Alexey Gavrilov. ## Where Quality Breaks in Compressed Short-Text Generation: Staged Bottleneck Localization Canonical page: https://aogavrilov.com/publications/where-quality-breaks/ DOI: https://doi.org/10.23919/FRUCT70069.2026.11506553 PDF: https://aogavrilov.com/publications/where-quality-breaks/paper.pdf Scholarly link set: https://aogavrilov.com/publications/where-quality-breaks/linkset.json Authors: Alexey Gavrilov; Alan-Barsag Gazzaev; Sergey Muravyov Venue: 2026 39th Conference of Open Innovations Association (FRUCT) Published: 2026-04-28 Page last modified: 2026-07-29 Research guide: https://aogavrilov.com/projects/codec-bottleneck-diagnosis/ ### Paper in 30 seconds Problem: In compressed short-text generation, poor decoded text may come from information lost by the codec or weak latent-space generation. End-to-end scores blur those failure modes and can send optimization effort to the wrong component. Approach: The staged protocol scores originals, paired codec reconstructions, token-space MDLM outputs, and code-space diffusion outputs with one external GPT-2 evaluator. Codebook and geometry measures remain diagnostics rather than substitutes for decoded-text quality. Main result: Codec reconstruction raises median external perplexity from 15.17 to 27.36 and p95 from 25.10 to 98.91. Code-space MDLM still reduces median perplexity by 30.9% versus token-space MDLM. Why it matters: The result turns codec bottleneck diagnosis into an actionable order of work: prioritize codec improvements, then compare token-space and code-space generation, and trust latent proxy gains only when decoded text improves too. ### Abstract Compressed short-text generators can fail in two different places: the codec may discard information before generation starts, or the latent generator may produce weak codes. Without separating these failure modes, researchers can spend compute improving the wrong component. We study this problem in a controlled 64-to-16 TinyStories case study built from a hierarchical VQ-VAE-2 codec and a masked discrete diffusion generator (MDLM). We use a staged validation protocol that separates codec reconstruction fidelity, latent generation quality, and auxiliary latent diagnostics under one shared external GPT-2 scorer, while reporting complementary semantic metrics for the geometry study. In the tested configuration, codec reconstruction alone raises median external perplexity from 15.17 to 27.36 (+80.4%) and p95 from 25.10 to 98.91 (+294.1%), showing that the dominant quality loss appears before latent generation begins. Under the same scorer, code-space MDLM remains materially stronger than token-space diffusion, reducing mean, median, and p95 by 32.9%, 30.9%, and 36.6%, respectively. Geometry-aware regularization improves local latent proxies but does not improve decoded-text metrics in the available runs. The contribution is methodological rather than algorithmic: the paper presents a reusable staged diagnosis for one concrete pipeline and shows that, in this setting, codec fidelity rather than latent denoising sets the practical quality ceiling. ### Questions this paper helps answer ### Where does quality break in compressed short-text generation? In the tested TinyStories 64-to-16 pipeline, the dominant degradation appears at codec reconstruction, before latent generation begins. Median external GPT-2 perplexity increases from 15.17 for original text to 27.36 after reconstruction, while p95 increases from 25.10 to 98.91. This is a result for one configuration rather than a universal codec ranking. ### How can codec loss be separated from latent-generation loss? The staged protocol evaluates originals, paired codec reconstructions, and final generated text with one shared external scorer. The original-to-reconstruction gap estimates the codec contribution, while the reconstruction-to-generated-output comparison helps localize the additional generation-stage loss. Latent health metrics are reported separately from decoded-text evidence. ### How should discrete-latent text generation be evaluated? The paper recommends checking reconstruction fidelity before comparing generators, applying the same decoded-text scorer at every stage, and reporting central and tail statistics. It also treats codebook usage, geometry, and other latent proxies as diagnostics rather than substitutes for the quality of decoded text. ### Does code-space diffusion outperform token-space diffusion? Under the shared scorer and tested setup, code-space MDLM reduces mean, median, and p95 perplexity by 32.9%, 30.9%, and 36.6% relative to token-space MDLM. The comparison is descriptive and configuration-specific: it does not establish a universal ranking across datasets, compression ratios, codecs, or diffusion architectures. ### Do better latent geometry metrics guarantee better generated text? No. In the available matched runs, geometry-aware regularization improved local latent-space proxies but did not improve decoded-text metrics. The result supports auditing every proxy gain at the final decoded output and treating the absence of transfer as a useful negative result rather than evidence of generation improvement. ### Key results - evaluation: Original texts; n: 256; mean: 16.24; median: 15.17; p95: 25.1 - evaluation: Codec reconstructions; n: 256; mean: 37.26; median: 27.36; p95: 98.91 - evaluation: AR baseline; n: 251; mean: 30.98; median: 23.27; p95: 56.11 - evaluation: Token-space MDLM; n: 256; mean: 44.74; median: 38.42; p95: 93.6 - evaluation: Code-space MDLM; n: 256; mean: 30.01; median: 26.55; p95: 59.36 Takeaway: Most of the observed quality loss is introduced before generation; code-space diffusion still reduces median perplexity by 30.9% versus token-space diffusion. Dataset: TinyStories Sample size: 256 paired reconstruction samples; 251–256 generated samples per mode; four matched geometry settings. Uncertainty: The reported comparisons are descriptive single runs; confidence intervals and multi-seed significance estimates were not computed. Conditions: GPT-2 token sequences of length 64 compressed to 16 top-level codes with a hierarchical VQ-VAE-2; all generation modes use the shared external scorer. ### Limitations - The empirical study uses TinyStories only. - The main analysis covers one aggressive 64-to-16 compression regime. - The evaluated system combines one hierarchical VQ-VAE-2 codec family with one MDLM generator. - Comparisons are based on single runs and are descriptive rather than multi-seed statistical estimates. - External GPT-2 perplexity is a shared diagnostic, not a universal semantic-quality metric. - The conclusions should not be transferred directly to all datasets, codec architectures, or compression ratios. ### Availability Public implementation: none is currently linked or claimed. Public data package: none is currently linked or claimed. BibTeX: https://aogavrilov.com/publications/where-quality-breaks/citation.bib CSL-JSON: https://aogavrilov.com/publications/where-quality-breaks/citation.json ## Inspectable Control for Structure-Preserving Software Regeneration Canonical page: https://aogavrilov.com/publications/inspectable-control/ DOI: https://doi.org/10.1145/3803437.3807386 PDF: https://aogavrilov.com/publications/inspectable-control/paper.pdf Scholarly link set: https://aogavrilov.com/publications/inspectable-control/linkset.json Authors: Alexey Gavrilov; Alan-Barsag Gazzaev; Mikhail Mozikov; Ilya Makarov; Sergey Muravyov Venue: Proceedings of the 34th ACM International Conference on the Foundations of Software Engineering Published: 2026-07-05 Page last modified: 2026-07-29 Research guide: https://aogavrilov.com/projects/discrete-latent-generation/ ### Paper in 30 seconds Problem: AI-assisted code modification often needs one bounded change while selected program structure remains fixed. Regenerating the entire program can disturb unrelated regions, and token-level constraints do not expose a coarse control surface. Approach: The study encodes 64-token Python functions with a hierarchical VQ-VAE, locks selected coarse discrete codes, and uses masked discrete generation for localized code regeneration in the remaining latent positions. Main result: Locking four top-level codes raises parse rate from 0.453 to 0.591 while unlocked positions still change at rate 0.936 and conditional samples remain 0.998 unique. Why it matters: The results show a measurable stability–freedom trade-off for controllable code editing and partial program regeneration. They provide early evidence for an inspectable latent control layer, not proof of semantic equivalence or functional correctness. ### Abstract Software-engineering workflows such as constrained repair, staged refinement, and structure-preserving modification require control over what changes and what remains fixed. Token-level generation is a weak control surface for these operations because it constrains local surface text rather than the coarse structural invariants that software engineering often aims to preserve. We study hierarchical discrete latents as an inspectable intermediate representation for software artifacts: a hierarchical VQ-VAE compresses a 64-token Python function into coarse and fine discrete codes, and masked discrete generation regenerates only selected positions under partial constraints. On 2,000 preprocessed Python functions, locking four top-level codes improves parse rate from 0.453 to 0.591 while preserving substantial change in unlocked positions (edit freedom, 0.936) and near-maximal sample uniqueness (diversity, 0.998). Under fixed coarse context, lower-level refinement is weaker but remains monotonic, supporting a coarse-to-fine reading of the hierarchy. Overall, these results provide early evidence for a practical control layer that supports bounded, structure-preserving software-artifact regeneration above the token level. ### Questions this paper helps answer ### How can AI edit code without rewriting everything? The paper studies partial code regeneration above the token level. A hierarchical VQ-VAE maps a short Python function to coarse and fine discrete codes; selected coarse positions are locked, and masked discrete generation changes only the remaining latent positions before decoding. This provides an explicit preservation boundary instead of regenerating the whole function. ### What methods preserve program structure during code generation? This work tests hierarchical discrete latent control. Coarse latent positions can be fixed while unlocked positions are regenerated, after which parse rate and structural proxies are measured. The evidence concerns probabilistic structural stability on short Python functions; it does not establish exact AST preservation, semantic equivalence, or functional correctness. ### Can hierarchical discrete latents provide localized control over code? In the reported 2,000-function experiment, locking four top-level codes increased parse rate from 0.453 to 0.591. At the same time, 0.936 of unlocked positions changed and conditional samples were 0.998 unique. These results are early evidence that coarse latent constraints can preserve some structure without eliminating local edit freedom or sample diversity. ### How can code generation balance structural stability and diversity? The paper evaluates stability and freedom together rather than optimizing only validity. Coarse-code locking raises syntactic validity while unlocked-position change remains high and conditional samples remain almost entirely unique. The result demonstrates a measurable stability-freedom trade-off under the tested configuration, not a universal optimum. ### How does this work relate to LLM-assisted code editing? The tested model is a hierarchical VQ-VAE with masked discrete generation, not a large language model. The control problem is nevertheless relevant to LLM-assisted editing because unnecessary changes outside a requested region are a practical concern. The paper contributes a complementary latent-space mechanism and evaluation framing, not an LLM editing benchmark. ### Key results - setting: Input (truncated); parse: 0.994; skeleton: 0.994; signature: 0.994; change: — - setting: Codec reconstruction; parse: 0.857; skeleton: 0.848; signature: 0.493; change: 0 - setting: Unconditional generation; parse: 0.453; skeleton: 0.08; signature: 0; change: 0.995 - setting: Conditional, prefix k=4; parse: 0.591; skeleton: 0.295; signature: 0.061; change: 0.936 - setting: Conditional, signature span; parse: 0.6; skeleton: 0.302; signature: 0.063; change: not reported Takeaway: Coarse latent locking improves syntactic stability without collapsing change in the editable region; the result demonstrates structural control, not guaranteed functional equivalence. Dataset: 2,000 preprocessed Python functions from a CodeParrot Clean subset Sample size: 2,000 preprocessed Python functions; conditional sample uniqueness is 0.998. Uncertainty: The two-page study reports point estimates without confidence intervals or multi-seed statistical analysis. Conditions: 64-token functions, argmax decoding, 16 top-level codes and 32 lower-level codes; full locking exactly recovers the codec reconstruction. ### Limitations - The study is limited to short Python functions truncated to 64 tokens. - Evaluation uses argmax decoding and syntactic or structural proxies rather than tests of functional equivalence. - Exact signature preservation remains weak. - Lower-level control is weaker than top-level control. - Latent positions are not yet aligned to semantic regions such as AST spans, signatures, or control-flow structure. - The results do not establish correctness for practical repair, refactoring, or repository-level changes. ### Availability Public implementation: none is currently linked or claimed. Public data package: none is currently linked or claimed. BibTeX: https://aogavrilov.com/publications/inspectable-control/citation.bib CSL-JSON: https://aogavrilov.com/publications/inspectable-control/citation.json