dianqk's blog

Contributions to Rust and LLVM in 2025

2025-12-13T00:00:00+00:00

2025</p>

Rust: Merged 28 PRs</a>, reviewed 24 PRs</a></li>
LLVM: Merged 13 PRs</a></li> </ul>
2024</p>
- Rust: Merged 40 PRs</a></li>
- LLVM: Merged 20 PRs</a></li> </ul>
  I have much less spare time this year because I have a baby :p. And I'm looking for a sustainable way to contribute.</p>
  I joined the Rust compiler team (in 2024! :3).</p>
  LLVM: A performance regression in LLVM that affected Ajla and Python</h2>
  This regression has been discussed elsewhere; see lobste.rs/s/9paxz2/performance_python_3_14_tail_call</a>.</p>
  I introduced the regression due to a limit for compile time in llvm#78582</a>. Finally, I learned a resolve</a> from GCC, and then I fixed the regression in llvm#114990</a> and llvm#132536</a>.</p>
  Rust: Transforming “Clone” to “Copy”</h2>
  To me, the most interesting issue is rust#128081</a>.</p>
  The "Clone" method can be transformed to "Copy" in GVN. I have several PRs for this and am working on more.</p>
  The first key PR (rust#128299</a>) exposed variant miscompilations. Camille Gillot identified the root cause in rust#147844</a>:</p>
  We can reason with the value behind a reference because it is UB to directly assign to the underlying local while the reference is live. We allow creating new derefs, this means extending the liveness of references</em>, so we are creating UB.</p> </blockquote>
  Rust: Debuginfo in MIR Basic Blocks</h2> rust#129931</a> turns out that handling Debuginfo in MIR Basic Blocks is required. I implemented this in rust#142771</a>.</p>
  This left some stuff:</p>
  
  Better remapping locals for debuginfos; see rust#147525</a>.</li>
  A tracking issue: Figure out all debugging information representations of statements #147679</a>.</li>
  Add documentation for the change in the Rust Compiler Development Guide</a>.</li> </ul>
  Rust: 4 P-critical</h2>
  I caused 4 P-critical issues. :(</p>
  
  Miscompilation due to MatchBranchSimplification MIR pass mixing up discriminants #124150</a></li>
  Panic while compiling http-body v0.4.6 with rustc 1.83.0-nightly (7042c269c 2024-09-23) #130769</a></li>
  Panic in nightly 1.83.0 and 1.84.0 with opt-level >= 1 when unwrapping Some variant #132353</a></li>
  staticlib with lto=fat now includes compiler_builtins as llvm bitcode after rustup update #146145</a></li> </ul>
  The rust#124150</a> and rust#132353</a> are miscompilations in MIR opt. I'm investigating some translation validation tools, such as Miri, Alive2, and model checker, but I haven't made any progress. So far, I have only read Program Z3, and I have forgotten many things. Furthermore, I'm thinking about picking it up next year. :p</p>
  Other</h2>
  While reviewing PRs can be exhausting, it's also a great learning opportunity. For instance, working through PRs like rust#142707</a>, rust#143784</a>, rust#136840</a>, and rust#133832</a> taught me a great deal.</p>
  I realize that the knowledge of the LLVM backend is essential to me, since more and more issues happened in the LLVM backend. I'm not sure how to tackle these issues, but I have begun studying LLVM Code Generation: A deep dive into compiler backend development</a></em>.</p>
  MIR optimizations are still important to me. I'd like to thank Camille Gillot</a> for their help on MIR.</p>
  I'm trying to immerse myself in English, and I have stopped using LLM for Chinese-to-English translation anymore. :p</p>
  I'm also learning Japanese for fun. If you are interested in anime and manga, I recommend you read learnjapanese.moe</a>.</p>

2023 Summary

2023-10-28T00:00:00+00:00

Summary</h1>
This year marks my first continuous involvement in open-source contributions. I've fixed numerous mis-compilation, added missing optimizations, and included LTO, debugging info, built-in functions, WebAssembly, and even an issue with Apple Silicon.</p>
During this journey, I was fortunate to receive numerous invaluable help, especially from those who reviewed my pull requests. Thank everyone very much. (●´∀｀●)</strong></p>
Highlight:</p>

Resolving a long-standing issue causing debugging info corruption under LTO.</li>
Adding the crate for built-in functions to LTO in Rust, addressing another long-standing issue.</li>
Adopting the English name "Quentin Dian" for better communication.</li> </ul>
I've categorized my contributions using these emojis:</p>

💢 Mis-compilation</li>
🍻 Missing optimization</li>
ℹ️ Debugging info</li>
🦀 Rust-related</li>
🐉 LLVM-related</li>
🐦 Swift-related</li>
🍎 Apple-related</li>
⏳ Blocked by other issues</li> </ul>
LLVM</h1>
Commit history: https://github.com/llvm/llvm-project/commits?author=DianQK&since=2023-01-01&until=2023-12-31</a>.</p>
By the way, my first LLVM commit was [MachineOutliner] Fix the side effect of outlined function when the register is implicit use and implicit-def in the same instruction. D112911</a>.</p>

🍻🦀⏳ [SimplifyCFG] Check if the return instruction causes undefined behavior D144319</a> Blocked by PR69498</a></li>
🍻⏳ [SimplifyCFG] Improve the precision of PtrValueMayBeModified</code> D144563</a> Follows D144319, not blocked by PR69498</a></li>
💢🦀 [SROA] Remove UB-implying metadata when promoting speculative instruction. D148456</a> Fixes PR110256</a></li>
💢🦀 [GlobalOpt] Don't replace the aliasee if it has other references. D145293</a> Fixes PR108030</a></li>
🦀🐦ℹ️ [Verifier] definition subprograms cannot be nested within DICompositeType when enabling ODR. D152095</a></li>
🐦ℹ️ [Coroutine][DebugInfo] Update the linkage name of the declaration of coro-split functions in the debug info. D157184</a></li>
💢🦀 [TailCallElim] Remove the readonly attribute of byval. D156793</a></li>
💢🦀 [JumpThreading] Invalidate LVI after combineMetadataForCSE. #65219</a></li>
🍻🦀 [EarlyCSE] Compare GEP instructions based on offset #65875</a></li>
🍻🦀 [SimplifyCFG] Hoist common instructions on switch #67077</a></li>
💢🦀 [MemCpyOpt] Merge alias metadatas when replacing arguments #67539</a></li>
💢🦀 [LVI][CVP] Treat undef like a full range #68190</a></li>
💢 [LVI][CVP] Treat undef like a full range on abs(x, false) #68711</a></li>
💢🦀 [MemCpyOpt] Combine alias metadatas when replacing byval arguments #70580</a></li>
💢🦀🍎 [AArch64] ORRWrs is copy instruction when there's no implicit def of the X register #75184</a></li> </ol> Rust</h1> Commit history: https://github.com/rust-lang/rust/commits?author=DianQK&since=2023-01-01&until=2023-12-31</a>.</p> Apart from simple pull requests like updating test cases, my contributions mainly revolved around built-ins and LTO:</p> Add the no-builtins</code> attribute to functions when no_builtins</code> is applied at the crate level. #113716</a></li> Restore #![no_builtins]</code> crates participation in LTO. #113923</a></li> Add thinlto support to codegen, assembly and coverage tests #118036</a></li> Avoid adding builtin functions to symbols.o</code> #118568</a></li> </ol> Despite addressing many issues during this process, unfortunately, there are still some known issues</a> that need resolution. I'll submit MCP once these issues are resolved.</p> Ruslin</h1> I use Joplin for note-taking, but I don't want Joplin's Android client. Hence, I developed a compatible client: ruslin-android</a>. I apologize for the lack of recent updates. I plan to adjust my spare time next year. I'm also considering desktop development.</p> Other Projects</h1> ℹ️🍎 [swift][IRGen][DebugInfo] split method declaration and definition. #67077</a></li> ℹ️🍎 [swift][Backtracing][Linux] Support the DW_AT_specification</code> attribute. #67592 </a></li> 🦀 [wasm3] Fix the issue of incorrectly freeing the stack address when unwinding local functions. #436</a></li> 🦀 [dioxus] Fix the issue of duplicate unique ID for atoms using newtype</code>. #1112</a></li> </ul> Personal Details</h1> DianQK is just a casual Chinese pseudonym formed from the pinyin abbreviation of "靛(dian)" and "青(qing)", where "靛青" refers to indigo. It has led to confusion about how to call me. To resolve this, I asked ChatGPT for a name: Quentin Dian. So, you can call me Dian or dianqk.</p> I switched entirely from Arch Linux to NixOS, a fantastic operating system where I can manage everything with code.</p> To enhance my efficiency, I also purchased a GPD handheld console. I complete most of the code above using the GPD during my subway commutes. I assembled a machine with a 7950X processor, cutting my local build time in half. I can now complete a full compile in just a few minutes. I also bought a Mac Mini (M2) to test issues on that platform.</p> </p>
Finding LLVM bugs in rust using good-bad comparisons 2023-10-15T00:00:00+00:00 Abstract</h1> Finding problems in large projects is always complicated, and the LLVM bugs mixed in with rust is one such case. In this article, I will describe how I solved the rust unit test failure issue under stage2. I'll discuss the issues around Failing tests when rustc is compiled with 1 CGU</a> and Implementing niche checks</a> to document my process for solving this type of issue, which I hope will help solve this type of issue in the future.</p> Before We Start</h1> Since this is a summary post of my solution, I may have omitted some details of the process, so please forgive me.</li> Feel free to submit an issue</a> or PR</a> to correct errors in the article.</li> </ul> I have prepared a corresponding project for this article, see:</p> https://github.com/DianQK/rust/tree/blog/repro-1-cgu</a></li> https://github.com/DianQK/llvm-project/tree/blog/repro-1-cgu</a></li> https://github.com/DianQK/rust/tree/blog/mir-niche-checks</a></li> https://github.com/DianQK/llvm-project/tree/blog/mir-niche-checks</a></li> </ul> The PR to fix these two issues is:</p> https://github.com/llvm/llvm-project/pull/67539</a></li> https://github.com/llvm/llvm-project/pull/68190</a></li> </ul> How to reproduce the first problem</h1> First, we need to switch to a revision that can be reproduced:</p> git</span> clone https://github.com/rust-lang/rust.git </span>git</span> checkout 085acd02d4abaf2ccaf629134caa83cfe23283c8 </span></code></pre> Then we need to change config.toml</code>:</p> profile </span>= "</span>dist</span>" </span> </span>[build] </span>profiler </span>= </span>true </span> </span>[rust] </span>codegen-units </span>= </span>1 </span>optimize </span>= </span>2 </span></code></pre> We also need to understand how to use rustc-perf</a> and then build rustc with PGO optimization using the following script:</p> ./x</span> build</span> --rust-profile-generate</span>=/tmp/profiles</span> --stage</span> 2 library </span>cargo</span> run</span> --bin</span> collector bench_local</span> --include</span> serde,syn <path to stage2/bin/rustc> </span>./build/ci-llvm/bin/llvm-profdata</span> merge</span> -o</span> profiles.profdata /tmp/profiles </span>./x</span> build</span> --rust-profile-use</span>=profiles.profdata</span> --stage</span> 2 library </span></code></pre> The problem can be reproduced by compiling the following code with this version of rustc:</p> #![</span>feature</span>(inline_const)] </span> </span>fn </span>main</span>() { </span> </span>const </span>{ </span> assert!(-</span>9.223372036854776e18</span>f64 </span>as </span>i64 </span>== </span>0x8000000000000000</span>u64 </span>as </span>i64</span>); </span> } </span>} </span></code></pre> The reproduced error log is as follows:</p> error[E0080]:</span> evaluation of `</span>main::{constant#0</span>}` failed </span> </span>--</span>> ./test.rs:5:9 </span> | </span>5 </span>| </span>assert!</span>(-9.223372036854776e18f64 as i64 == 0x8000000000000000u64 as i64); </span> | </span>^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^</span> the evaluated program panicked at '</span>assertion failed: -9.223372036854776e18f64 as i64 == 0x8000000000000000u64 as i64</span>', ./test.rs:5:9 </span> | </span> = </span>note:</span> this error originates in the macro `</span>assert</span>` (in Nightly builds, run with</span> -Z</span> macro-backtrace for more info) </span> </span>note:</span> erroneous constant encountered </span> </span>--</span>> ./test.rs:4:5 </span> | </span>4 </span>| </span>/</span> const { </span>5 | | assert!(-9.223372036854776e18f64 as i64 == 0x8000000000000000u64 as i64); </span>6 | | } </span> | |</span>_____^ </span> </span>error:</span> aborting due to previous error </span> </span>For</span> more information about this error, try `</span>rustc --explain</span> E0080`. </span></code></pre> Do some preliminary analysis</h2> Use the error stack to determine a problematic crate</h2> We can use -Z treat-err-as-bug</code> to get the error stack, where the incorrectly compiled function is likely to be.</p> ›</span> ./build/host/stage2/bin/rustc ./test.rs</span> -Z</span> treat-err-as-bug </span>error[E0080]:</span> evaluation of `</span>main::{constant#0</span>}` failed </span> </span>--</span>> ./test.rs:5:9 </span> | </span>5 </span>| </span>assert!</span>(-9.223372036854776e18f64 as i64 == 0x8000000000000000u64 as i64); </span> | </span>^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^</span> the evaluated program panicked at '</span>assertion failed: -9.223372036854776e18f64 as i64 == 0x8000000000000000u64 as i64</span>', ./test.rs:5:9 </span> | </span> = </span>note:</span> this error originates in the macro `</span>assert</span>` (in Nightly builds, run with</span> -Z</span> macro-backtrace for more info) </span> </span>thread </span>'</span>rustc</span>' panicked at compiler/rustc_errors/src/lib.rs:1724:30: </span>aborting</span> due to `</span>-Z</span> treat-err-as-bug=1` </span>stack</span> backtrace: </span> </span>... </span> </span>23:</span> 0x7f32189f3bd7 - rustc_const_eval</span>[</span>7551efff2730a760</span>]</span>::const_eval::eval_queries::eval_to_const_value_raw_provider </span> </span>... </span></code></pre> By examining the stack, I think rustc_const_eval</code> is the crate to focus on, and we can do a simple verification to prove my guess. Change Cargo.toml</code> to the following:</p> [profile.release.package.rustc_const_eval] </span>codegen-units </span>= </span>16 </span></code></pre> We can see that the problem is no longer reproducing. I think rustc on stage1 compiles rustc_const_eval</code> erroneously.</p> Simplify test.rs</code></h2> After some tweaking of the reproduction use case, I found that the following code can also be reproduced:</p> #![</span>crate_type </span>= "</span>lib</span>"] </span> </span>const </span>_: </span>u32 </span>= -</span>1.1</span>f32 </span>as </span>i32 </span>as </span>u32 </span>- </span>1 </span>as </span>u32</span>; </span></code></pre> Also, we can see that the issue here behaves as if any negative floating-point number converted to a signed integer will go straight to 0.</p> Roughly identify the problem function</h2> By reading the rustc_const_eval</code> code and analyzing the call stack, I guess the issue is in float_to_float_or_int</a> and cast_from_float</a> nearby.</p> To verify this, we can use #[inline(never)]</code> to prevent partial optimization. I found that adding #[inline(never)]</code> to float_to_float_or_int</code> is still an issue while adding to cast_from_float</code> makes the test code compile correctly. I guess the issue is in float_to_float_or_int</code> and the inline function cast_from_float</code>.</p> Is it really about PGO?</h2> In fact, we can reproduce the issue directly using the stage2 version of rustc used to generate PGO:</p> ./x</span> build</span> --rust-profile-generate</span>=/tmp/profiles</span> --stage</span> 2 library </span></code></pre> Use git bisect to find which commit caused the problem</h1> While this result may not be a faulty commit, it can give us a concrete way to control the compilation of errors. We can further pinpoint the problem by tuning this commit.</p> Pick a good commit</h2> In execute git bisect, we need to find a good commit. If we can't find a good commit between major releases, I'm going to give up on git bisect because a commit that's too far back in time might not make sense. And the more commits we ignore, the more likely we are to have other unrelated issues. Here's a simple approach: LLVM's tags have a creation rule where we create a tag that raises the major version at the same time we create a new release branch, which is llvmorg-{version}-init</code>, which has a linear relationship and is very bisect-friendly. I would treat llvmorg-18-init</code> and llvmorg-17.0.1</code> as consistent code.</p> Here we have chosen the comparison version:</p> Good commit: b0daacf5</a>, which is the previous commit of llvmorg-17-init</code>, and is similar to llvmorg-16.0.0</code>. Since rust uses version numbers to adapt to LLVM API changes, we need to keep the version numbers consistent with the API.</li> Bad commit: d0b54bb5</a>.</li> </ul> Preparing the LLVM build configuration</h2> Since each step of the bisect takes a long time, I first recommend using a higher-performance computer to shorten this time.</p> Then change config.toml</code> to reduce the repetitive build time, my changes are as follows:</p> [build] </span>submodules </span>= </span>false </span> </span>[llvm] </span>download-ci-llvm </span>= </span>false </span>assertions </span>= </span>false </span>ccache </span>= "</span>sccache</span>" </span>targets </span>= "</span>X86</span>" </span>experimental-targets </span>= "" </span> </span>[target.x86_64-unknown-linux-gnu] </span># The commit of the patch will be removed after using bisect. </span>llvm-has-rust-patches </span>= </span>false </span> </span>[rust] </span>codegen-units </span>= </span>256 </span></code></pre> Also, change Cargo.toml</code>:</p> [profile.release.package.rustc_const_eval] </span>codegen-units </span>= </span>1 </span></code></pre> Reducing functions processed by PGO</h2> This reduces build time and further identifies the problem.</p> We change PGOInstrumentation.cpp#L1761</a> filtering rules, for example:</p> static bool </span>skipPGO</span>(</span>const</span> Function &</span>F</span>) { </span> </span>// ... </span> </span>if </span>(!F.</span>getName</span>().</span>contains</span>("</span>rustc_const_eval</span>")) </span> </span>return </span>true</span>; </span> </span>if </span>(!F.</span>getName</span>().</span>contains</span>("</span>float_to_float_or_int</span>")) </span> </span>return </span>true</span>; </span> </span>if </span>(!F.</span>getName</span>().</span>contains</span>("</span>cast_from_float</span>")) </span> </span>return </span>true</span>; </span> </span>// ... </span>} </span></code></pre> Execute git bisect</h2> The bisect process is a bit different from the standard LLVM project. Instead of using git bisect skip</code>, we need to manually adapt API changes when we encounter a PassWrapper.cpp</code> compilation failure. Instead of using git bisect skip</code>. Since PassWrapper.cpp</code> has to be modified so often, we can't automate the process using git bisect run</code>, but have to do it manually and check the results each time. With any luck, it takes less than 12 runs to get the results.</p> After running it for a while, I got a bisect result of 361464c0</a>. Because the LLVM is so complex, I also generally can't tell what the problem is directly from this commit, and this commit may not necessarily be a faulty commit.</p> I generally categorize bisect results into several categories:</p> It was a standalone commit that caused the mis-compilation to occur.</li> This is a standalone commit that caused a mis-compilation to occur. It was a commit with misleading information that caused a later Pass to produce a "mis" compilation (this is the first type of problem).</li> it's just a coincidence that a later mis-compiled Pass provided matching input or exposed an existing mis-compilation (this is the second type of problem).</li> </ul> At this point, however, we can't categorize the result of this one. But we can use this result to continue debugging the problem.</p> Finding the transformation of interest by changing the LLVM source code</h1> At this point, we don't know where the mis-compilation is, and we can't debug it by getting an IR directly. We still compile and run rustc to locate the problem until we have a clear conclusion.</p> Based on the bisect result, we need to find which function ended up mis-compiling after processImmutArgument</code></strong>.</p> Using code like the following can help us step-by-step to pinpoint which function is affected.</p> bool </span>MemCpyOptPass::</span>processImmutArgument</span>(CallBase &</span>CB</span>, </span>unsigned </span>ArgNo</span>) { </span> </span>// ... </span> </span>auto</span> FnName = CB.</span>getFunction</span>()-></span>getName</span>(); </span> </span>if </span>(FnName.</span>contains</span>("</span>rustc_const_eval</span>") && </span> FnName.</span>contains</span>("</span>CompileTimeInterpreter</span>") && </span> FnName.</span>contains</span>("</span>float_to_float_or_int</span>")) { </span> </span>errs</span>() << "</span>LLVMLOG: Skip </span>" << FnName << "</span>\n</span>"; </span> </span>return </span>false</span>; </span> </span>// ... </span>} </span></code></pre> Looking through the log I can find that the function impacted in MemCpyOptPass</code> is:</p> <rustc_const_eval::interpret::eval_context::InterpCx<rustc_const_eval::const_eval::machine::CompileTimeInterpreter>>::float_to_float_or_int </span></code></pre> From the log I found that this function completes multiple memcpy</code> transformations.</p> So we can continue filtering to find which transformation is causing the problem.</p> auto</span> FnName = CB.</span>getFunction</span>()-></span>getName</span>(); </span>bool</span> IsKeyFunction = FnName.</span>contains</span>("</span>rustc_const_eval</span>") && </span> FnName.</span>contains</span>("</span>CompileTimeInterpreter</span>") && </span> FnName.</span>contains</span>("</span>float_to_float_or_int</span>"); </span>if </span>(!IsKeyFunction) </span> </span>return </span>false</span>; </span>static int</span> Count = </span>0</span>; </span>Count += </span>1</span>; </span>if </span>(Count != </span>3</span>) { </span> </span>errs</span>() << "</span>LLVMLOG: Skip </span>" << Count << "</span>\n</span>"; </span> </span>return </span>false</span>; </span>} </span>errs</span>() << "</span>LLVMLOG: Use </span>" << Count << "</span>\n</span>"; </span></code></pre> When I got to this point, I began to suspect that the mis-compilation had nothing to do with PGO. At this point, I tried canceling the PGO which also reproduced the problem. My guess is that PGO was just a coincidence that exposed the mis-compilation to the runtime. But we don't yet know what type of MemCpyOptPass</code> it is, and it could be a new coincidence, or a mis-compilation, or a misdirected subsequent Pass.</p> Using -opt-bisect-limit</code></h2> We can use -opt-bisect-limit</code> to locate which pass changes instructions that cause problems at runtime.</p> There are two types of passes found using -opt-bisect-limit</code>:</p> The previous pass performed the correct transformation but did not update the metadata and other information in time, resulting in a problem with the found passes.</li> There is a mis-compilation in the found pass itself.</li> </ul> Execute -opt-bisect-limit</code> for a specific crate.</h2> Trivia: This real debugging was done by changing OptBisect.cpp</code>. However, while writing this article, I found a simpler and more efficient solution.</p> If we set -Cllvm-args=-opt-bisect-limit=-1</code> via RUSTFLAGS_NOT_BOOTSTRAP</code> directly, we will get a lot of invalid logs. We want to apply this only to rustc_const_eval</code>.</p> The nightly version of cargo</code> provides this. We need to switch to nightly first. make the following changes:</p> [build] </span>cargo </span>= "</span><path to home>/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/bin/cargo</span>" </span></code></pre> Then update Cargo.toml</code>.</p> cargo-features </span>= ["</span>profile-rustflags</span>"] </span># ... </span>[profile.release.package.rustc_const_eval] </span>rustflags </span>= [ </span> "</span>-C</span>", "</span>llvm-args=-opt-bisect-limit=-1</span>", </span>] </span></code></pre> Next, we can build using --keep-stage</code>:</p> ./x</span> build</span> --stage</span> 2 library</span> --keep-stage</span> 0</span> --keep-stage-std</span> 1 </span>2</span>> build.log </span>./build/host/stage2/bin/rustc</span> ./test.rs </span></code></pre> My final bisect result was:</p> BISECT: running pass (560953) MemCpyOptPass on _RNvMNtNtCsiODAygBxQYA_16rustc_const_eval9interpret4castINtNtB4_12eval_context8InterpCxNtNtNtB6_10const_eval7machine22CompileTimeInterpreterE21float_to_float_or_intB6_ </span>BISECT: running pass (560954) DSEPass on _RNvMNtNtCsiODAygBxQYA_16rustc_const_eval9interpret4castINtNtB4_12eval_context8InterpCxNtNtNtB6_10const_eval7machine22CompileTimeInterpreterE21float_to_float_or_intB6_ </span>BISECT: NOT running pass (560955) MoveAutoInitPass on _RNvMNtNtCsiODAygBxQYA_16rustc_const_eval9interpret4castINtNtB4_12eval_context8InterpCxNtNtNtB6_10const_eval7machine22CompileTimeInterpreterE21float_to_float_or_intB6_ </span></code></pre> It's weird to me. If I set the limit to 560953</code>, the result does not stop at MemCpyOptPass</code>. The result is a bit untrustworthy.</p> BISECT: running pass (560953) BDCEPass on _RNvXsg_NtNtCsjGTw0T6X7N4_16rustc_const_eval9interpret7operandNtB5_9ImmediateNtNtCs6sCMhXnFQZh_4core3fmt5Debug3fmtB9_ </span>BISECT: NOT running pass (560954) InstCombinePass on _RNvXsg_NtNtCsjGTw0T6X7N4_16rustc_const_eval9interpret7operandNtB5_9ImmediateNtNtCs6sCMhXnFQZh_4core3fmt5Debug3fmtB9_ </span></code></pre> However, we can verify that the problem is related to DSEPass</code> in a similar way, by changing the code to skip DSEPass</code>. Next, as with debugging MemCpyOptPass</code>, we can find out exactly which instruction DSEPass</code> transformation is causing the final mis-compilation.</p> Update: For more stable results with -opt-bisect-limit</code>, we can try -Z no-parallel-llvm</code>. Also, rustc</code> tries to perform ThinLTO by default, which can be turned off with -C lto=false</code>.</p> Getting IR to prepare for LLVM debugging</h1> This part is related to LLVM debugging, and I don't have any good experience to share yet.</p> But the two important points are:</p> We know exactly which transformation affects the result in MemCpyOptPass</code>.</li> We also know which transformation in DSEPass</code> caused the final mis-compilation.</li> </ul> With these two transformations, we can use opt</code> to debug IR. We don't need to use rustc</code> to compile as often!</p> The recommended way to get the IR is by changing Cargo.toml</code>:</p> [profile.release.package.rustc_const_eval] </span>codegen-units </span>= </span>1 </span>rustflags </span>= [ </span> "</span>-C</span>", "</span>save-temps</span>", </span>] </span></code></pre> Then find *.no-opt.bc</code> to debug.</p> Of course, during this debugging process, we need to know the details of the logic of the two transformations of MemCpyOptPass</code> and DSEPass</code>. Here we'll learn that this is a transformation via alias analysis, which ultimately pinpoints the fact that MemCpyOptPass</code> doesn't update the corresponding alias information when replacing the value used by the instruction.</p> Next question - mir niche checks</h1> Although this problem is easier to reproduce than the previous one, it will be more troublesome to solve. First, let's switch to cf8d85e4</a>.</p> This can be reproduced by running a test with --stage 2</code>:</p> ./x</span> test tests/ui</span> --stage</span> 2 </span></code></pre> config.toml</code> reference:</p> profile </span>= "</span>codegen</span>" </span> </span>[llvm] </span>assertions </span>= </span>false </span>enable-warnings </span>= </span>false </span>download-ci-llvm </span>= </span>false </span>ccache </span>= "</span>sccache</span>" </span>targets </span>= "</span>X86</span>" </span>experimental-targets </span>= "" </span>link-shared </span>= </span>true </span>use-linker </span>= "</span>lld</span>" </span>optimize </span>= </span>true </span>release-debuginfo </span>= </span>true </span> </span>[rust] </span>debug </span>= </span>false </span>incremental </span>= </span>false </span>optimize </span>= </span>3 </span>debug-logging </span>= </span>false </span>deny-warnings </span>= </span>false </span>codegen-backends </span>= ["</span>llvm</span>"] </span>use-lld </span>= </span>true </span>lto </span>= "</span>off</span>" </span>debug-assertions </span>= </span>true </span>debug-assertions-std </span>= </span>false </span></code></pre> The error log is as follows:</p> thread 'rustc' panicked at compiler/rustc_mir_build/src/thir/pattern/deconstruct_pat.rs:560:22: </span>occupied niche: found 0x7f7700000000 but must be in 0x0..=0x2 in type std::option::Option<thir::pattern::deconstruct_pat::SliceKind> at offset 0 with type Int(I64, false) </span>stack backtrace: </span> ... </span> 13: 0x7f77e92ade54 - <rustc_mir_build[48ebee3fe6c2e2b3]::thir::pattern::deconstruct_pat::Constructor>::split::<core[8d828210e7f791ba]::iter::adapters::map::Map<core[8d828210e7f791ba]::iter::adapters::map::Map<core[8d828210e7f791ba]::slice::iter::Iter<rustc_mir_build[48ebee3fe6c2e2b3]::thir::pattern::usefulness::PatStack>, <rustc_mir_build[48ebee3fe6c2e2b3]::thir::pattern::usefulness::Matrix>::heads::{closure#0}>, <rustc_mir_build[48ebee3fe6c2e2b3]::thir::pattern::deconstruct_pat::DeconstructedPat>::ctor>> </span> ... </span></code></pre> I found that if I set codegen-units=1</code>, the panic disappears. So we can use codegen-units</code> to determine which crate is impacted. I guessed rustc_mir_build</code> based on the stack and tried toggling codegen-units=1</code> to verify. Since this issue occurred at roughly the same point in time as the previous one, we used the same LLVM commit at the start of the bisect. Unfortunately, with the optimize=3</code> set, we can't find a good commit in LLVM 16. But I also tried setting optimize=2</code> and can find LLVM 16 is a good commit.</p> Tricky git bisect</h2> During the bisect process, I get an unexpected result:</p> ---- [ui] tests/ui/issue-11881.rs stdout ---- </span> </span>error: test compilation failed although it shouldn't! </span>status: exit status: 1 </span>command: RUSTC_ICE="0" "/home/dianqk/rust-workspace/rust/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "/home/dianqk/rust-workspace/rust/tests/ui/issue-11881.rs" "-Zthreads=1" "-Zsimulate-remapped-rust-src-base=/rustc/FAKE_PREFIX" "-Ztranslate-remapped-path-to-local-path=no" "-Z" "ignore-directory-in-diagnostics-source-blocks=/home/dianqk/.cargo" "--sysroot" "/home/dianqk/rust-workspace/rust/build/x86_64-unknown-linux-gnu/stage2" "--target=x86_64-unknown-linux-gnu" "-O" "--error-format" "json" "--json" "future-incompat" "-Ccodegen-units=1" "-Zui-testing" "-Zdeduplicate-diagnostics=no" "-Zwrite-long-types-to-disk=no" "-Cstrip=debuginfo" "-C" "prefer-dynamic" "-o" "/home/dianqk/rust-workspace/rust/build/x86_64-unknown-linux-gnu/test/ui/issue-11881/a" "-A" "internal_features" "-Crpath" "-Cdebuginfo=0" "-Lnative=/home/dianqk/rust-workspace/rust/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "-Clink-arg=-fuse-ld=lld" "-Clink-arg=-Wl,--threads=1" "-L" "/home/dianqk/rust-workspace/rust/build/x86_64-unknown-linux-gnu/test/ui/issue-11881/auxiliary" </span>stdout: none </span>--- stderr ------------------------------- </span>error: unexpected token: `&` </span> --> /home/dianqk/rust-workspace/rust/tests/ui/issue-11881.rs:18:25 </span> | </span>LL | fn encode(&self, s: &mut S) -> Result<(), S::Error>; </span> | ^ unexpected token after this </span> </span>error: unexpected token: `&` </span> --> /home/dianqk/rust-workspace/rust/tests/ui/issue-11881.rs:33:23 </span> | </span>LL | fn fmt(&self, _f: &mut fmt::Formatter<'_>) -> fmt::Result { </span> | ^ unexpected token after this </span></code></pre> For this result, we can't use good/bad to execute bisect, which leads bisect to the wrong result.</p> Unfortunately, even with a skip, we can't get a bisect result. When bisect is in f7deb69f2...7c78cb4b</a> internally, it is possible to get this unexpected error. If it's an earlier commit, it's good, and if it's a later commit, it's bad. This is because new issues were introduced with the change in the semantics of nonnull</code> and so on, and we overlooked some issue fixes during bisect that led to the exposure of a new issue.</p> The history of this commit is as follows:</p> bad </span>Revert "[SimplifyCFG][LICM] Preserve nonnull, range and align metadat… 7c78cb4 </span>skip </span>[SimplifyCFG][LICM] Preserve nonnull, range and align metadata when s… 78b1fbc </span>good </span></code></pre> Since we ignored some commits that caused a new problem, bisect had no results. Luckily, this time the problem was exceptional and we were able to drop 78b1fbc</code> and 7c78cb4</code> using rebase.</p> The final bisect result is [AggressiveInstCombine] Enable also for -O2</a>. I also found the critical transformation in AggressiveInstCombine</code> by changing the code, but I can't see anything wrong with it from the code or IR. It could be that I'm missing something, or it could be that this is just a lucky opportunity for a triggering. We need to keep this point of suspicion in mind and continue troubleshooting.</p> codegen-units=256</code> & -opt-bisect-limit=n</code></h1> This time we have no way to use -opt-bisect-limit</code> because codegen-units</code> is not equal to 1. It doesn't make sense to bisect multiple IRs at the same time. We need to modify the rust code to support selecting a specific CGU to bisect under multiple CGUs.</p> First use -C save-temps</code> to find the corresponding IR. See bfd759b7</a> for the details.</p> Trivia: Here I also tried to find the one transformation in AggressiveInstCombine that had an effect. It was a headache that adding -C save-temps</code> changed the symbol names, which made me look again for the associated symbols.</p> I wrote a simple script to find this IR:</p> for</span> bitcode </span>in</span> build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps/rustc_mir_build-*-cgu.*.rcgu.no-opt.bc; </span>do </span> </span>if </span>llvm-nm -U </span>$</span>bitcode </span>| </span>grep -q </span>"</span>example</span>"; </span>then </span> </span>echo </span>$</span>bitcode </span> </span>fi </span>done </span></code></pre> I viewed the implementation of OptBisect</code>, and although the -opt-bisect-limit</code> option is global, we can replace an empty OptBisect</code> with a separate one for the CGU. The full modification is at a9f62a4a</a>.</p> Some key changes are referenced below:</p> struct </span>RunAllOptPassGate : </span>public </span>OptPassGate </span>{ </span> </span>bool </span>shouldRunPass</span>(</span>const</span> StringRef </span>PassName</span>, StringRef </span>IRDescription</span>) </span>override </span>{ </span> </span>return </span>true</span>; </span> } </span> </span>bool </span>isEnabled</span>() </span>const override </span>{ </span>return </span>true</span>; } </span>}; </span> </span>static</span> RunAllOptPassGate &</span>getRunAllOptPassGate</span>() { </span> </span>static</span> RunAllOptPassGate RunAllOptPassGate; </span> </span>return</span> RunAllOptPassGate; </span>} </span> </span>extern </span>"</span>C</span>" </span>void </span>LLVMRustContextSetSetRunAllOptPassGate</span>(LLVMContextRef </span>C</span>) { </span> </span>unwrap</span>(C)-></span>setOptPassGate</span>(</span>getRunAllOptPassGate</span>()); </span>} </span></code></pre> I also added a command line argument -Z llvm-opt-bisect-limit-cgu</code> so that I could bisect using the following script:</p> export </span>RUSTFLAGS_NOT_BOOTSTRAP</span>="</span>-C llvm-args=-opt-bisect-limit=-1 -Z llvm-opt-bisect-limit-cgu=rustc_mir_build.63d28fcded2a05ed-cgu.007</span>" </span>./x</span> build</span> --stage</span> 2 library</span> --keep-stage</span> 0</span> --keep-stage-std</span> 1 </span>2</span>> build.log </span>./build/host/stage2/bin/rustc</span> ./tests/ui/consts/const_prop_slice_pat_ice.rs </span></code></pre> I also wrote a simple automated bisect script:</p> function </span>iterate</span>() { </span> </span>local </span>good</span>=`</span>sed -n </span>'</span>1p</span>' bisect_result` </span> </span>local </span>bad</span>=`</span>sed -n </span>'</span>2p</span>' bisect_result` </span> </span>local </span>result</span>=$</span>((bad </span>-</span> good)) </span> </span>echo </span>"</span>good: </span>$</span>good</span>, bad: </span>$</span>bad</span>" </span> </span>if </span>[ </span>$</span>result -eq</span> 1 </span>]</span>; </span>then </span> </span>echo </span>"</span>done</span>" </span> </span>exit</span> 0 </span> </span>else </span> </span>local </span>next</span>=$</span>((good </span>+ </span>(result </span>/ </span>2</span>))) </span> </span>echo </span>$</span>next </span> </span>bash</span> bisect.sh $</span>next </span> </span>exit_code</span>=$</span>? </span> </span>case </span>$</span>exit_code </span>in </span> 0</span>) </span> </span>good</span>=$</span>next </span> ;; </span> 1</span>) </span> </span>bad</span>=$</span>next </span> ;; </span> *</span>) </span> </span>echo </span>"</span>failed: </span>$</span>exit_code</span>" </span> </span>exit</span> 1 </span> ;; </span> </span>esac </span> </span>fi </span> </span>echo </span>$</span>good </span>> bisect_result </span> </span>echo </span>$</span>bad </span>>> bisect_result </span>} </span> </span>while </span>true</span>; </span>do </span> </span>iterate </span> </span>sleep</span> 1 </span>done </span></code></pre> But I got a weird result:</p> ISECT: running pass (13444) InlinerPass on (symbol) </span>BISECT: NOT running pass (13445) PostOrderFunctionAttrsPass on (symbol) </span></code></pre> I don't think inline is directly related to this mis-compilation. I simply changed OptBisect.cpp</code> to skip InlinerPass</code>:</p> bool </span>OptBisect::</span>shouldRunPass</span>(</span>const</span> StringRef </span>PassName</span>, </span> StringRef </span>IRDescription</span>) { </span> </span>if </span>(PassName == "</span>InlinerPass</span>") { </span> </span>printPassMessage</span>(PassName, -</span>1</span>, IRDescription, </span>true</span>); </span> </span>return </span>true</span>; </span> } </span> </span>// ... </span>} </span></code></pre> Eventually, I got:</p> BISECT: running pass (10040) CorrelatedValuePropagationPass on symbol </span>BISECT: NOT running pass (10041) SimplifyCFGPass on symbol </span></code></pre> To verify that CorrelatedValuePropagationPass</code> is related, it is still done by removing unrelated code, see a08f2c14</a>. I also added a line of logging for simple validation:</p> LLVMLOG: Delete %102 = and i64 %101, 4294967295 -> and i64 %101, 0xffffffff </span>... </span>occupied niche: found 0x7fba00000001 but must be in 0x0..=0x2 in type std::option::Option<thir::pattern::deconstruct_pat::SliceKind> at offset 0 with type Int(I64, false) </span>0x7fba00000001 & 0xffffffff = 0x1 </span></code></pre> I'm pretty sure this is what we're looking for! This correlates very well with rustc's panic log. Eventually, I realized that %101</code> could get undef</code> results under certain control flows. In this case, we should not remove %102</code>.</p> Here I am curious why it is related to AggressiveInstCombine</code>, if we remove this Pass, the instruction to delete becomes %123 = and i64 %122, 72057594037927935(0xffffffffffffff)</code>. We still can't delete the instruction, it's just that this huge value makes it hard for the program to encounter during runtime.</p> Summary</h1> What have we compared?</p> Finding the affected crate by comparing different codegen-units</code>.</li> Compare results by adding [inline(never)]</code> to find the affected functions.</li> git bisect to find out which commit affected the result.</li> by changing the code to find out which transformations were affected.</li> Finding related passes through -opt-bisect-limit</code> results.</li> </ul> I'm getting closer to the truth with these approaches. By the way, luck is also important, I don't keep records of my missteps :].</p> In these methods, I have also introduced some specific methods:</p> git bisect can find results by removing some commits when it doesn't (this may be a case-specific trick)</li> cargo provides the ability to apply rustc parameters to a given crate</li> Changing the rustc code to apply opt-bisect-limit</code> to a specific CGU should make this useful for large projects.</li> </ul> The generalized process for solving this type of problem is:</p> first find a way to shorten the time to single replication, too long debugging time is annoying</li> the key objective is to find the key transformations that are causing the mis-compilation by comparing the above methods</strong>.</li> extract the IR and locate the problem based on the crucial transformations</li> fix the problem</li> </ol> I don't have any experience with step 3 yet, but I would use llvm-extract</code> and llvm-reduce</code> to reduce the amount of IRs I get, which would be helpful for debugging. I would also use -opt-bisect-limit</code> to extract the IR for intermediate processes and manually remove some functions or instructions to debug the problem.</p> I don't have a clear idea of how to submit a proper fix yet, and currently, I rarely manage to get PRs for LGTM without modifications. l need to learn and practice more ;).</p> Refer</h1> https://rustc-dev-guide.rust-lang.org/compiler-debugging.html</a></li> https://rustc-dev-guide.rust-lang.org/backend/debugging.html</a></li> https://llvm.org/docs/GitBisecting.html</a></li> https://doc.rust-lang.org/cargo/reference/unstable.html#profile-rustflags-option</a></li> https://www.llvm.org/docs/AliasAnalysis.html</a></li> https://doc.rust-lang.org/rustc/codegen-options/index.html#lto</a></li> </ul>