-
Notifications
You must be signed in to change notification settings - Fork 1.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
internal: port rust-analyzer to new Salsa #18964
internal: port rust-analyzer to new Salsa #18964
Conversation
e2767ec
to
3a7fd79
Compare
Chayim fixed the test failures: this was due us mixing and matching Salsa interning with rust-analyzer's own, home-grown interning. He put it succinctly here:
As a short-term fix, he's required interned values to implement |
c85177e
to
c762e4e
Compare
contains a typo in the link |
Regarding |
c762e4e
to
54ef4ea
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(partial review)
#[cfg(feature = "ra-salsa")] | ||
use ra_salsa::InternId; | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we re-introduce this cfg'ing for the new salsa? The point of that is that the proc-macro server has no need to depend on salsa
@Veykril Regarding error[E0277]: the type `(dyn salsa::accumulator::accumulated::AnyAccumulated + 'static)` may not be safely transferred across an unwind boundary
--> crates/rust-analyzer/src/cli/rustc_tests.rs:163:60
|
163 | let res = std::panic::catch_unwind(move || {
| ___________________________________------------------------_^
| | |
| | required by a bound introduced by this call
164 | | analysis.full_diagnostics(
165 | | diagnostic_config,
166 | | ide::AssistResolveStrategy::None,
... |
169 | | });
| |_________________________^ `(dyn salsa::accumulator::accumulated::AnyAccumulated + 'static)` may not be safely transferred across an unwind boundary
|
= help: the trait `UnwindSafe` is not implemented for `(dyn salsa::accumulator::accumulated::AnyAccumulated + 'static)`
= note: required for `std::ptr::Unique<(dyn salsa::accumulator::accumulated::AnyAccumulated + 'static)>` to implement `UnwindSafe`
note: required because it appears within the type `std::boxed::Box<(dyn salsa::accumulator::accumulated::AnyAccumulated + 'static)>`
--> /Users/dbarsky/.rustup/toolchains/nightly-aarch64-apple-darwin/lib/rustlib/src/rust/library/alloc/src/boxed.rs:231:12
|
231 | pub struct Box<
| ^^^
= note: required for `HashMap<IngredientIndex, std::boxed::Box<(dyn salsa::accumulator::accumulated::AnyAccumulated + 'static)>, FxBuildHasher>` to implement `UnwindSafe`
note: required because it appears within the type `salsa::accumulator::accumulated_map::AccumulatedMap`
--> /Users/dbarsky/.cargo/git/checkouts/salsa-deab236162935b01/592552b/src/accumulator/accumulated_map.rs:10:12
|
10 | pub struct AccumulatedMap {
| ^^^^^^^^^^^^^^
note: required because it appears within the type `salsa::active_query::ActiveQuery`
--> /Users/dbarsky/.cargo/git/checkouts/salsa-deab236162935b01/592552b/src/active_query.rs:17:19
|
17 | pub(crate) struct ActiveQuery {
| ^^^^^^^^^^^
note: required because it appears within the type `std::marker::PhantomData<salsa::active_query::ActiveQuery>`
--> /Users/dbarsky/.rustup/toolchains/nightly-aarch64-apple-darwin/lib/rustlib/src/rust/library/core/src/marker.rs:757:12
|
757 | pub struct PhantomData<T: ?Sized>;
| ^^^^^^^^^^^
note: required because it appears within the type `alloc::raw_vec::RawVec<salsa::active_query::ActiveQuery>`
--> /Users/dbarsky/.rustup/toolchains/nightly-aarch64-apple-darwin/lib/rustlib/src/rust/library/alloc/src/raw_vec.rs:70:19
|
70 | pub(crate) struct RawVec<T, A: Allocator = Global> {
| ^^^^^^
note: required because it appears within the type `Vec<salsa::active_query::ActiveQuery>`
--> /Users/dbarsky/.rustup/toolchains/nightly-aarch64-apple-darwin/lib/rustlib/src/rust/library/alloc/src/vec/mod.rs:397:12
|
397 | pub struct Vec<T, #[unstable(feature = "allocator_api", issue = "32838")] A: Allocator = Global> {
| ^^^
note: required because it appears within the type `std::cell::UnsafeCell<Vec<salsa::active_query::ActiveQuery>>`
--> /Users/dbarsky/.rustup/toolchains/nightly-aarch64-apple-darwin/lib/rustlib/src/rust/library/core/src/cell.rs:2076:12
|
2076 | pub struct UnsafeCell<T: ?Sized> {
| ^^^^^^^^^^
note: required because it appears within the type `RefCell<Vec<salsa::active_query::ActiveQuery>>`
--> /Users/dbarsky/.rustup/toolchains/nightly-aarch64-apple-darwin/lib/rustlib/src/rust/library/core/src/cell.rs:730:12
|
730 | pub struct RefCell<T: ?Sized> {
| ^^^^^^^
note: required because it appears within the type `ZalsaLocal`
--> /Users/dbarsky/.cargo/git/checkouts/salsa-deab236162935b01/592552b/src/zalsa_local.rs:30:12
|
30 | pub struct ZalsaLocal {
| ^^^^^^^^^^
note: required because it appears within the type `Storage<RootDatabase>`
--> /Users/dbarsky/.cargo/git/checkouts/salsa-deab236162935b01/592552b/src/storage.rs:25:12
|
25 | pub struct Storage<Db: Database> {
| ^^^^^^^
note: required because it appears within the type `std::mem::ManuallyDrop<Storage<RootDatabase>>`
--> /Users/dbarsky/.rustup/toolchains/nightly-aarch64-apple-darwin/lib/rustlib/src/rust/library/core/src/mem/manually_drop.rs:157:12
|
157 | pub struct ManuallyDrop<T: ?Sized> {
| ^^^^^^^^^^^^
note: required because it appears within the type `RootDatabase`
--> /Users/dbarsky/Developer/rust-analyzer/crates/ide-db/src/lib.rs:79:12
|
79 | pub struct RootDatabase {
| ^^^^^^^^^^^^
note: required because it appears within the type `Analysis`
--> /Users/dbarsky/Developer/rust-analyzer/crates/ide/src/lib.rs:219:12
|
219 | pub struct Analysis {
| ^^^^^^^^
note: required because it's used within this closure
--> crates/rust-analyzer/src/cli/rustc_tests.rs:163:60
|
163 | let res = std::panic::catch_unwind(move || {
| ^^^^^^^
note: required by a bound in `std::panic::catch_unwind`
--> /Users/dbarsky/.rustup/toolchains/nightly-aarch64-apple-darwin/lib/rustlib/src/rust/library/std/src/panic.rs:357:40
|
357 | pub fn catch_unwind<F: FnOnce() -> R + UnwindSafe, R>(f: F) -> Result<R> {
| ^^^^^^^^^^ required by this bound in `catch_unwind` I think that #19093 is also slightly problematic from a "get rust-analyzer to compile" standpoint, but I don't know enough about unwind safety to say whether I should wrap analysis— and therefore, accumulators—in a |
54ef4ea
to
83fe206
Compare
I decided to wrap some |
That's fine, the problematic type only appears within |
Alright, few updates: Debug PrintingI put up salsa-rs/salsa#676, which makes it possible for us to access the underlying data without involving a Unfortunately, the approach is not applicable to debug printing in EnumsI've also opened salsa-rs/salsa#677, which mostly just publishes @ChayimFriedman2's Footnotes
|
83fe206
to
5b4a994
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Another partial review, still need to go through the ide crates and the main rust-analyzer crate
Also feel free to squash the history here to make your rebase life easier, I don't really care for my commit(s) in there |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Okay I've done a full pass now
5b4a994
to
96c645e
Compare
dea1fe7
to
65d45e3
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Judging by the ide test failure, the weird editioned file handling is likely the cause (we are scrambling the editions around somewhere)
the keyword highlighting one? Yeah, good point. Will dig around, must be something kinda inadvertently silly. I'll fix the weird construction/deconstruction of |
47fda29
to
57353c7
Compare
Updates:
We'll still need to:
|
1cf3bdb
to
b8a6a3d
Compare
crates/span/src/hygiene.rs
Outdated
// FIXME: is this right? | ||
if self.is_root() { | ||
return Edition::LATEST; | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This might be the cause for the syntax highlighting test failures
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
// FIXME: is this right? | |
if self.is_root() { | |
return Edition::LATEST; | |
} | |
if self.is_root() { | |
return Edition::from_u32(self.0.as_u32() - (SyntaxContext::MAX_ID - Edition::LATEST as u32)); | |
} |
Note Edition::from_u32
isn't a thing, gotta create that as well
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good catch! Applying both makes test_keyword_macro_edition_highlighting
pass, but test_keyword_highlighting
and test_rainbow_highlighting
still fail. Not sure why, but working on vendoring the query group macro (and updating it to the latest Salsa...).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
rainbow i think isnt a problem, we likely are hashing different data now, as long as the hashes are stable and equal as before (wrt to positions they appear in). The other keyword one is still edition related though hmm
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actually it is weird that the hashes change, they shouldn't nothing there goes through salsa ... Ah nvm, rustc-hash
got bumped in this PR, that's why they changed I imagine
b8a6a3d
to
8ae7f64
Compare
8ae7f64
to
43ec93c
Compare
Rebased + tests should be fixed. Please review the test changes in |
|
okay! I will squash and add this to the merge queue (!) |
43ec93c
to
f956b77
Compare
f956b77
to
a68aa7d
Compare
a68aa7d
to
74620e6
Compare
This PR ports rust-analyzer to the new Salsa. It is big, partially incomplete, and best reviewed commit-by-commit. I will be rebasing often.
(Why new Salsa? It unblocks a lot of work described in #17491.)
Migration Approach
To get rust-analyzer compiling and passing tests, this PR did the following:
db-ext-macro
to mimic rust-analyzer's usage of old Salsa's query groups. Over time, rust-analyzer will migrate to new Salsa's idioms, moving away from the query group pattern. For additional details on the differences between old Salsa and new Salsa, please refer to this document.crates/span
to new Salsa, which makes use of self-referential interning. This is needed to support macro hygiene.'db
lifetime. However, the presence of the'db
lifetime made migrating a large number of interned structs challenging, hence feature: add#[salsa::interned_sans_lifetime]
macro salsa-rs/salsa#632.With these changes, all tests were passing (no correctness issues!),
analysis-stats
produced the same results as rust-analyzer on the old Salsa, and—critically—the new Salsa-powered rust-analyzer was usable as my daily driver. Unfortunately, memory usage was too high. I'm really grateful to the following folks who helped fix that regression:analysis-stats
from 8 gigabytes to 4 gigabytes by through https://lgithub.com/salsa-rs/salsa/pull/614/.#[salsa::enum]
, which reduced memory usage by about 150 megabytes and made rust-analyzer's usage of new Salsa more idiomatic.Performance
rust-analyzer analysis-stats
, new Salsa is slightly slower and uses slightly more memory.crates/rust-analyzer/src/integrated_benchmarks.rs
, new Salsa is slightly faster on both cold and warm updates. These benchmarks should probably be more rigorous, but new Salsa was consistently faster.Things to Fix Before to Merging
There's a few things to do before this change is merged:
enum
/#[salsa::supertype]
commit cause tests to fail to type confusion in the underlying Salsa pages. Chayim is working on root-causing this, but if he's unable to determine the cause, I can make the enum changes a separate pull request.db-ext-macro
.db-ext-macro
to something else, or have it live in Salsa. I'm unsure which option I'd prefer right now.Acknowledgements
This wouldn't have been possible without the assistance of many folks including @ChayimFriedman2, @ShoyuVanilla, @nikomatsakis, and @Veykril. I'm sure I'm forgetting some people, to whom I apologize.