Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement support for HVF Nested Virtualization #288

Merged
merged 7 commits into from
Apr 3, 2025

Conversation

slp
Copy link
Contributor

@slp slp commented Mar 26, 2025

Apple introduced Nested Virt (EL2) support in macOS Sequoia, available
on Apple Silicon devices based on M3 and later SoCs.

This commit introduces the infrastructure to enable Nested Virt on
libkrun. The biggest change is setting up the vCPU reset registers to
values that are acceptable in EL2 according. This isn't easy since HVF
doesn't document its expectations, but the current implementation allows
the guest to boot in EL2 and run a nested guest.

Instead of linking directly against the new functions, we're using
libloader again to find the new symbols, to avoid breaking the binaries
in Sonoma.

@slp
Copy link
Contributor Author

slp commented Mar 26, 2025

This depends on #280, creating this as a draft until it's merged.

slp added 7 commits April 1, 2025 10:18
PSCI supports both hvc and smc as conduits. So far, we were only
supporting the first. Add support for the latter too.

Signed-off-by: Sergio Lopez <[email protected]>
PSCI can use both hvc and smc as conduits. Both are supported on macOS,
but only the latter works on both EL1 and EL2 modes, so use it by
default.

Signed-off-by: Sergio Lopez <[email protected]>
So far we only supported ICC sysregs, but for enabling EL2 we also need
to support non-ICC sysregs. Generalize sysreg management.

Signed-off-by: Sergio Lopez <[email protected]>
Apple introduced Nested Virt (EL2) support in macOS Sequoia, available
on Apple Silicon devices based on M3 and later SoCs.

This commit introduces the infrastructure to enable Nested Virt on
libkrun. The biggest change is setting up the vCPU reset registers to
values that are acceptable in EL2 according. This isn't easy since HVF
doesn't document its expectations, but the current implementation allows
the guest to boot in EL2 and run a nested guest.

Instead of linking directly against the new functions, we're using
libloader again to find the new symbols, to avoid breaking the binaries
in Sonoma.

Signed-off-by: Sergio Lopez <[email protected]>
After introducing Nested Virt (EL2) support on macOS, let's add a new
API to enable library users to request its enablement.

Signed-off-by: Sergio Lopez <[email protected]>
Add the flag "-n" to request the enablement of nested virtualization.

Signed-off-by: Sergio Lopez <[email protected]>
Update KRUN_EFI binary built from
github.com/slp/edk2:13e8adac8a83141b51375c799996946082e1eb43

This version includes a patch to build with strict alignment,
which we need to when starting the vCPUs in EL2 as required when
enabling Nested Virtualization.

Signed-off-by: Sergio Lopez <[email protected]>
@slp slp force-pushed the hvf-nested-virt branch from c822ad2 to 8c110fa Compare April 1, 2025 10:50
@slp slp marked this pull request as ready for review April 1, 2025 10:55
@slp
Copy link
Contributor Author

slp commented Apr 1, 2025

Rebased, working and ready to be reviewed!

Copy link
Member

@jakecorrenti jakecorrenti left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code LGTM but I can't test the nested virt since I don't have an M3.

Copy link
Member

@tylerfanelli tylerfanelli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One small nit, but overall LGTM! I also don't have an M3 to test however.

libloading::Library::new(
"/System/Library/Frameworks/Hypervisor.framework/Versions/A/Hypervisor",
)
.unwrap()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it make sense to fail gracefully here in the case that the Hypervisor library is either unavailable or of a different version?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That'd be an ABI break on the macOS side, it's ~ the equivalent of .so versions on Linux

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, in the unlikely case Apple stops shipping version 'A' of Hypervisor.framework, we can't do much other than printing a different error message. ;-P

@slp slp merged commit df5fc2f into containers:main Apr 3, 2025
6 checks passed
@slp slp deleted the hvf-nested-virt branch April 3, 2025 10:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants