RFC: enable openhcl to run in VTL0, use nested virtualization #281
Conversation
| @@ -54,11 +54,20 @@ pub enum IgvmManifestPath { | |||
| LocalOnlyCustom(PathBuf), | |||
| } | |||
|
|
|||
| #[derive(Serialize, Deserialize, Clone, Debug)] | |||
| pub enum InitrdRootfsPath { | |||
There was a problem hiding this comment.
nit: here and elsewhere, I would suggest using the slightly more verbose name InitrdRootfsConfigPath, to make it marginally more clear we're talking about the rootfs.config files here.
it took me a sec to realize this wasn't related to the pre-packaged initrd layers themselves
| dir /lib/modules/000 0755 0 0 | ||
| dir /lib/modules/001 0755 0 0 | ||
| dir /lib/modules/999 0755 0 0 | ||
| dir /lib/modules/auto 0755 0 0 |
There was a problem hiding this comment.
what's the "auto" here for?
There was a problem hiding this comment.
Anything in auto will be automatically loaded. Modules elsewhere need explicit code to load.
I'll add a comment.
| //! it at some point. Right now, there are enough differences in requirements | ||
| //! that this is not practical. |
There was a problem hiding this comment.
what sort of differences are we talking about here?
| } else if vendor.is_amd_compatible() { | ||
| Some("amd") | ||
| } else { | ||
| None |
There was a problem hiding this comment.
shouldn't this have a louder error path?
|
You got an example of how one would run this? |
|
Can we add some sort of basic vmm test for this? I can work with you on petri & macro support for whatever's needed (haven't looked too in depth yet). |
Yeah. Shouldn't be too bad. Linux x64 host only for now, until we add nested support to virt_whp. |
I'll add one, once the kernel updates are in. But basically it's similar to how you run openhcl within openvmm today, except you don't pass |
| /// exit flowing through the paravisor. | ||
| /// | ||
| /// This is best effort. Exits for this range may still flow through the paravisor. | ||
| fn register_host_io_port_fast_path(&self, range: RangeInclusive<u16>) -> Box<dyn Send>; |
There was a problem hiding this comment.
What is this return value useful for?
| @@ -20,6 +20,11 @@ openssl-vendored = ["underhill_attestation/openssl-vendored"] | |||
| # Enable VPCI device support | |||
| vpci = [] | |||
|
|
|||
| # Enable support for running the guest OS via nested virtualization with KVM. | |||
| # (Note that the virt_kvm crate is always a dependency just to avoid build | |||
There was a problem hiding this comment.
Is this a binary size concern?
| } | ||
| } | ||
|
|
||
| impl OpenhclPartition for virt_kvm::KvmPartition { |
There was a problem hiding this comment.
Maybe worth mentioning in a comment that this is not a shipping configuration, hence all the stubs?
| @@ -292,6 +291,8 @@ pub struct UnderhillEnvCfg { | |||
| pub no_sidecar_hotplug: bool, | |||
| /// Enables the GDB stub for debugging the guest. | |||
| pub gdbstub: bool, | |||
| /// Use KVM instead of mshv_vtl. | |||
| pub kvm: bool, | |||
There was a problem hiding this comment.
enum Backend? Presumably we may have more than just mshv_vtl and kvm in the future?
There was a problem hiding this comment.
That could also allow carrying the ProtoPartition on an enum, instead of having to do all the if lets on it. Like it feels a little off to me that every branch where we don't have a proto partition assumes kvm, an enum could make the link clearer.
This change enables OpenHCL to be launched in a non-VSM capable partition (i.e., in VTL0) and to use nested virtualization to run the target guest OS.
To achieve this, we build the existing
virt_kvmbackend into the paravisor openvmm build. So this has all the same limitations asvirt_kvmhas on the host.With this, it becomes possible to do some basic OpenHCL testing on a Linux platform, with no Windows dependencies. Of course, since we are using nested virtualization, there are some limitations:
The host is not aware of the L2 guest, so it cannot offer vmbus or assigned devices directly to it. All devices must be emulated in the paravisor. There is no support for an interactive graphics console or anything else we haven't built a relay for yet.
Performance is pretty bad in my testing. Of course, I tested this with 3x nested virtualization, so that is not too surprising. I expect this to be much better on a native Linux host.
Currently, we still depend on the GET and other Hyper-V HCL devices, so this can really only be used in conjunction with openvmm on the host. Future changes can support alternate configuration mechanisms so that this would be able to run in qemu or another VMM.
Also, virt_whp and virt_hvf do not yet support nested virtualization (even though WHP and HVF do), so even with openvmm, this only works on Linux.
Finally, I only built and tested this on x64, because I do not have a KVM ARM64 environment that supports nested virtualization.