Sorry guys for another hiatus, my job at Calxeda keeps me busy. I was recently discussing ARM’s virtualization support with my friend Ali Hussain (yup, that’s our idea of a fun dinner conversation) and found some very interesting facts. I requested Ali to share his knowledge in a blog post series on this topic, so here you go. Ali is in ARM’s performance modeling team and has been working on ARM cores since 2008.
The idea for this blog post stemmed from talking to people that had the impression that ARM’s virtualization support, even with the virtualization extensions in Cortex-A15, is limited. I plan to write a few posts exploring virtualization, and the support for it in the ARM and x86 ISAs. This post will draw heavily on my understanding of the ARM architecture and operating systems.
What is Virtualization?
Before we can look at virtualization, we need to define a few key things. The first is virtualization itself. Virtualization in general is creating an environment in software to emulate something physical. More specifically, when we talk about hardware virtualization, it is running an operating system on a sandboxed virtual machine (VM) as opposed to having access to the physical hardware. This is done through the hypervisor which manages guest operating systems the same way an operating system manages applications.
Virtualization does not require hardware support. It can be performed completely in software. An existing operating system can be patched to work at a lower privilege level and trap to the hypervisor. This is called paravirtualization. The advantage of hardware support is in simplifying the software work for the hypervisor and providing performance improvements. For this post, I want to discuss how hardware can help the hypervisor in performing its functions. Those interested in paravirtualization will find this talk about problems faced while implementing paravirtualization on ARM interesting by VMWare: http://labs.vmware.com/academic/arm-core-virtualization Let’s explore some of the duties of the hypervisor and how the hardware accomplishes them to better understand hardware-assisted virtualization.
Management Has Its Privileges
To sandbox an OS, the hypervisor has to be at a higher privilege level than the guest OS. ARM, for example, creates a higher privilege level called hypervisor mode. The hypervisor mode has access to its own set of system registers that are analogous to the registers present in the system mode. For example, just like the OS tracks process IDs using an Address Space IDentifier (ASID which is a part of the TTBR), the hypervisor tracks current VM using a VMID (which is a part of the VTTBR).
The hypervisor also has similar access controls as the supervisor mode. In addition, the hypervisor can read and write the system control registers for the OS. Having two layers of access control does create a lot of interesting scenarios where the hypervisor and the guest OS compete for a trap. ARM has gone with the philosophy that a good manager is one that intrudes into your work as little as possible. So, if an exception occurs, the guest OS is typically given the opportunity to service it before the hypervisor because the OS is better equipped to handle the applications’ requirements. Let me explain this with an example. Both the OS and the hypervisor provide a similar feature for disabling access to the floating point and SIMD units, using the CPACR (Coprocessor Access Control Register) and HCPTR (Hypervisor Coprocessor Trap Register) respectively. When an application running in the guest OS is disallowed access by both the CPACR and HCPTR, the CPACR has priority.
This is an interesting design choice. It has two advantages. First, it improves the response time of the exception. Second, it allows the OS to behave as normal. However, it also makes it extremely difficult to provide certain functionality, e.g., making the floating point unit invisible to the guest. Next week I’ll add another post talking about the memory management and interrupt handling in a hypervisor.
Disclaimers:
- All opinions expressed here are my own and not of ARM or any other entity.
- I haven’t implemented a hypervisor or operating system myself so I would love to hear comments from experts in the field.
Resources:
- http://en.wikipedia.org/wiki/Virtualization
- http://en.wikipedia.org/wiki/Operating_system
- http://infocenter.arm.com/help/index.jsp
- http://www.linaro.org/documents/download/d7fe510d8eb46775afc3953d217b15224fbb93086598a
Reference: ARM Virtualization Extensions – Introduction (Part 1) from our JCG partner Aater Suleman at the Future Chips blog.
