Xen is a GPLv2-licensed type 1 hypervisor for Intel® and ARM® architectures. FreeBSD has included i386™ and AMD® 64-Bit DomU and Amazon EC2 unprivileged domain (virtual machine) support since FreeBSD 8.0 and includes Dom0 control domain (host) support in FreeBSD 11.0. Support for para-virtualized (PV) domains has been removed from FreeBSD 11 in favor of hardware virtualized (HVM) domains, which provides better performance.
Xen™ is a bare-metal hypervisor, which means that it is the
first program loaded after the BIOS. A special privileged guest
called the Domain-0 (Dom0 for short) is then
started. The Dom0 uses its special privileges to directly
access the underlying physical hardware, making it a
high-performance solution. It is able to access the disk
controllers and network adapters directly. The Xen™ management
tools to manage and control the Xen™ hypervisor are also used
by the Dom0 to create, list, and destroy VMs. Dom0 provides
virtual disks and networking for unprivileged domains, often
called DomU. Xen™ Dom0 can be compared to
the service console of other hypervisor solutions, while the
DomU is where individual guest VMs are run.
Xen™ can migrate VMs between different Xen™ servers. When the two xen hosts share the same underlying storage, the migration can be done without having to shut the VM down first. Instead, the migration is performed live while the DomU is running and there is no need to restart it or plan a downtime. This is useful in maintenance scenarios or upgrade windows to ensure that the services provided by the DomU are still provided. Many more features of Xen™ are listed on the Xen Wiki Overview page. Note that not all features are supported on FreeBSD yet.
To run the Xen™ hypervisor on a host, certain hardware functionality is required. Hardware virtualized domains require Extended Page Table (EPT) and Input/Output Memory Management Unit (IOMMU) support in the host processor.
Users of FreeBSD 11 should install the emulators/xen-kernel47 and sysutils/xen-tools47 packages that are based on Xen version 4.7. Systems running on FreeBSD-CURRENT with at least revision r336475 or higher, can use Xen 4.11 provided by emulators/xen-kernel411 and sysutils/xen-tools411, respectively.
Configuration files must be edited to prepare the host
for the Dom0 integration after the Xen packages are installed.
An entry to /etc/sysctl.conf disables the
limit on how many pages of memory are allowed to be wired.
Otherwise, DomU VMs with higher memory requirements will not
run.
#sysrc -f /etc/sysctl.conf vm.max_wired=-1
Another memory-related setting involves changing
/etc/login.conf, setting the
memorylocked option to
unlimited. Otherwise, creating DomU
domains may fail with Cannot allocate
memory errors. After making the change to
/etc/login.conf, run
cap_mkdb to update the capability database.
See Section 13.13, “Resource Limits” for
details.
#sed -i '' -e 's/memorylocked=64K/memorylocked=unlimited/' /etc/login.conf#cap_mkdb /etc/login.conf
Add an entry for the Xen™ console to
/etc/ttys:
#echo 'xc0 "/usr/libexec/getty Pc" xterm on secure' >> /etc/ttys
Selecting a Xen™ kernel in
/boot/loader.conf activates the Dom0.
Xen™ also requires resources like CPU and memory from the
host machine for itself and other DomU domains. How much CPU
and memory depends on the individual requirements and hardware
capabilities. In this example, 8 GB of memory and 4
virtual CPUs are made available for the Dom0. The serial
console is also activated and logging options are
defined.
The following command is used for Xen 4.7 packages:
#sysrc -f /boot/loader.conf hw.pci.mcfg=0#sysrc -f /boot/loader.conf xen_kernel="/boot/xen"#sysrc -f /boot/loader.conf xen_cmdline="dom0_mem=8192Mdom0_max_vcpus=4dom0pvh=1 console=com1,vga com1=115200,8n1 guest_loglvl=all loglvl=all"
For Xen versions 4.11 and higher, the following command should be used instead:
#sysrc -f /boot/loader.conf hw.pci.mcfg=0#sysrc -f /boot/loader.conf xen_kernel="/boot/xen"#sysrc -f /boot/loader.conf xen_cmdline="dom0_mem=8192Mdom0_max_vcpus=4dom0=pvh console=com1,vga com1=115200,8n1 guest_loglvl=all loglvl=all"
Log files that Xen™ creates for the Dom0 and DomU VMs
are stored in /var/log/xen. This
directory does not exist by default and must be
created.
#mkdir -p /var/log/xen#chmod 644 /var/log/xen
Xen™ provides a boot menu to activate and de-activate
the hypervisor on demand in
/boot/menu.rc.local:
#echo "try-include /boot/xen.4th" >> /boot/menu.rc.local
Activate the xencommons service during system startup:
#sysrc xencommons_enable=yes
These settings are enough to start a Dom0-enabled
system. However, it lacks network functionality for the
DomU machines. To fix that, define a bridged interface with
the main NIC of the system which the DomU VMs can use to
connect to the network. Replace
igb0 with the host network
interface name.
#sysrc autobridge_interfaces=bridge0#sysrc autobridge_bridge0=igb0#sysrc ifconfig_bridge0=SYNCDHCP
Restart the host to load the Xen™ kernel and start the Dom0.
#reboot
After successfully booting the Xen™ kernel and logging
into the system again, the Xen™ management tool
xl is used to show information about the
domains.
#xl listName ID Mem VCPUs State Time(s) Domain-0 0 8192 4 r----- 962.0
The output confirms that the Dom0 (called
Domain-0) has the ID 0
and is running. It also has the memory and virtual CPUs
that were defined in /boot/loader.conf
earlier. More information can be found in the Xen™
Documentation. DomU guest VMs can now be
created.
Unprivileged domains consist of a configuration file and
virtual or physical hard disks. Virtual disk storage for
the DomU can be files created by truncate(1) or ZFS
volumes as described in Section 19.4.2, “Creating and Destroying Volumes”.
In this example, a 20 GB volume is used. A VM is
created with the ZFS volume, a FreeBSD ISO image, 1 GB of
RAM and two virtual CPUs. The ISO installation file is
retrieved with fetch(1) and saved locally in a file
called freebsd.iso.
#fetchftp://ftp.freebsd.org/pub/FreeBSD/releases/ISO-IMAGES/10.3/FreeBSD-10.3-RELEASE-amd64-bootonly.iso-ofreebsd.iso
A ZFS volume of 20 GB called
xendisk0 is created to serve as the disk
space for the VM.
#zfs create -V20G -o volmode=dev zroot/xendisk0
The new DomU guest VM is defined in a file. Some specific
definitions like name, keymap, and VNC connection details are
also defined. The following freebsd.cfg
contains a minimum DomU configuration for this example:
#cat freebsd.cfgbuilder = "hvm"name = "freebsd"
memory = 1024
vcpus = 2
vif = [ 'mac=00:16:3E:74:34:32,bridge=bridge0' ]
disk = [ '/dev/zvol/tank/xendisk0,raw,hda,rw',
'/root/freebsd.iso,raw,hdc:cdrom,r'
] vnc = 1
vnclisten = "0.0.0.0" serial = "pty" usbdevice = "tablet"
These lines are explained in more detail:
| This defines what kind of virtualization to use.
|
| Name of this virtual machine to distinguish it from others running on the same Dom0. Required. |
| Quantity of RAM in megabytes to make available to the VM. This amount is subtracted from the hypervisor's total available memory, not the memory of the Dom0. |
| Number of virtual CPUs available to the guest VM. For best performance, do not create guests with more virtual CPUs than the number of physical CPUs on the host. |
| Virtual network adapter. This is the bridge connected
to the network interface of the host. The
|
| Full path to the disk, file, or ZFS volume of the disk storage for this VM. Options and multiple disk definitions are separated by commas. |
| Defines the Boot medium from which the initial operating system is installed. In this example, it is the ISO imaged downloaded earlier. Consult the Xen™ documentation for other kinds of devices and options to set. |
| Options controlling VNC connectivity to the serial
console of the DomU. In order, these are: active VNC
support, define IP address on which to listen, device node
for the serial console, and the input method for precise
positioning of the mouse and other input methods.
|
After the file has been created with all the necessary
options, the DomU is created by passing it to xl
create as a parameter.
#xl create freebsd.cfg
Note:
Each time the Dom0 is restarted, the configuration file
must be passed to xl create again to
re-create the DomU. By default, only the Dom0 is created
after a reboot, not the individual VMs. The VMs can
continue where they left off as they stored the operating
system on the virtual disk. The virtual machine
configuration can change over time (for example, when adding
more memory). The virtual machine configuration files must
be properly backed up and kept available to be able to
re-create the guest VM when needed.
The output of xl list confirms that the
DomU has been created.
#xl listName ID Mem VCPUs State Time(s) Domain-0 0 8192 4 r----- 1653.4 freebsd 1 1024 1 -b---- 663.9
To begin the installation of the base operating system,
start the VNC client, directing it to the main network address
of the host or to the IP address defined on the
vnclisten line of
freebsd.cfg. After the operating system
has been installed, shut down the DomU and disconnect the VNC
viewer. Edit freebsd.cfg, removing the
line with the cdrom definition or
commenting it out by inserting a #
character at the beginning of the line. To load this new
configuration, it is necessary to remove the old DomU with
xl destroy, passing either the name or the
id as the parameter. Afterwards, recreate it using the
modified freebsd.cfg.
#xl destroy freebsd#xl create freebsd.cfg
The machine can then be accessed again using the VNC viewer. This time, it will boot from the virtual disk where the operating system has been installed and can be used as a virtual machine.