This is the first Fedora virt status report in a month now, and it's
been a heck of busy month!
- The F11 feature/beta freeze
- Scrambling to complete the F11 virt features
- Legendary work getting the qemu/kvm package merge in shape
- A re-designed virt-manager UI
- Various upstream releases
- Progress on Xen Dom0
- A massive amount of bugzilla activity
We're not there yet, though. The final important item on the F11
2009-04-14 Final freeze (20 days)
There's a huge pile of bug-fixing and polish work to do in that
time. If you're looking to help out, there's no better place to start
than the F11VirtBlocker/F11VirtTarget tracker bugs:
Fedora 11 Beta
The Beta was due to be released at this point, but a 7 day slip was
announced last week:
Jesse isn't pointing fingers, but I am ... it's anaconda's fault :-)
It looks like today's rawhide may be the final Beta compose, as the
blocker list is looking much healthier:
Fedora Weekly News
With so much going on in the Fedora Virtualization space in the last
number of weeks, it's great to see Dale Bewley continuing to
contribute excellent virt sections to FWN:
F11 Virt Features
Chris Lalance summarised a recent FECSo meeting where four virt
features were discussed:
The upshot is that the Shared Network Interface is postponed until
Also, the KVM and QEMU Merge feature fell off the radar briefly, but
will hopefully be approved as an exception by FESCo this week.
So, the final list of F11 virt features is:
* KVM PCI Device Assignment
* KVM and QEMU merge
* SVirt Mandatory Access Control
Planning for Fedora 12 has already begun, and the current set of
features proposed is:
* Shared Network Interface
KVM PCI Device Assignment
This feature is now complete for F11:
Just in time for feature freeze, the libvirt node device attach,
re-attach and reset APIs were added (in 0.6.1). Cole also added the
--host-device parameter to virt-install and host device support to
Also, /sys/bus/pci/devices/pci-stub/remove_id (used by libvirt) was
recently backported from 2.6.30 to rawhide:
KVM and QEMU Merge
Glauber continued his awesome work on merging the qemu and kvm
-- QEMU 0.10 --
Upstream QEMU did a 0.10.0 release, followed closely by a 0.10.1
This release will form the basis of the Fedora 11 version of
QEMU. However, in order for that to happen, the KVM upstream
maintainers need to release a version of kvm-userspace which is based
on the qemu-0.10.x series. This is planned in the near future. In the
meantime, rawhide will ship snapshots of latest kvm-userspace.git.
QEMU upstream plan to do regular releases every three months. There
will also be releases from the current stable branch (e.g. 0.10.x for
now) as needed. This is great news for Fedora, as it means we can
track QEMU development in Fedora while still being able to keep QEMU
stable in Fedora releases.
-- BIOS Packages --
Work continued on the gnarly issue of build BIOS images for QEMU. The
vgabios and openbios packages were approved:
Review Request: vgabios - vga option rom for bochs/qemu
Review Request: openbios - Open Source implementation of IEEE 1275-1994
Also, the bochs-bios package was approved, but with koji's ability to
build noarch sub-packages, it was no longer needed:
Review Request: bochs-bios - bios implementation from the bochs
Later, it was realized that a KVM specific version of bochs was
needed, so Glauber added that to the bochs package:
Cannot run F10/F11 i386 guests without disabling ACPI
doesn't boot RHEL 5 kernel
-- The Merge --
Glauber committed his merged package to rawhide and various folks
jumped in helping with testing and cleaning up various packaging
kvm : Conflicts with other packages
/usr/bin/qemu-nbd: conflict with qemu-common-1.0-0.4.svn6666.fc11.x86_64
problems with qemu scripts
bochs-bios >= 2.3.8-0.5 is needed by
New qemu package breaks kvm virtualization
qemu missing some files for kvm support
qemu-kvm has wrong directory for keymaps
The version included in the F11 Beta release seems in really good
shape considering how significant a change this was.
-- GCC Bug --
Another issue that arose was that latest gcc is breaking plain QEMU. A
patch has been proposed upstream, but it's not in rawhide yet:
segfault in stw_kernel when qemu is run
GCC register allocation wrongly using ebp
The sVirt patch set was added to libvirt-0.6.1 and is in the Fedora 11
Some issues remain, but things are looking in pretty good shape, all
Some of the related issues include:
selinux: virt-manager downloaded kernel/initrd cannot be accessed by KVM guest
Nasty issue where, because virt-manager no longer runs as root,
kernel and initrd images are readable by qemu because of their
Dan Walsh committed some policy changes to fix this, but
virt-manager will need to call restorecon on downloaded images.
Dan also suggested a patch from virt-manager using this
interesting bit of SELinux functionality:
>> import selinux
$> cat /etc/selinux/targeted/contexts/virtual_image_context
SELinux prevented qemu-kvm from reading an LVM logical volume
LVs created in an existing VG have wrong SELinux label
Issues around the appropriate labelling of logical volumes used
for guest installs. Should work better in rawhide.
Sound card breaks new and existing vm's
A pulseaudio error caused by sVirt restrictions on guests was
causing VMs with sound hardware to fail to start. Fixed in
libvirt by disabling sound hardware if sVirt is enabled.
AVC denied errors when starting KVM guest
Another issue cropped up with the selinux changes for
libvirt 0.6.0. Fix is in F10 now.
Improved Graphical Console
Dan Berrange added a feature page describing the changes made to have
the graphical console be 1024x768 by default and use a USB tablet in
absolute mode rather than a PS2 mouse.
The relevant bugs for each of the improvements are listed on the wiki
These changes will make a huge difference to the user experience with
Fedora virtualization tools. No longer a crappy screen resolution and
no longer schizophrenic mouse behaviour.
One related problem that Dan had to fix was a particularly nasty
'invisible wall' bug in gtk-vnc to do with pointer grabs and relative
Cursor Movement is Constrained
A particularly nasty 'invisible wall' bug in gtk-vnc to do with
pointer grabs and relative mode.
Dan Berrange got his VNC SASL authentication patches for QEMU and
backported them to rawhide.
Testing is welcome for this feature, but you'll need to bug Dan about
filling in the "How To Test" section of the feature page first :-)
One problem which has just become apparent is that, in rawhide
currently, we are shipping a snapshot of kvm-userspace.git which
includes the SASL patches. However, when upstream KVM release a
version of kvm-userspace based on qemu-0.10.x, that will not include
the SASL patches. This means the patches will have to be backported
again. Three cheers for confusion!
New virtinst and virt-manager Releases
Cole released a new version of virtinst:
This release includes:
- New virt-clone option --original-xml, allows cloning a guest from
an xml file, rather than require an existing, defined guest.
- New virt-install option --import, allows creating a guest from an
existing disk image, bypassing any OS install phase.
- New virt-install option --host-device, for connecting a physical
host device to the guest.
- Allow specifying 'cache' value via virt-install's --disk options
- New virt-install option --nonetworks (John Levon)
- Lots of backend cleanups and documentation improvements.
and also a new virt-manager release:
This release includes:
- Redesigned 'New Virtual Machine' wizard
(Jeremy Perry, Tim Allen, Cole Robinson)
- Option to remove storage when deleting a virtual machine.
- File browser for libvirt storage pools and volumes, for use when
attaching storage to a new or existing guest.
- Physical device assignment (PCI, USB) for existing virtual
- Bug fixes and minor improvements.
Both releases resolve many Fedora bugs and add new and interesting
features, but by far the most user-visible change is the re-designed
'New Virtual Machine' wizard. More details Cole's blog:
I think everyone will agree that this is a huge improvement.
I particularily like the "detect distro" feature where we now make
every effort to try and figure out what OS is being installed rather
than require the user to tell us.
libvirt-0.6.1 was released:
+ New features:
- new APIs for Node device detach reattach and reset (Mark McLoughlin)
- sVirt mandatory access control support (James Morris and Dan Walsh)
This is available in rawhide, the F11 Beta and F10
updates-testing. Note, however, that rawhide and F11Beta has quite a
number of fixes not yet in updates-testing.
Tricky Time Troubles
Hans De Goede tracked down a nasty problem with anaconda running under
Unable to mount filesystem: device /dev/sda1 does not exist
The problem turned out to be that anaconda was sleeping for a single
second to give the kernel time to reload the partition table after
committing changes to a disk.
However, one cannot predict predict how many CPU cycles a virtual
machine will execute in a fixed about of real time. If the host or
other guests are busy, then the guest may be starved of CPU for a
In this case, the kernel was sometimes not getting enough CPU cycles
in the time that anaconda was sleeping and so the partition table
wasn't reloaded. The fix was to use udev_settle so that we could know
for certain when the devices nodes were visible.
Michael Young continued posting more Dom0 kernel builds. Gerd has
helped out by builing new versions of the xen userspace package to
work with these kernels.
Upstream recently saw some lively debate on whether the patches should
be include at all:
It's hard to predict these things, but it seems likely the patches
will end up being accepted.
Pasi Kärkkäinen has spent a good deal of time working through various
issues with getting a Dom0 install working.
One problem identified is that /sys/hypervisor was missing
upstream prompting Jeremy Fitzhardinge added a patch to enable that.
Pasi also filed some bugs arising from his testing:
libvirt "missing source" error message overwritten by "failed to
build sexpr" error
Basically, libvirt is giving a useless error message when the
bridge name for a given network can't be found.
libvirtd fails to start if pid file exists but init script still
says it started OK
libvirt's handling of its PID file isn't quite correct; nasty
Pasi later reported further success:
Today I was able to run the following on Fedora 10 32bit PAE pv_ops dom0:
- CentOS 5.3 32bit PAE PV domU
- Fedora 10 32bit PAE PV domU
- Use virt-install to install Fedora 10 32bit PAE PV domU
DOOM-O-METER: 192 one month ago, 185 now. Not bad!
The summary of bugzilla activity is seriously long, so I'll try and
categorize things a little:
-- Installer --
anaconda installs the wrong kernel for i686 xen guests
danpb confirms this is now fixed because anaconda will always install PAE
kernels on systems which support it.
QEMU disks show up in anaconda as UNKNOWN
Parted needed some smarts to return an appropriate model string
for virtio disks. Joel Granados came up with a patch to look in
/proc/devices for "virtblk".
anaconda "unitialized drive" warning is a little too terrifying
When virtinst creates a blank disk, anaconda throws up a
slightly terrifying warning - "YOU WILL LOSE ALL DATA". We
should be able to make the user experience a tad better here.
installation of guest fails with encryption
Looks like yet another anaconda storage-rewrite bug.
Anaconda eats CPU
A report that anaconda was stuck in a loop in KVM. Turned out to be
a simple "poll() on invalid fd" bug in anaconda itself.
-- QEMU --
qemu package missing debuginfo for qemu-img
Glauber has a fix which he submitted upstream. Turns out the
upstream makefile was using "install -s", causing the binary to
be stripped before find-debuginfo got to it.
qemu-img crashes creating 5TB qcow2 file
Integer wraparound creating a large disk image in VPC format
Dan Berrange reports some issues with qemu's handling of very
kvm: Build qemu with $RPM_OPT_FLAGS
Glauber nicely fixed this long-standing issue.
Update QEMU to use gPXE roms for iSCSI boot support
Review Request: gpxe - A network boot loader
Discussion around moving to gPXE option ROMs instead of
etherboot so as to get iSCSI initiator support during PXE boot.
The blocking issue here is modifying QEMU to allow more space
for the large gPXE ROM images.
block-rw-range-check.patch breaks qcow2
"Cannot open disk image" when using qcow1
Our only significant not-upstream qemu patch has finally been
merged (and fixed) upstream. Yay!
Fedora11 qemu-kvm installation using qcow2 is not recognizing
the hard drive
Another bug filed by IBM folks testing the alpha. Almost
certainly the same as the qcow2 issue caused by
no working networking in guests with qemu-kvm
A transient networking issue reported by Jeremy Katz. Seems to
have gone away of its own accord.
Can't use AES encryption in qcow2
F10 qemu seems to lock up with encrypted qcow2 disks before
asking for the password. It works fine in rawhide - so long as
you think of switching to the qemu monitor with Ctrl-Alt-2.
Need to add subsystem vendor ID for kvm/qemu cirrus card
xorg should use 1024x768 for kvm/qemu cirrus card
Now with an F11 host and an F11 guest, xorg's cirrus driver will
default to 1024x768 instead of 800x600. Woo!
qemu-kvm segfaults on startup in SDL_memcpyMMX/SSE
Eduardo tracked down and fixed this nasty crasher in libSDL.
F10 kvm: network stall with qemu rtl8139 NIC emulation
People continue to have issues with using RHEL5 and QEMU's
qemu-kvm not working with evdev giving wrong key mappings
Finally fixed, it seems. Not clear on the details.
Adding ac97-module (from qemu-repo)
ac97 emulation is enabled by default since kvm-84.
-- KVM --
recent fedora guests in KVM are unstable with kvm-clock and !constant_tsc
This was fixed a while back by an F10 kernel update, but there
was some confusion. It sounds like some people may be
experiencing other KVM hangs which are unrelated to this issue.
error when hot-unplugging a file-backed virtio disk
Device 'virtio2' does not have a release() function, it is
broken and must be fixed.
Reported against F10, but has since been fixed upstream/rawhide.
Network transfer from qemu guest machine stalls
A reported of a virtio_net related hang. May be similar to one
that was reported on kvm@vger previously.
Unable to run RHEL-5 Xen within KVM guest
Dan Berrange reports that F11 has regressed in its ability to be
able to run the Xen hypervisor inside a guest. It looks to be
related to KVM's in-kernel IOAPIC emulation.
Fedora11 Kernel BUG while installing kvm guest os using qcow2
Some IBM folks testing the F11 Alpha hit this oops during
installs. We had previously come to the conclusion that this was
caused by the pvmmu bug reported in #480822.
kvm: Oops while installing rawhide/x86_64 guest on F-10/x86_64 host
2.6.26-0.124.rc9.git5.fc10.x86_64 oops in new_slab in kvm guest
More dups of the the pvmmu issue that Marcelo previously fixed
windows server 2003 x64 guest does not poweroff
QEmu uses 100% of two assigned cores for XP guest
clalance reports that this issue seems fixed in F11.
qemu-kvm: samba/smb causes "bad page state" and/or freeze
F9 vintage KVM oops that seems to have gone away in F11.
KVM: Intel/VMX: host kernel should support DEBUGCTLMSR=0x1d9
KVM does not emulate last-branch-recording which causes
PTRACE_SINGLEBLOCK. Looks like the option is either to implement
LBR virtualization or have the guest kernel detect that it is
running on KVM and that the MSR isn't supported.
Coredump on start when no kvm modules is loaded
Looks like a bug which was fixed in kvm-80.
-- libvirt --
adding a new cd-rom device to a KVM guest fails in libvirt
Looks like a bug with cd-rom hotplug in libvirt.
Sound under KVM requires exclusive access to the sound device
Dan Berrange explains why sound has essentially been disabled
for now in KVM guests.
Ownership of serial console allow only root access
If you configure a serial console for a kvm guest in libvirt,
the console is only accessible by root ... even if it was a user
other than root that started it.
libvirt does not do safe replacement of guest/storage
pool/network config files
When libvirt writes configuration files to disk, it does not use
the recommended write-to-tmpfile/flush/sync/close/rename method
to ensure data safety.
libnuma: Warning: /sys not mounted or no numa system. Assuming
one node: No such file or directory
This annoying warning which I thought had long ago been removed
had only been removed on F10, not rawhide.
Useless "domain didn't show up" error when starting a guest with
too much RAM
It seems that if you assign too much RAM to a QEMU guest,
libvirt gives a very unhelpful error message.
F9 iozone w/ direct I/O performing higher than the raw device
Resolved by the option to set the disk caching mode in libvirt.
-- virt tools --
virt-manager (libvirt?) 0.6.1 fails to manage RHEL-5.3 machines
virt-manager was failing to properly handle old-style <console/>
when remotely connecting to a RHEL5 Xen host. Fix in the latest
version of virt-manager.
virt-viewer jumps workspaces on guest restart with virt-install
Because virt-viewer is run multiple times by virt-install, it
can mean that the window moves unexpectedly between workspaces.
virt-viewer no longer auto adjusts to guest screen size
Sounds like we have a screen resizing issue during installs with
virt-manager displays incorrect number of vcpus
Strange virt-manager bug artificially limiting the number of
vcpus allowed. May turn out to be a gtk bug?
Wrong Ok/Cancel button order in virt-manager cd-rom connect
Another minor virt-manager bug; easy fix.
GConf error when starting virt-manager-0.6.1-1.fc10 after "su"
(works with "su -")
A weird virt-manager startup failure turns out to be a known bug
ConsoleKit doesn't know about vncserver sessions
Because virt-manager now uses PolicyKit instead of console
helper, and because ConsoleKit doesn't see vncserver sessions,
virt-manager only opens read-only libvirt connections when
running under vncserver.
virt-manager has hardcoded Requires: PolicyKit-gnome
Cole fixed in rawhide to require PolicyKit-authentication-agent.
virt-manager/virtinst should not set a keymap unless one is explicitly requested
Sounds like keyboard mappings will work correctly because of
gtk-vnc scancode extension, so long as virtinst doesn't
explicitly set a keymap.
-- Xen --
kernel/lockdep.c warning in xen guest boot dmesg
Fixed upstream in 2.6.29-rc6-git5
-- Misc --
grub-install doesn't support virtio_blk devices
What looks like a simple patch to grub-install to make it