In F14 x86, we finally have the ability to add an ARM based VM via the libvirt management tool!! IE you can go through the GUI and set up an qemu based arm VM*.
It defaults to the integratorcp board. But our image/kernel for F12 is for the versatile board.)
The physical specs for the integrator board support a max of 256 megs of ram and only support the arm9 processors. I assume qemu only supports this, or at least I ran into issues when i tried to bump up the ram for the versatile boards.
I am -assuming- this isn't the fastest board/processor combo... Does anyone have a clue what is? Then we can change it upstream and make a f13-alpha image for it.
*since we don't seem to support kickstart, and we don't have an installer, you can really only import in an existing image. But hey it is better then what we had!
Sean
omalleys@msu.edu wrote:
In F14 x86, we finally have the ability to add an ARM based VM via the libvirt management tool!! IE you can go through the GUI and set up an qemu based arm VM*.
What's the performance like? Is there multi-core capability? I'm wondering if a 3+GHz Core2 can emulate an ARM fast enough to be faster than native. I very much doubt it, but it'd be a real boon for large cross-building.
It defaults to the integratorcp board. But our image/kernel for F12 is for the versatile board.)
This is something I'll be looking into soon. I really want to try to come up with the smallest possible number of kernels that will support the biggest range of devices. I'm not sure yet how plausible that is on the ARM, though.
The physical specs for the integrator board support a max of 256 megs of ram and only support the arm9 processors. I assume qemu only supports this, or at least I ran into issues when i tried to bump up the ram for the versatile boards.
I am -assuming- this isn't the fastest board/processor combo... Does anyone have a clue what is? Then we can change it upstream and make a f13-alpha image for it.
Is it armv5tel or something else? I can confirm that the f13 alpha image works fine on my Toshiba AC100. Haven't tried it on the Sheeva yet.
*since we don't seem to support kickstart,
What's wrong with kickstart? Doesn't it work?
and we don't have an installer, you can really only import in an existing image. But hey it is better then what we had!
My understanding is that the bigger problem is that a lot of ARM devices can't network boot. Some, like my AC100 don't even have an ethernet port (built in 3G, wifi, and USB ethernet bridge). Sheeva can TFTP boot a kernel, though.
The other complication is that boot-loaders are all over the place. Efika Genesi has a nice setup with a new uboot that looks for boot.scr in the root of the first partition and boots accordingly. I've not yet found any docs on doing that on the sheeva, but I'm hoping to reverse engineer the setup for it from the Genesi laptop, maybe next weekend. The Toshiba AC100 uses something different again.
I have no idea whether any of them will boot off a CD, but that might be an interesting experiment. If they can (they all have USB ports, so USB CD-ROM drives are a distinct possibility), then that would give a reasonably universal way of installing, but there'd be a real mess of custom boot-loader setups for all the different devices. Not sure there's a way around that at the moment. I certainly plan to try to get all of mine working in the same way with the same boot loader, but I'm not sure if I will succeed.
Plus, it'd be _REALLY_ nice if uboot were able to provide a boot menu and boot multiple kernels via boot.scr. It'd make botched kernel images much less problematic. But I'm not sure if it contains enough functionality for that.
Gordan
On Thu, 2010-12-30 at 16:02 +0000, Gordan Bobic wrote:
omalleys@msu.edu wrote:
In F14 x86, we finally have the ability to add an ARM based VM via the libvirt management tool!! IE you can go through the GUI and set up an qemu based arm VM*.
What's the performance like? Is there multi-core capability? I'm wondering if a 3+GHz Core2 can emulate an ARM fast enough to be faster than native. I very much doubt it, but it'd be a real boon for large cross-building.
In tests under F12 and F13, QEMU ARM performed at about 10% of the speed of native hardware when building packages (Intel Q6600 quadcore vs. Sheevaplug).
-Chris
On Thu, 2010-12-30 at 10:47 -0500, omalleys@msu.edu wrote:
In F14 x86, we finally have the ability to add an ARM based VM via the libvirt management tool!! IE you can go through the GUI and set up an qemu based arm VM*.
It defaults to the integratorcp board. But our image/kernel for F12 is for the versatile board.)
The F12 ARM project doesn't provide a kernel, and the rootfs is pretty much board-agnostic, so you just need to use/supply a suitable kernel.
I haven't used QEMU ARM under F14, but under F12-F13 you could do an initial manual configuration after which you can use manage it using virt-manager:
http://fedoraproject.org/wiki/Architectures/ARM/HowToQemu#Using_QEMU_with_li...
In that context, the versatile kernel worked fine.
(Would you update that wiki page with F14 setup instructions?)
*since we don't seem to support kickstart, and we don't have an installer, you can really only import in an existing image. But hey it is better then what we had!
The concept of an "installer" in ARM land is pretty limited. There is a graphical installer for Sheeva/Guruplugs, but it's primarily a tool for installing a kernel and prebuilt rootfs (it would be hard to package for Fedora, too, since it contains an entire prebuilt Linux image for use during bootstrapping, and building that from source would be Big).
-Chris
Chris Tyler wrote:
*since we don't seem to support kickstart, and we don't have an installer, you can really only import in an existing image. But hey it is better then what we had!
The concept of an "installer" in ARM land is pretty limited. There is a graphical installer for Sheeva/Guruplugs, but it's primarily a tool for installing a kernel and prebuilt rootfs (it would be hard to package for Fedora, too, since it contains an entire prebuilt Linux image for use during bootstrapping, and building that from source would be Big).
Sheeva can get a kernel image to boot via TFTP, so making an "installer" kickstart a base image shouldn't be too difficult. the problem is booting the installer in the first place.
Having said that - it would be much, much more efficient if the Fedora/RH installer installed the minimum build from a single tar ball (i.e. the rootfs). The extra disk I/O required by maintaining the RPM database at every step really slows things down on cheap flash, not to mention that it wastes an awful lot of life of a SD/CF card with all those small writes. Installing a gig of packages onto an SD card on the Sheeva can take literally hours. The only way I have managed to make this sort of thing bearable is by LD_PRELOAD=libeatmydata.so before I run yum.
Gordan
On Thu, 2010-12-30 at 16:51 +0000, Gordan Bobic wrote:
Chris Tyler wrote:
*since we don't seem to support kickstart, and we don't have an installer, you can really only import in an existing image. But hey it is better then what we had!
The concept of an "installer" in ARM land is pretty limited. There is a graphical installer for Sheeva/Guruplugs, but it's primarily a tool for installing a kernel and prebuilt rootfs (it would be hard to package for Fedora, too, since it contains an entire prebuilt Linux image for use during bootstrapping, and building that from source would be Big).
Sheeva can get a kernel image to boot via TFTP, so making an "installer" kickstart a base image shouldn't be too difficult. the problem is booting the installer in the first place.
You could boot an installer via TFTP, but why? These systems aren't really strong for running software interactively -- you'd probably end up on the serial console (which you have to use to get the TFTP boot configured anyway).
I think that installing a rootfs via SD/microSD is a much lower-pain option, and the right way forward is unifying the rootfs tools across the various archs. A rootfs and a live CD/live USB image are pretty much the same thing; we're currently using different tools on different archs, and the current tools are broken in various ways (LiveUSB with overlays stops with no notice and unrecoverable errors once the overlay is full -- a big problem for Sugar on a Stick -- and the rootfs tools for the secondary archs are different, many of them not using kickstarts) -- it's time to refactor and unify.
I'm proposing a hackfest on this at FUDCon Tempe.
-Chris
On 12/30/2010 05:44 PM, Chris Tyler wrote:
On Thu, 2010-12-30 at 16:51 +0000, Gordan Bobic wrote:
Chris Tyler wrote:
*since we don't seem to support kickstart, and we don't have an installer, you can really only import in an existing image. But hey it is better then what we had!
The concept of an "installer" in ARM land is pretty limited. There is a graphical installer for Sheeva/Guruplugs, but it's primarily a tool for installing a kernel and prebuilt rootfs (it would be hard to package for Fedora, too, since it contains an entire prebuilt Linux image for use during bootstrapping, and building that from source would be Big).
Sheeva can get a kernel image to boot via TFTP, so making an "installer" kickstart a base image shouldn't be too difficult. the problem is booting the installer in the first place.
You could boot an installer via TFTP, but why? These systems aren't really strong for running software interactively -- you'd probably end up on the serial console (which you have to use to get the TFTP boot configured anyway).
Sure, for some things, but there is a growing number of netbooks based on ARM coming out, from the tiny ones with 128MB of RAM that go on ebay for £30, up to really decent ones like the Genesi Efika and Toshiba AC100. For these, an x86-style installer would easily be justifiable (if only their flash disk subsystem wasn't woefully slow - but as I said, that can be addressed by LD_PRELOAD-ing libeatmydata during the install, and syncing at the end. Or have the installer do the minimum install by extracting a tar ball and taking it from there with RPM for the extra packages.
I think that installing a rootfs via SD/microSD is a much lower-pain option, and the right way forward is unifying the rootfs tools across the various archs. A rootfs and a live CD/live USB image are pretty much the same thing; we're currently using different tools on different archs, and the current tools are broken in various ways (LiveUSB with overlays stops with no notice and unrecoverable errors once the overlay is full -- a big problem for Sugar on a Stick -- and the rootfs tools for the secondary archs are different, many of them not using kickstarts) -- it's time to refactor and unify.
Definitely agreeing on unifying the approach.
Gordan
Quoting Chris Tyler chris@tylers.info:
On Thu, 2010-12-30 at 10:47 -0500, omalleys@msu.edu wrote:
In F14 x86, we finally have the ability to add an ARM based VM via the libvirt management tool!! IE you can go through the GUI and set up an qemu based arm VM*.
It defaults to the integratorcp board. But our image/kernel for F12 is for the versatile board.)
The F12 ARM project doesn't provide a kernel, and the rootfs is pretty much board-agnostic, so you just need to use/supply a suitable kernel.
The point I am trying to get at is, not whether the it will work, but what is the fastest combo so we can make that the default target in the F14 virt manager and have a matching working image/install system for it.
I do update the wiki if I find something useful. I don't want to have to go back and re-figure everything out again.
There is a downloadable versatile kernel along with instructions on how to make it on the wiki.
The F12 image along with your workaround script do work in F14. Minus the fact when you stop the VM from the GUI it leaves a qemu process hanging. I think I got an armv7 kernel working with like the versatilea/b board, but I stopped there. I kind of lost patience with 5 hour kernel compiles and qemu crashing when I tried to add more then 128M of ram. I think there were some device issues, so some setting I was probably missing. Im not exactly an arm guru.
While real hardware is better, some of us don't have 2 devices to test with, and make rootfs and kernels with and test software with. I debated for months on whether a guruplug would be fast enough for my needs, and if i would have solely looked at the results of my qemu testing, I wouldn't have purchased one.
One instance is when I tried to do the mkrootfs-13 on the guruplug with the default software (apt-get yum rpmbuild etc) the checksums couldn't be verified because the versions of python were different and yum couldn't support the sha256 or whatever checksum algorithm. I had to set up a VM to do that.
If you can get performance similar or faster to actual hardware, there will be more interest. If it is pretty easy to set up a VM and add it to your build system, you might actually do it like we did for openafs. (which takes like 3 hours and you cannot get it to cross compile.) Virtio and 9P support are coming for qemu arm, which would be a nice default setting, but I don't think all of the patches have made it to the mainline kernel yet.
On 12/30/10 18:26, Somebody in the thread at some point said:
Hi -
One instance is when I tried to do the mkrootfs-13 on the guruplug with the default software (apt-get yum rpmbuild etc) the checksums couldn't be verified because the versions of python were different and yum couldn't support the sha256 or whatever checksum algorithm. I had to set up a VM to do that.
That sounds like the rpm change to xz compression. That's an unsually tough nut because pre-F12 rpmlib can't even understand post-F12 packages; normally this would work OK. IIRC they updated F11 rpm back at that time so it could cope with xz via fedora-updates yum repo, enabling a simple upgrade action. But I am not sure that was ever cooked for arm fedora.
The quickest solution is uplevel your main rootfs to the f12 tarball and do it from there with an rpm that understands xz already.
If you can get performance similar or faster to actual hardware, there will be more interest. If it is pretty easy to set up a VM and add it
Last time I looked at ARM qemu on quite a beefy Intel box I saw the same kind of 10% of ARM11 performance that Chris mentioned. Qmeu like that is just not useful for build duty, and real ARMs are getting faster much quicker than x86 iron is pretending to be an ARM so it never will be useful either.
-Andy
Quoting Andy Green andy@warmcat.com:
On 12/30/10 18:26, Somebody in the thread at some point said:
Hi -
One instance is when I tried to do the mkrootfs-13 on the guruplug with the default software (apt-get yum rpmbuild etc) the checksums couldn't be verified because the versions of python were different and yum couldn't support the sha256 or whatever checksum algorithm. I had to set up a VM to do that.
That sounds like the rpm change to xz compression. That's an unsually tough nut because pre-F12 rpmlib can't even understand post-F12 packages; normally this would work OK. IIRC they updated F11 rpm back at that time so it could cope with xz via fedora-updates yum repo, enabling a simple upgrade action. But I am not sure that was ever cooked for arm fedora.
The quickest solution is uplevel your main rootfs to the f12 tarball and do it from there with an rpm that understands xz already.
The fastest solution was to run the F12 vm since it took about a minute to install and get the script running.
If you can get performance similar or faster to actual hardware, there will be more interest. If it is pretty easy to set up a VM and add it
Last time I looked at ARM qemu on quite a beefy Intel box I saw the same kind of 10% of ARM11 performance that Chris mentioned. Qmeu like that is just not useful for build duty, and real ARMs are getting faster much quicker than x86 iron is pretending to be an ARM so it never will be useful either.
Are you saying you tried the ARM11 kernel with qemu F12 or you just tried the versatile kernel with the F12 qemu image off the wiki and it gave 10% ARM11 performance?
On 12/30/10 21:37, Somebody in the thread at some point said:
Quoting Andy Green andy@warmcat.com:
The quickest solution is uplevel your main rootfs to the f12 tarball and do it from there with an rpm that understands xz already.
The fastest solution was to run the F12 vm since it took about a minute to install and get the script running.
Well, it being a VM has nothing to do with it: you ran an F12 main rootfs as suggested and so avoided the xz problem.
If you can get performance similar or faster to actual hardware, there will be more interest. If it is pretty easy to set up a VM and add it
Last time I looked at ARM qemu on quite a beefy Intel box I saw the same kind of 10% of ARM11 performance that Chris mentioned. Qmeu like that is just not useful for build duty, and real ARMs are getting faster much quicker than x86 iron is pretending to be an ARM so it never will be useful either.
Are you saying you tried the ARM11 kernel with qemu F12 or you just tried the versatile kernel with the F12 qemu image off the wiki and it gave 10% ARM11 performance?
I tried to get started with qemu a couple of years ago and have forgotten what I ran, but as I recall it was a precooked image. Even booting the kernel was slow. I tried to compile an app using gcc so the emulator was having to do "real work".
At that time the alternative it was being compared to was a real 532MHz ARM11 with 128MB of DDR, it was able to compile "much" faster than the qemu box. This year there are Cortex A9 ARM boxes available are much faster than that and next year there'll be Cortex A15 ~2GHz quad core natively. Qemu and intel boxes aren't getting faster at that kind of rate any more.
-Andy
Quoting Andy Green andy@warmcat.com:
On 12/30/10 21:37, Somebody in the thread at some point said:
Quoting Andy Green andy@warmcat.com:
The quickest solution is uplevel your main rootfs to the f12 tarball and do it from there with an rpm that understands xz already.
The fastest solution was to run the F12 vm since it took about a minute to install and get the script running.
Well, it being a VM has nothing to do with it: you ran an F12 main rootfs as suggested and so avoided the xz problem.
If you can get performance similar or faster to actual hardware, there will be more interest. If it is pretty easy to set up a VM and add it
Last time I looked at ARM qemu on quite a beefy Intel box I saw the same kind of 10% of ARM11 performance that Chris mentioned. Qmeu like that is just not useful for build duty, and real ARMs are getting faster much quicker than x86 iron is pretending to be an ARM so it never will be useful either.
Are you saying you tried the ARM11 kernel with qemu F12 or you just tried the versatile kernel with the F12 qemu image off the wiki and it gave 10% ARM11 performance?
I tried to get started with qemu a couple of years ago and have forgotten what I ran, but as I recall it was a precooked image. Even booting the kernel was slow. I tried to compile an app using gcc so the emulator was having to do "real work".
At that time the alternative it was being compared to was a real 532MHz ARM11 with 128MB of DDR, it was able to compile "much" faster than the qemu box. This year there are Cortex A9 ARM boxes available are much faster than that and next year there'll be Cortex A15 ~2GHz quad core natively. Qemu and intel boxes aren't getting faster at that kind of rate any more.
No there is a huge performance difference from what I have seen. However I did get some boost by using a newer kernel, and by switching the boards.
I was just trying to determine whether or not I should bother try to test ARM11xx for qemu compilation speed or whether I should try something else.
On 12/30/10 22:28, Somebody in the thread at some point said:
At that time the alternative it was being compared to was a real 532MHz ARM11 with 128MB of DDR, it was able to compile "much" faster than the qemu box. This year there are Cortex A9 ARM boxes available are much faster than that and next year there'll be Cortex A15 ~2GHz quad core natively. Qemu and intel boxes aren't getting faster at that kind of rate any more.
No there is a huge performance difference from what I have seen. However I did get some boost by using a newer kernel, and by switching the boards.
I was just trying to determine whether or not I should bother try to test ARM11xx for qemu compilation speed or whether I should try something else.
I don't think it'll make much difference since what's making qemu slow will remain the same if it's emulating ARM9 or ARM11 or whatever.
But the bigger point is it'd be OK suffering somewhat with a slow qemu for a while if you know in the long run that the way you're doing it will win out and you gained experience with it. But it seems to me qemu is already way inferior to native ARM devices and the performance gap between qemu and native hardware is set to get dramatically worse in the next year or so.
The only benefit you can squeeze out of qemu that's hard with a real board is radical amounts of RAM, and I guess that too will be possible with real boards shortly at a reasonable price.
-Andy
Quoting Andy Green andy@warmcat.com:
On 12/30/10 22:28, Somebody in the thread at some point said:
At that time the alternative it was being compared to was a real 532MHz ARM11 with 128MB of DDR, it was able to compile "much" faster than the qemu box. This year there are Cortex A9 ARM boxes available are much faster than that and next year there'll be Cortex A15 ~2GHz quad core natively. Qemu and intel boxes aren't getting faster at that kind of rate any more.
No there is a huge performance difference from what I have seen. However I did get some boost by using a newer kernel, and by switching the boards.
I was just trying to determine whether or not I should bother try to test ARM11xx for qemu compilation speed or whether I should try something else.
I don't think it'll make much difference since what's making qemu slow will remain the same if it's emulating ARM9 or ARM11 or whatever.
But the bigger point is it'd be OK suffering somewhat with a slow qemu for a while if you know in the long run that the way you're doing it will win out and you gained experience with it. But it seems to me qemu is already way inferior to native ARM devices and the performance gap between qemu and native hardware is set to get dramatically worse in the next year or so.
The only benefit you can squeeze out of qemu that's hard with a real board is radical amounts of RAM, and I guess that too will be possible with real boards shortly at a reasonable price.
The ability to easily revert and test things. :)
I actually like VM's for testing and development. You just spin off a clone of your base dev environment for each project and a clone of a base image for your production environment. Then make your image, test it, then move it to production hardware. But I was doing a lot of solaris development and between gcc and Sun cc and various versions of make, autoconf, etc. It is just easier to waste the disk space and shut it off and archive it when you are done.
Granted in this case it is rather pokey. :)
On 12/30/10 18:26, Somebody in the thread at some point said:
Hi -
One instance is when I tried to do the mkrootfs-13 on the guruplug with the default software (apt-get yum rpmbuild etc) the checksums couldn't be verified because the versions of python were different and yum couldn't support the sha256 or whatever checksum algorithm. I had to set up a VM to do that.
That sounds like the rpm change to xz compression. That's an unsually tough nut because pre-F12 rpmlib can't even understand post-F12 packages; normally this would work OK. IIRC they updated F11 rpm back at that time so it could cope with xz via fedora-updates yum repo, enabling a simple upgrade action. But I am not sure that was ever cooked for arm fedora.
The quickest solution is uplevel your main rootfs to the f12 tarball and do it from there with an rpm that understands xz already.
If you can get performance similar or faster to actual hardware, there will be more interest. If it is pretty easy to set up a VM and add it
Last time I looked at ARM qemu on quite a beefy Intel box I saw the same kind of 10% of ARM11 performance that Chris mentioned. Qmeu like that is just not useful for build duty, and real ARMs are getting faster much quicker than x86 iron is pretending to be an ARM so it never will be useful either.
-Andy
On 12/30/10 18:26, Somebody in the thread at some point said:
Hi -
One instance is when I tried to do the mkrootfs-13 on the guruplug with the default software (apt-get yum rpmbuild etc) the checksums couldn't be verified because the versions of python were different and yum couldn't support the sha256 or whatever checksum algorithm. I had to set up a VM to do that.
That sounds like the rpm change to xz compression. That's an unsually tough nut because pre-F12 rpmlib can't even understand post-F12 packages; normally this would work OK. IIRC they updated F11 rpm back at that time so it could cope with xz via fedora-updates yum repo, enabling a simple upgrade action. But I am not sure that was ever cooked for arm fedora.
The quickest solution is uplevel your main rootfs to the f12 tarball and do it from there with an rpm that understands xz already.
If you can get performance similar or faster to actual hardware, there will be more interest. If it is pretty easy to set up a VM and add it
Last time I looked at ARM qemu on quite a beefy Intel box I saw the same kind of 10% of ARM11 performance that Chris mentioned. Qmeu like that is just not useful for build duty, and real ARMs are getting faster much quicker than x86 iron is pretending to be an ARM so it never will be useful either.
-Andy
-
Andy Green -
Chris Tyler -
Gordan Bobic -
omalleys＠msu.edu