Is there such a thing available? So far I have been working with a Sheeva Plug which is ARMv5 (Marvell Kirkwood) based, so the armv5tel was a good match.
But this morning, my Genesi Efika Smartbook arrived in the post, which is ARMv7 (Freescale i.MX515 Cortex-A8), and I think there might be considerably more performance to be extracted from it if an ARMv7 compiled distro is used. I have found mentions of a F10 armv7l build for a Beagle Board, but I was hoping for a more recent distro. Is there such a thing available with a recent version of Fedora (ideally F12)? Or am I going to have to build one myself?
Thanks.
Gordan
On Wed, 2010-12-22 at 14:19 +0000, Gordan Bobic wrote:
Is there such a thing available? So far I have been working with a Sheeva Plug which is ARMv5 (Marvell Kirkwood) based, so the armv5tel was a good match.
But this morning, my Genesi Efika Smartbook arrived in the post, which is ARMv7 (Freescale i.MX515 Cortex-A8), and I think there might be considerably more performance to be extracted from it if an ARMv7 compiled distro is used. I have found mentions of a F10 armv7l build for a Beagle Board, but I was hoping for a more recent distro. Is there such a thing available with a recent version of Fedora (ideally F12)? Or am I going to have to build one myself?
Thanks.
Gordan
Hi Gordan,
There are two separate issues involved in going to armv7l from armv5tel -- first, optimizing for the arm v7 architecture, and secondly, switching from softfp (optional use of the FPU with fp function arguments passed in CPU registers) to hardfp (mandatory use of the FPU with fp function arguments passed in FPU registers).
I just recently got a report from a Seneca student studying the arm v5 vs. arm v7 (softfp) performance on a BeagleBoard xM, and his conclusion was that there is little or no appreciable performance gain.
There is an expected performance gain with the switch to hardfp, but that's a big challenge: you can't mix softfp and hardfp, so switching to hardfp is like bootstrapping an entirely new architecture. The other challenge is that the gcc used in Fedora does not yet support hardfp with vfp; using another compiler might be possible, but could lead to compatibility challenges with the primary arch.
Thus, my leaning is to hold off on the armv7l move until our gcc provides arm v7 hardfp+vfp, which is probably F15 territory.
-Chris
Chris Tyler wrote:
On Wed, 2010-12-22 at 14:19 +0000, Gordan Bobic wrote:
Is there such a thing available? So far I have been working with a Sheeva Plug which is ARMv5 (Marvell Kirkwood) based, so the armv5tel was a good match.
But this morning, my Genesi Efika Smartbook arrived in the post, which is ARMv7 (Freescale i.MX515 Cortex-A8), and I think there might be considerably more performance to be extracted from it if an ARMv7 compiled distro is used. I have found mentions of a F10 armv7l build for a Beagle Board, but I was hoping for a more recent distro. Is there such a thing available with a recent version of Fedora (ideally F12)? Or am I going to have to build one myself?
There are two separate issues involved in going to armv7l from armv5tel -- first, optimizing for the arm v7 architecture, and secondly, switching from softfp (optional use of the FPU with fp function arguments passed in CPU registers) to hardfp (mandatory use of the FPU with fp function arguments passed in FPU registers).
I just recently got a report from a Seneca student studying the arm v5 vs. arm v7 (softfp) performance on a BeagleBoard xM, and his conclusion was that there is little or no appreciable performance gain.
That is interesting. Presumably this is with a recent (CodeSourcery?) GCC? Is the lack of obvious improvement down to a GCC deficiency?
There is an expected performance gain with the switch to hardfp, but that's a big challenge: you can't mix softfp and hardfp
Ah - I hadn't realized that. I thought a hardfp binary could work on a softfp distro with a hardfp kernel.
so switching to hardfp is like bootstrapping an entirely new architecture.
Just out of interest, is there documentation on how to do that for RPM based distros? Or is it just he obvious packageless cross-build, then packaged re-build?
The other challenge is that the gcc used in Fedora does not yet support hardfp with vfp; using another compiler might be possible, but could lead to compatibility challenges with the primary arch.
Undoubtedly. I've been playing with rolling my own repository for ICC build x86 RPMs (because I have recently seen ICC produce up to 8x performance improvements compared to GCC produced code when it comes to number crunching). The performance is quite spectacular in some cases. On the other hand, I don't think anyone has yet managed to get glibc to build with anything other than GCC. One encouraging thing is that there is now some success with getting the kernel to build using ICC.
The reason I mention this is because given the relative maturity of ICC in this sense with such major outstanting issues, it will likely be some years before a whole distro will build with the likes of RVCT or TI compilers.
The performance improvements aren't as great compared to ICC on x86, but probably still worthwhile. But I'm not going to hold my breath.
Thus, my leaning is to hold off on the armv7l move until our gcc provides arm v7 hardfp+vfp, which is probably F15 territory.
I think you're right. Trying to build a GNU/Linux distro with something other than GCC isn't really plausible at the moment. Pitty - the lack of workable hardfp is really going to cripple things like my Efika and the Toshiba AC100.
Gordan
On 12/22/2010 06:30:13 AM, Chris Tyler wrote:
There is an expected performance gain with the switch to hardfp, but that's a big challenge: you can't mix softfp and hardfp, so switching to hardfp is like bootstrapping an entirely new architecture.
Since fedora has no problem mixing 32 and 64 bit executables it seems like there's alot of infrastructure already done for this. Just saying...
The other challenge is that the gcc used in Fedora does not yet support hardfp with vfp
ack. hard to work around that!
Andrew Burgess wrote:
On 12/22/2010 06:30:13 AM, Chris Tyler wrote:
There is an expected performance gain with the switch to hardfp, but that's a big challenge: you can't mix softfp and hardfp, so switching to hardfp is like bootstrapping an entirely new architecture.
Since fedora has no problem mixing 32 and 64 bit executables it seems like there's alot of infrastructure already done for this. Just saying...
I've not looked at it all the way down to binary level, but I have a feeling the problem is NOT quite analogous to mixing x86 with x86_64 or x86 softfp/hardfp. But I'd love to hear otherwise. :)
Has anyone got any links to information on the subject of why missing softfp and hardfp on ARMv7 isn't as straightforward as on x86?
Gordan
On Wed, 2010-12-22 at 08:01 -0800, Andrew Burgess wrote:
On 12/22/2010 06:30:13 AM, Chris Tyler wrote:
There is an expected performance gain with the switch to hardfp, but that's a big challenge: you can't mix softfp and hardfp, so switching to hardfp is like bootstrapping an entirely new architecture.
Since fedora has no problem mixing 32 and 64 bit executables it seems like there's alot of infrastructure already done for this. Just saying...
It's not a Fedora infrastructure issue, the ABIs are incompatible. I wish you could mix'n'match but that doesn't look possible.
-Chris
Chris Tyler wrote:
On Wed, 2010-12-22 at 08:01 -0800, Andrew Burgess wrote:
On 12/22/2010 06:30:13 AM, Chris Tyler wrote:
There is an expected performance gain with the switch to hardfp, but that's a big challenge: you can't mix softfp and hardfp, so switching to hardfp is like bootstrapping an entirely new architecture.
Since fedora has no problem mixing 32 and 64 bit executables it seems like there's alot of infrastructure already done for this. Just saying...
It's not a Fedora infrastructure issue, the ABIs are incompatible. I wish you could mix'n'match but that doesn't look possible.
What about kernel level FPU emulation? Is there such a thing? I could have sworn there used to be something that could be used as such. And it might be possible to make it quite transparent if it isn't required (so you can always have it in the kernel). Have the kernel trap the exception on the missing FPU instructions, save state, and then pass the to an emulation library. When that returns, restore state and resume.
It would mean much slower performance than softfp on FPU-less hardware, but it would allow for a transitional period where only VFP distro needs to be maintained, which still works (albeit slowly) on a non-VFP processor.
Then again, I'm not volunteering to add such a feature to the Linux kernel. :)
Gordan
Gordan Bobic wrote:
Chris Tyler wrote:
On Wed, 2010-12-22 at 08:01 -0800, Andrew Burgess wrote:
On 12/22/2010 06:30:13 AM, Chris Tyler wrote:
There is an expected performance gain with the switch to hardfp, but that's a big challenge: you can't mix softfp and hardfp, so switching to hardfp is like bootstrapping an entirely new architecture.
Since fedora has no problem mixing 32 and 64 bit executables it seems like there's alot of infrastructure already done for this. Just saying...
It's not a Fedora infrastructure issue, the ABIs are incompatible. I wish you could mix'n'match but that doesn't look possible.
What about kernel level FPU emulation? Is there such a thing? I could have sworn there used to be something that could be used as such. And it might be possible to make it quite transparent if it isn't required (so you can always have it in the kernel). Have the kernel trap the exception on the missing FPU instructions, save state, and then pass the to an emulation library. When that returns, restore state and resume.
It would mean much slower performance than softfp on FPU-less hardware, but it would allow for a transitional period where only VFP distro needs to be maintained, which still works (albeit slowly) on a non-VFP processor.
Then again, I'm not volunteering to add such a feature to the Linux kernel. :)
And just as I hit send, I found this: http://netwinder.osuosl.org/users/s/scottb/public_html/notes/FP-Notes-all.ht...
No idea how relevant it might still be - the article is about 11 years old.
Gordan
Quoting Gordan Bobic gordan@bobich.net:
What about kernel level FPU emulation? Is there such a thing? I could have sworn there used to be something that could be used as such. And it might be possible to make it quite transparent if it isn't required (so you can always have it in the kernel). Have the kernel trap the exception on the missing FPU instructions, save state, and then pass the to an emulation library. When that returns, restore state and resume.
There is already kernel level fpu emulation for x86 and there is already vfp, fpu, and even neon in qemu. While I don't think it would be trivial to do, it doesn't sound quite as hard because you aren't going to have to reverse engineer the whole thing. I'm not volunteering either as I assume you mean -working- fpu/vfp support. :)
On 12/22/2010 08:40:23 AM, Gordan Bobic wrote:
It's not a Fedora infrastructure issue
all i meant is that that part is solved, getting the correct shared libraries loading with the correct executables.
, the ABIs are incompatible. I
wish you could mix'n'match but that doesn't look possible.
What about kernel level FPU emulation? Is there such a thing? I could have sworn there used to be something that could be used as such. And it might be possible to make it quite transparent if it isn't required (so you can always have it in the kernel). Have the kernel trap the exception on the missing FPU instructions, save state, and then pass the to an emulation library. When that returns, restore state and resume.
i was only thinking of an arm7 kernel handling arm5 executables so no emulation should be needed. so like for NEON now, a NEON program wont work on hardware without NEON.
i assume the arm5 instruction set is upward compatible so it would seem even easier than an existing x86_64 kernel running 32bit code which is not binary compatible.
are you sure the fp arg style is a kernel issue? are there system calls that take floats (not in structs or arrays but as a single argument)?
i could be totally wrong. it just seems almost there...
Quoting Andrew Burgess aab@cichlid.com:
i assume the arm5 instruction set is upward compatible so it would seem even easier than an existing x86_64 kernel running 32bit code which is not binary compatible.
are you sure the fp arg style is a kernel issue? are there system calls that take floats (not in structs or arrays but as a single argument)?
i could be totally wrong. it just seems almost there...
What you are saying is that it should work more akin to the i386, i586 scenario. Where you recompiled the kernel for your arch (i586) and all the i386 packages just worked. But you got the speed bump from the kernel from the recompile. Then you recompiled everything you used heavily as i586 (if it would compile).
What they are saying is this doesnt work because of the ABI.
I vaguely recall (and maybe incorrectly) the original fpu emulation in the linux kernel (ie .99) had this issue too. Im thinking both glibc and the kernel emulation had to be fixed before this worked correctly.
You can always try it. :P
On Wed, Dec 22, 2010 at 5:19 PM, Andrew Burgess aab@cichlid.com wrote:
On 12/22/2010 08:40:23 AM, Gordan Bobic wrote:
It's not a Fedora infrastructure issue
all i meant is that that part is solved, getting the correct shared libraries loading with the correct executables.
Its not really as x86-64 can happily run i686 binaries with appropriate libraries etc side by side, that's not the case with softfp vs hardfp
, the ABIs are incompatible. I
wish you could mix'n'match but that doesn't look possible.
What about kernel level FPU emulation? Is there such a thing? I could have sworn there used to be something that could be used as such. And it might be possible to make it quite transparent if it isn't required (so you can always have it in the kernel). Have the kernel trap the exception on the missing FPU instructions, save state, and then pass the to an emulation library. When that returns, restore state and resume.
i was only thinking of an arm7 kernel handling arm5 executables so no emulation should be needed. so like for NEON now, a NEON program wont work on hardware without NEON.
That is possible if you use arm7 with softfp but then you don't get much advantage of using that. To get NEON programs to be optimised but still run on HW without Neon you need multiple code paths and run time detection. My understanding is the way the application compile FP support you either get hardfp instructions or softfp but not a means of detecting and changing on the fly.
i assume the arm5 instruction set is upward compatible so it would seem even easier than an existing x86_64 kernel running 32bit code which is not binary compatible.
Its not that the instruction set isn't compatible its how the compiler optimised the instructions for explicit hardfp or softfp. The resulting code can't fall back.
are you sure the fp arg style is a kernel issue? are there system calls that take floats (not in structs or arrays but as a single argument)?
From all the documentation about it seems so. You need to remember
there's a lot of mainline distros (and I'm not talking about small embedded ones) looking to move to arm7 + hardfp but none that have yet done so. There's lots of notes and people that have done customer recompiles etc for testing and eval purposes but none that have actually done it yet across the entire mainline distro to run everything from apache to gnome.
i could be totally wrong. it just seems almost there...
Its is but that doesn't mean there's still not a lot of work to do. The Fedora ARM team has had issues with gcc / glibc etc just trying to get Fedora 13 compiled on on arm5tel. I've seen mention in the thread "but it works great with ICC" but you have to remember Fedora is an open distro so we're not going to be using a closed compiler to fix the problem. At the moment we're concentrating on arm5tel as it works and we need to get Fedora 14 and rawhide there and its the combination that can support the broadest amount of hardware, one that's on the road to completion people can and will start looking at a version optimised for newer more power HW. Its only in the last week or so the team have managed to get enough of F-13 built to be able to compose a rootfs for testing it.
Peter
Peter Robinson wrote:
are you sure the fp arg style is a kernel issue? are there system calls that take floats (not in structs or arrays but as a single argument)?
From all the documentation about it seems so. You need to remember
there's a lot of mainline distros (and I'm not talking about small embedded ones) looking to move to arm7 + hardfp but none that have yet done so. There's lots of notes and people that have done customer recompiles etc for testing and eval purposes but none that have actually done it yet across the entire mainline distro to run everything from apache to gnome.
i could be totally wrong. it just seems almost there...
Its is but that doesn't mean there's still not a lot of work to do. The Fedora ARM team has had issues with gcc / glibc etc just trying to get Fedora 13 compiled on on arm5tel. I've seen mention in the thread "but it works great with ICC" but you have to remember Fedora is an open distro so we're not going to be using a closed compiler to fix the problem.
ICC is x86 only. I mentioned it to illustrate that there is so much GCC specific stuff in important core libraries like glibc that it is unlikely we will get it working with any other compiler any time soon. There are actually two problems regarding this:
1) People shouldn't be writing code that isn't portable across compilers. 2) GCC isn't a very good compiler performance-wise - not by a long way.
Once the GCC hardfp issue is resolved (I'm not going to hold my breath - I've moaned in the past about various areas of GCC's uselessness (e.g. vectorization) and in the past 3 years no meaningful movement has been achieved to close the <= 800% FPU performance gap with ICC), I suspect it'll lead to having two ARM distros. armv5tel for ARMs without FPU and arm7vfp for FPU enabled ARMs. Similar to what we currently have with x86 and x86_64, only if the ABI is different, arm5tel and arm7 will be pretty much distinct separate architectures in terms of software target.
At the moment we're concentrating on arm5tel as it works and we need to get Fedora 14 and rawhide there and its the combination that can support the broadest amount of hardware, one that's on the road to completion people can and will start looking at a version optimised for newer more power HW. Its only in the last week or so the team have managed to get enough of F-13 built to be able to compose a rootfs for testing it.
Ooo! Any chance of a link to this rootfs and the F13 arm5tel yum repository?
Gordan
On Thu, Dec 23, 2010 at 12:59 PM, Gordan Bobic gordan@bobich.net wrote:
Peter Robinson wrote:
are you sure the fp arg style is a kernel issue? are there system calls that take floats (not in structs or arrays but as a single argument)?
From all the documentation about it seems so. You need to remember
there's a lot of mainline distros (and I'm not talking about small embedded ones) looking to move to arm7 + hardfp but none that have yet done so. There's lots of notes and people that have done customer recompiles etc for testing and eval purposes but none that have actually done it yet across the entire mainline distro to run everything from apache to gnome.
i could be totally wrong. it just seems almost there...
Its is but that doesn't mean there's still not a lot of work to do. The Fedora ARM team has had issues with gcc / glibc etc just trying to get Fedora 13 compiled on on arm5tel. I've seen mention in the thread "but it works great with ICC" but you have to remember Fedora is an open distro so we're not going to be using a closed compiler to fix the problem.
ICC is x86 only. I mentioned it to illustrate that there is so much GCC specific stuff in important core libraries like glibc that it is unlikely we will get it working with any other compiler any time soon. There are actually two problems regarding this:
I thought you were referring to ImageCraft C Compiler which arm and others http://www.imagecraft.com/
- People shouldn't be writing code that isn't portable across compilers.
I don't disagree.
- GCC isn't a very good compiler performance-wise - not by a long way.
It doesn't matter, its currently the core compiler that Fedora uses and hence it what we need to get to work. When upstream Fedora changes to something else so can we but I don't see that happening in the short to medium term. In fact I suspect that its more likely to get upstream gcc integrate some LLVM perf improvements before that happens.
Once the GCC hardfp issue is resolved (I'm not going to hold my breath - I've moaned in the past about various areas of GCC's uselessness (e.g. vectorization) and in the past 3 years no meaningful movement has been achieved to close the <= 800% FPU performance gap with ICC), I suspect it'll lead to having two ARM distros. armv5tel for ARMs without FPU and arm7vfp for FPU enabled ARMs. Similar to what we currently have with x86 and x86_64, only if the ABI is different, arm5tel and arm7 will be pretty much distinct separate architectures in terms of software target.
I think its not far away. Moaning doesn't fix problems but now that there's a number of mainline distros looking at it and putting $$$ and human resources into the problems its now moving faster. Both MeeGo and Ubuntu are pushing these efforts via Linaro in combination with a number of hardware vendors. MeeGo plans on having support for it for the MeeGo 1.2 release due in April 2011 and they're "upstream first" so there's already reasonable movement on that.
As for the two branches of arm for armv5tel and arm7+hardfp that has already been discussed and is considered worthwhile and when people get time and the issues with gcc/glibc and other core dependant libraries are fixed it will happen. Again moaning isn't going to get that happening. So what you say is nothing new and well discussed. Most distros are going that way. MeeGo will in the 1.2 release, Ubuntu is going that way, I know Debian is investigating it as it Fedora.
At the moment we're concentrating on arm5tel as it works and we need to get Fedora 14 and rawhide there and its the combination that can support the broadest amount of hardware, one that's on the road to completion people can and will start looking at a version optimised for newer more power HW. Its only in the last week or so the team have managed to get enough of F-13 built to be able to compose a rootfs for testing it.
Ooo! Any chance of a link to this rootfs and the F13 arm5tel yum repository?
It was on planet Fedora in the last week or so posted by Paul W (of Fedora ARM fame @ Seneca college). It might be linked on the Fedora ARM wiki page too (sorry, don't have my browser history to hand).
Peter
Peter Robinson wrote:
are you sure the fp arg style is a kernel issue? are there system calls that take floats (not in structs or arrays but as a single argument)? From all the documentation about it seems so. You need to remember
there's a lot of mainline distros (and I'm not talking about small embedded ones) looking to move to arm7 + hardfp but none that have yet done so. There's lots of notes and people that have done customer recompiles etc for testing and eval purposes but none that have actually done it yet across the entire mainline distro to run everything from apache to gnome.
i could be totally wrong. it just seems almost there...
Its is but that doesn't mean there's still not a lot of work to do. The Fedora ARM team has had issues with gcc / glibc etc just trying to get Fedora 13 compiled on on arm5tel. I've seen mention in the thread "but it works great with ICC" but you have to remember Fedora is an open distro so we're not going to be using a closed compiler to fix the problem.
ICC is x86 only. I mentioned it to illustrate that there is so much GCC specific stuff in important core libraries like glibc that it is unlikely we will get it working with any other compiler any time soon. There are actually two problems regarding this:
I thought you were referring to ImageCraft C Compiler which arm and others http://www.imagecraft.com/
No, I was refering to Intel C Compiler: http://software.intel.com/en-us/articles/intel-compilers/
- People shouldn't be writing code that isn't portable across compilers.
I don't disagree.
- GCC isn't a very good compiler performance-wise - not by a long way.
It doesn't matter, its currently the core compiler that Fedora uses and hence it what we need to get to work. When upstream Fedora changes to something else so can we but I don't see that happening in the short to medium term. In fact I suspect that its more likely to get upstream gcc integrate some LLVM perf improvements before that happens.
Maybe, maybe not. I'm planning to put up a yum repository of ICC-built RHEL6 RPMs.
Not everything will build.
Not everything that builds will work.
Not everything that works will necessarily bring performance advantages - for example, on RHEL5, after extensive testing and a lot of attempts at tweaks, perl's performance just wouldn't budge - it's as slow when built using ICC as with GCC, and it took a lot of trial and error to just get it to build at all, let alone pass the test suite.
But of the things that do, build and work, the improvement is definitely worth having.
Of course I don't expect Fedora to switch to a commercial, non-GPL compiler. But considering that GCCs performance has degraded as often as it has improved between releases, I'm not too hopeful about the gap closing any time soon.
At the moment we're concentrating on arm5tel as it works and we need to get Fedora 14 and rawhide there and its the combination that can support the broadest amount of hardware, one that's on the road to completion people can and will start looking at a version optimised for newer more power HW. Its only in the last week or so the team have managed to get enough of F-13 built to be able to compose a rootfs for testing it.
Ooo! Any chance of a link to this rootfs and the F13 arm5tel yum repository?
It was on planet Fedora in the last week or so posted by Paul W (of Fedora ARM fame @ Seneca college). It might be linked on the Fedora ARM wiki page too (sorry, don't have my browser history to hand).
Thanks, that was enough to find it. :) http://paulfedora.wordpress.com/2010/12/15/fedora-13-arm-alpha-root-file-sys...
Gordan
ons 2010-12-22 klockan 11:15 -0500 skrev Chris Tyler:
It's not a Fedora infrastructure issue, the ABIs are incompatible. I wish you could mix'n'match but that doesn't look possible.
Should be possible I think using multilib techniques.
This assuming a hardfp kernel can run softfp bnaries. If not then multilib won't help.
Regards Henrik
-
Andrew Burgess -
Chris Tyler -
Gordan Bobic -
Henrik Nordström -
omalleys＠msu.edu -
Peter Robinson