Hi,
In case anyone is interested, I got this working on F13. It required building the cryptodev kernel module and rebuilding the standard F13 OpenSSL package with three additional parameters (the cryptodev support is already in the standard OpenSSL package sources, it just isn't enabled in the default build).
More details available here: http://www.altechnative.net/?p=174
Any chance we can have cryptodev enabled in the standard package build? I cannot see any drawbacks to having it available - when cryptodev device isn't there, it will simply fall back to the software implementation. (Note: required cryptodev header file provided by the external kernel driver).
Gordan
On Sun, May 22, 2011 at 2:11 AM, Gordan Bobic gordan@bobich.net wrote:
Hi,
In case anyone is interested, I got this working on F13. It required building the cryptodev kernel module and rebuilding the standard F13 OpenSSL package with three additional parameters (the cryptodev support is already in the standard OpenSSL package sources, it just isn't enabled in the default build).
More details available here: http://www.altechnative.net/?p=174
Any chance we can have cryptodev enabled in the standard package build? I cannot see any drawbacks to having it available - when cryptodev device isn't there, it will simply fall back to the software implementation. (Note: required cryptodev header file provided by the external kernel driver).
We use upstream Fedora mainline packages. File a bug and once its enabled in Fedora it will come to the ARM platform too.
Peter
On 05/22/2011 09:17 AM, Peter Robinson wrote:
On Sun, May 22, 2011 at 2:11 AM, Gordan Bobicgordan@bobich.net wrote:
Hi,
In case anyone is interested, I got this working on F13. It required building the cryptodev kernel module and rebuilding the standard F13 OpenSSL package with three additional parameters (the cryptodev support is already in the standard OpenSSL package sources, it just isn't enabled in the default build).
More details available here: http://www.altechnative.net/?p=174
Any chance we can have cryptodev enabled in the standard package build? I cannot see any drawbacks to having it available - when cryptodev device isn't there, it will simply fall back to the software implementation. (Note: required cryptodev header file provided by the external kernel driver).
We use upstream Fedora mainline packages. File a bug and once its enabled in Fedora it will come to the ARM platform too.
Filed: https://bugzilla.redhat.com/show_bug.cgi?id=706706
Gordan
Quoting Gordan Bobic gordan@bobich.net:
On 05/22/2011 09:17 AM, Peter Robinson wrote:
On Sun, May 22, 2011 at 2:11 AM, Gordan Bobicgordan@bobich.net wrote:
Hi,
In case anyone is interested, I got this working on F13. It required building the cryptodev kernel module and rebuilding the standard F13 OpenSSL package with three additional parameters (the cryptodev support is already in the standard OpenSSL package sources, it just isn't enabled in the default build).
More details available here: http://www.altechnative.net/?p=174
Any chance we can have cryptodev enabled in the standard package build? I cannot see any drawbacks to having it available - when cryptodev device isn't there, it will simply fall back to the software implementation. (Note: required cryptodev header file provided by the external kernel driver).
We use upstream Fedora mainline packages. File a bug and once its enabled in Fedora it will come to the ARM platform too.
That just rocks, Thanks!!
Sean
omalleys@msu.edu wrote:
Quoting Gordan Bobic gordan@bobich.net:
On 05/22/2011 09:17 AM, Peter Robinson wrote:
On Sun, May 22, 2011 at 2:11 AM, Gordan Bobicgordan@bobich.net wrote:
Hi,
In case anyone is interested, I got this working on F13. It required building the cryptodev kernel module and rebuilding the standard F13 OpenSSL package with three additional parameters (the cryptodev support is already in the standard OpenSSL package sources, it just isn't enabled in the default build).
More details available here: http://www.altechnative.net/?p=174
Any chance we can have cryptodev enabled in the standard package build? I cannot see any drawbacks to having it available - when cryptodev device isn't there, it will simply fall back to the software implementation. (Note: required cryptodev header file provided by the external kernel driver).
We use upstream Fedora mainline packages. File a bug and once its enabled in Fedora it will come to the ARM platform too.
That just rocks, Thanks!!
Yeah, it's pretty awesome. It makes the Sheevaplug catch up with the Atom that is 466MHz faster and 4x more power-hungry.
What I'm pondering now is something like a dkms package for the cryptodev kernel module, but I seem to remember reading somewhere that dkms is a non-Fedora RHEL thing. What do you guys think would be the best way to approach it, especially since we don't have "standard" kernels at the moment?
Gordan
Quoting Gordan Bobic gordan@bobich.net:
omalleys@msu.edu wrote:
Quoting Gordan Bobic gordan@bobich.net:
On 05/22/2011 09:17 AM, Peter Robinson wrote:
On Sun, May 22, 2011 at 2:11 AM, Gordan Bobicgordan@bobich.net wrote:
Hi,
In case anyone is interested, I got this working on F13. It required building the cryptodev kernel module and rebuilding the standard F13 OpenSSL package with three additional parameters (the cryptodev support is already in the standard OpenSSL package sources, it just isn't enabled in the default build).
More details available here: http://www.altechnative.net/?p=174
Any chance we can have cryptodev enabled in the standard package build? I cannot see any drawbacks to having it available - when cryptodev device isn't there, it will simply fall back to the software implementation. (Note: required cryptodev header file provided by the external kernel driver).
We use upstream Fedora mainline packages. File a bug and once its enabled in Fedora it will come to the ARM platform too.
That just rocks, Thanks!!
Yeah, it's pretty awesome. It makes the Sheevaplug catch up with the Atom that is 466MHz faster and 4x more power-hungry.
What I'm pondering now is something like a dkms package for the cryptodev kernel module, but I seem to remember reading somewhere that dkms is a non-Fedora RHEL thing. What do you guys think would be the best way to approach it, especially since we don't have "standard" kernels at the moment?
Good question. Although I thought dkms support was recently added like F13.
My question, is how hard is this to implement the hardware support non-openssl programs. OpenAFS could use this as it can use a lot of DES encryption, but it uses its own DES implementation. It also happens to be the only one I can think of off the top of my head that uses its own implementation. It would be nice to have.
omalleys@msu.edu wrote:
In case anyone is interested, I got this working on F13. It required building the cryptodev kernel module and rebuilding the standard F13 OpenSSL package with three additional parameters (the cryptodev support is already in the standard OpenSSL package sources, it just isn't enabled in the default build).
More details available here: http://www.altechnative.net/?p=174
Any chance we can have cryptodev enabled in the standard package build? I cannot see any drawbacks to having it available - when cryptodev device isn't there, it will simply fall back to the software implementation. (Note: required cryptodev header file provided by the external kernel driver).
We use upstream Fedora mainline packages. File a bug and once its enabled in Fedora it will come to the ARM platform too.
That just rocks, Thanks!!
Yeah, it's pretty awesome. It makes the Sheevaplug catch up with the Atom that is 466MHz faster and 4x more power-hungry.
What I'm pondering now is something like a dkms package for the cryptodev kernel module, but I seem to remember reading somewhere that dkms is a non-Fedora RHEL thing. What do you guys think would be the best way to approach it, especially since we don't have "standard" kernels at the moment?
Good question. Although I thought dkms support was recently added like F13.
I thought there was a phylosophical issue with dkms on fedora, because it tracks the mainline kernel or something like that.
My question, is how hard is this to implement the hardware support non-openssl programs.
Not particularly hard if you're writing your own crypto implementation anyway, but there's a lot to be said for just linking against OpenSSL. It's probably safer to link against the library that has a lot of eyes on it than it is to implement your own.
OpenAFS could use this as it can use a lot of DES encryption, but it uses its own DES implementation. It also happens to be the only one I can think of off the top of my head that uses its own implementation. It would be nice to have.
Is there any technical reason why it couldn't be built against OpenSSL?
This could also be extended to other things, such as md5sum and sha1sum from coreutils. They use their own implementation rather than OpenSSL (presumably because for something as core as coreutils needs to be dependencyless). A couple of thoughts on that subject:
1) Should md5sum, sha1sum and similar be subject to the alternatives framework, and be substituted by wrappers that instead call "openssl md5" and "openssl sha1" respectively?
2) My testing shows that the coreutils software implementation is actually quicker on checksumming large files. Not a lot, mind you, but the difference is measurable (1.924s for sha1sum and 1.998s for openssl sha1 for a kernel tar.bz2 ball, for the best of three runs of each). But the sys+user time for sha1sum adds up to the wallclock total, whereas for the cryptodev accelerated openssl run, the sys+user is 0.620s, i.e. less than a third of wallclock.
This might actually be a debate worth having on the primary arch list (Note: I'm not subscribed to that one.) since most of the current generation x86 processors from Intel, AMD and Via have similar crypto offload features at least for AES.
Gordan
On 05/23/2011 08:12:13 AM, Gordan Bobic wrote:
- My testing shows that the coreutils software implementation is
actually quicker on checksumming large files. Not a lot, mind you, but the difference is measurable (1.924s for sha1sum and 1.998s for openssl sha1 for a kernel tar.bz2 ball, for the best of three runs of each). But the sys+user time for sha1sum adds up to the wallclock total, whereas for the cryptodev accelerated openssl run, the sys+user is 0.620s, i.e. less than a third of wallclock.
cryptodev probably used the CESA hardware. since it isnt using cpu time i guess its technically not a bug.
i wonder how much you could actually use the cpu for other things? would a little cpu bound program running at idle prio get work done during your benchmark? that might be a big argument for cryptodev...
or even run both openssl and coreutils in parallel; total bytes per second (and heat and power) should increase.
Andrew Burgess wrote:
On 05/23/2011 08:12:13 AM, Gordan Bobic wrote:
- My testing shows that the coreutils software implementation is
actually quicker on checksumming large files. Not a lot, mind you, but the difference is measurable (1.924s for sha1sum and 1.998s for openssl sha1 for a kernel tar.bz2 ball, for the best of three runs of each). But the sys+user time for sha1sum adds up to the wallclock total, whereas for the cryptodev accelerated openssl run, the sys+user is 0.620s, i.e. less than a third of wallclock.
cryptodev probably used the CESA hardware.
It is - the mv_cesa kernel process starts to show up in top when the crypto engine is being used.
since it isnt using cpu time i guess its technically not a bug.
I never said it was a bug, I was merely meaning to say that it is great that the CPU is freed up to do other stuff. :)
i wonder how much you could actually use the cpu for other things? would a little cpu bound program running at idle prio get work done during your benchmark? that might be a big argument for cryptodev...
or even run both openssl and coreutils in parallel; total bytes per second (and heat and power) should increase.
As far as I can tell - yes. If I'm churning without cryptoddev, the CPU sits at 0% idle. With cryptodev and large blocks the idle CPU time is typically > 70%. So the remaining 70% of CPU could indeed be used for something else, such as additional crypto if needs be (see the comments in the article linked earlier in this thread that point out this very possibility).
Gordan
On 05/23/2011 10:18:29 AM, Gordan Bobic wrote:
cryptodev probably used the CESA hardware.
It is - the mv_cesa kernel process starts to show up in top when the crypto engine is being used.
since it isnt using cpu time i guess its technically not a bug.
I never said it was a bug,
i didn't mean to imply that you had. the confusion was all mine.
If I'm churning without cryptoddev, the CPU sits at 0% idle. With cryptodev and large blocks the idle CPU time is typically > 70%. So the remaining 70% of CPU could indeed be used for something else, such as additional crypto if needs be (see the comments in the article linked earlier in this thread that point out this very possibility).
i missed the article reference. it would have saved me some googling :-)
here it is again for the similarly myopic: http://www.altechnative.net/?p=174
Gordan Bobic gordan@bobich.net writes:
I thought there was a phylosophical issue with dkms on fedora, because it tracks the mainline kernel or something like that.
Nope, DKMS builds a kernel module as necessary (either at boot time or at kernel update time). It doesn't care what kernel you're using, only that you have the kernel headers and a working compiler. The philosophical issue with DKMS is that it requires a build system and that your end users are plugging stuff into their kernel.
-derek
On 05/23/2011 04:12 PM, Gordan Bobic wrote:
omalleys@msu.edu wrote:
My question, is how hard is this to implement the hardware support non-openssl programs.
Not particularly hard if you're writing your own crypto implementation anyway, but there's a lot to be said for just linking against OpenSSL. It's probably safer to link against the library that has a lot of eyes on it than it is to implement your own.
OpenAFS could use this as it can use a lot of DES encryption, but it uses its own DES implementation. It also happens to be the only one I can think of off the top of my head that uses its own implementation. It would be nice to have.
gpg seems to use its own AES implementation that's slower than SSL's. It would certainly be nice to fix that to use acceleration.
Andrew.
On 05/24/2011 06:11 PM, Andrew Haley wrote:
On 05/23/2011 04:12 PM, Gordan Bobic wrote:
omalleys@msu.edu wrote:
My question, is how hard is this to implement the hardware support non-openssl programs.
Not particularly hard if you're writing your own crypto implementation anyway, but there's a lot to be said for just linking against OpenSSL. It's probably safer to link against the library that has a lot of eyes on it than it is to implement your own.
OpenAFS could use this as it can use a lot of DES encryption, but it uses its own DES implementation. It also happens to be the only one I can think of off the top of my head that uses its own implementation. It would be nice to have.
gpg seems to use its own AES implementation that's slower than SSL's. It would certainly be nice to fix that to use acceleration.
Sounds like it might be a good idea to post a feature request to the upstream bugzilla. Have you checked if there is a build option to make it link against OpenSSL instead of using the bundled crypto stack?
Gordan
On Tue, May 24, 2011 at 11:20, Gordan Bobic gordan@bobich.net wrote:
On 05/24/2011 06:11 PM, Andrew Haley wrote:
On 05/23/2011 04:12 PM, Gordan Bobic wrote:
omalleys@msu.edu wrote:
My question, is how hard is this to implement the hardware support non-openssl programs.
Not particularly hard if you're writing your own crypto implementation anyway, but there's a lot to be said for just linking against OpenSSL. It's probably safer to link against the library that has a lot of eyes on it than it is to implement your own.
OpenAFS could use this as it can use a lot of DES encryption, but it uses its own DES implementation. It also happens to be the only one I can think of off the top of my head that uses its own implementation. It would be nice to have.
gpg seems to use its own AES implementation that's slower than SSL's. It would certainly be nice to fix that to use acceleration.
Sounds like it might be a good idea to post a feature request to the upstream bugzilla. Have you checked if there is a build option to make it link against OpenSSL instead of using the bundled crypto stack?
There may be a license incompatibility. OpenSSL has an advertising clause in it I believe which makes it incompatible with various GPL unless an exception is given.
Gordan _______________________________________________ arm mailing list arm@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/arm
Have a look at this too:
http://fedoraproject.org/wiki/FedoraCryptoConsolidation
It was supposed to be the way forward regarding all this. Last I checked, it was only for applications, not libraries, though.
Adam
On Tue, May 24, 2011 at 13:32, Stephen John Smoogen smooge@gmail.com wrote:
On Tue, May 24, 2011 at 11:20, Gordan Bobic gordan@bobich.net wrote:
On 05/24/2011 06:11 PM, Andrew Haley wrote:
On 05/23/2011 04:12 PM, Gordan Bobic wrote:
omalleys@msu.edu wrote:
My question, is how hard is this to implement the hardware support non-openssl programs.
Not particularly hard if you're writing your own crypto implementation anyway, but there's a lot to be said for just linking against OpenSSL. It's probably safer to link against the library that has a lot of eyes on it than it is to implement your own.
OpenAFS could use this as it can use a lot of DES encryption, but it uses its own DES implementation. It also happens to be the only one I can think of off the top of my head that uses its own implementation. It would be nice to have.
gpg seems to use its own AES implementation that's slower than SSL's. It would certainly be nice to fix that to use acceleration.
Sounds like it might be a good idea to post a feature request to the upstream bugzilla. Have you checked if there is a build option to make it link against OpenSSL instead of using the bundled crypto stack?
There may be a license incompatibility. OpenSSL has an advertising clause in it I believe which makes it incompatible with various GPL unless an exception is given.
Gordan _______________________________________________ arm mailing list arm@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/arm
-- Stephen J Smoogen. "The core skill of innovators is error recovery, not failure avoidance." Randy Nelson, President of Pixar University. "Let us be kind, one to another, for most of us are fighting a hard battle." -- Ian MacLaren _______________________________________________ arm mailing list arm@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/arm
On 05/24/2011 06:37 PM, Adam Goode wrote:
Have a look at this too:
http://fedoraproject.org/wiki/FedoraCryptoConsolidation
It was supposed to be the way forward regarding all this. Last I checked, it was only for applications, not libraries, though.
Now that _IS_ interesting. So NSS is the consolidation target? I was under the impression that OpenSSL is a lot more prolific and it has more application level support, of the ones listed on the page above, at least the following:
1) Apache/mod_ssl (OK, this one's a tie since there's also mod_nss) 3) stunnel 5) wget 8) OpenLDAP 9) OpenSSH 14) postfix
So despite the effort to head in the direction of NSS, there are still more applications supporting OpenSSL than NSS out of the box (and some of those will build against either). IMO, that is a very good reason to go with OpenSSL, especially since pushing this sort of thing upstream can sometimes approximate herding cats.
But assuming I'm wasting my breath with that line of discussion, on a more pragmatic level - what is the NSS support for cryptodev like?
Gordan
On 24/05/11 19:39, Gordan Bobic wrote:
On 05/24/2011 06:37 PM, Adam Goode wrote:
Have a look at this too:
http://fedoraproject.org/wiki/FedoraCryptoConsolidation
It was supposed to be the way forward regarding all this. Last I checked, it was only for applications, not libraries, though.
Now that _IS_ interesting. So NSS is the consolidation target?
As an aside, it is *incredibly* annoying that "NSS" is used as the name for this, since NSS already has a different meaning on GNU/Linux systems: Name Service Switch. But anyway...
I was under the impression that OpenSSL is a lot more prolific and it has more application level support, of the ones listed on the page above, at least the following:
Perhaps, but its licence seems to be broken, so it can't be used as a general-purpose crypto library for Fedora.
Andrew.
Andrew Haley wrote:
On 24/05/11 19:39, Gordan Bobic wrote:
On 05/24/2011 06:37 PM, Adam Goode wrote:
Have a look at this too:
http://fedoraproject.org/wiki/FedoraCryptoConsolidation
It was supposed to be the way forward regarding all this. Last I checked, it was only for applications, not libraries, though.
[...]
I was under the impression that OpenSSL is a lot more prolific and it has more application level support, of the ones listed on the page above, at least the following:
Perhaps, but its licence seems to be broken, so it can't be used as a general-purpose crypto library for Fedora.
This doesn't seem to prevent it from being used as such, as far as I can see.
Gordan
* Gordan Bobic [25/05/2011 11:25] :
Perhaps, but its licence seems to be broken, so it can't be used as a general-purpose crypto library for Fedora.
This doesn't seem to prevent it from being used as such, as far as I can see.
Actually, it does: http://lwn.net/Articles/428111/
Emmanuel
Emmanuel Seyman wrote:
- Gordan Bobic [25/05/2011 11:25] :
Perhaps, but its licence seems to be broken, so it can't be used as a general-purpose crypto library for Fedora.
This doesn't seem to prevent it from being used as such, as far as I can see.
Actually, it does: http://lwn.net/Articles/428111/
That's all very theoretical and academic as long as half the distro is linking against OpenSSL anyway. We are where we are, and that will take time to change. Meanwhile, those of us living in the real world need to stick with the pragmatic approach of using what is there and works.
Gordan
On 05/25/2011 10:38 AM, Gordan Bobic wrote:
Emmanuel Seyman wrote:
- Gordan Bobic [25/05/2011 11:25] :
aph:
Perhaps, but its licence seems to be broken, so it can't be used as a general-purpose crypto library for Fedora.
This doesn't seem to prevent it from being used as such, as far as I can see.
Actually, it does: http://lwn.net/Articles/428111/
That's all very theoretical and academic as long as half the distro is linking against OpenSSL anyway. We are where we are, and that will take time to change. Meanwhile, those of us living in the real world need to stick with the pragmatic approach of using what is there and works.
I don't think it's merely theoretical. If there's one lesson we should have learned over the past few years, it's that licences really do matter. It looks as though Fedora is going the NSS route anyway, so presumably an OpenSSL-compatible library could be built on top of NSS, and the problem would go away.
Andrew.
Andrew Haley wrote:
On 05/25/2011 10:38 AM, Gordan Bobic wrote:
Emmanuel Seyman wrote:
- Gordan Bobic [25/05/2011 11:25] :
aph:
Perhaps, but its licence seems to be broken, so it can't be used as a general-purpose crypto library for Fedora.
This doesn't seem to prevent it from being used as such, as far as I can see.
Actually, it does: http://lwn.net/Articles/428111/
That's all very theoretical and academic as long as half the distro is linking against OpenSSL anyway. We are where we are, and that will take time to change. Meanwhile, those of us living in the real world need to stick with the pragmatic approach of using what is there and works.
I don't think it's merely theoretical. If there's one lesson we should have learned over the past few years, it's that licences really do matter.
It seems to me (and always has, thinking about it) that the argument of BSD vs. GPL vs. LGPL vs. whatever-other-open-source-licence has always been a passtime for people who lack either the ability or the inclination to get on with real productive work. Flame me for that statement all you want, but that's just what it's always looked like to me.
I don't see that this particular licensing "issue" is stopping OpenSSL shipping with every major distro, with a lot of packages linking against it. Hardlining on this issue seems like it blows things way out of proportion.
It looks as though Fedora is going the NSS route anyway, so presumably an OpenSSL-compatible library could be built on top of NSS, and the problem would go away.
I'm not against NSS - really I'm not. But there are other considerations to be taken into account.
1) Does NSS have any kind of support for hardware crypto offload? If so, I haven't found any references to it (but maybe my google-foo is weak today).
2) More abstraction (a OpenSSL->NSS shim library), means more bloat, more context switching and less performance. Is that really the way forward? I mean _really_?
3) Volume of supported commonly used software - if NSS has a clear advantage in terms of support base, then so be it, let's all put our weight behind it. But my perception is that this is not the case. Everything I touch upon on a daily basis seems to be linked against OpenSSL rather than NSS.
Gordan
On 05/25/2011 11:31 AM, Gordan Bobic wrote:
Andrew Haley wrote:
On 05/25/2011 10:38 AM, Gordan Bobic wrote:
Emmanuel Seyman wrote:
- Gordan Bobic [25/05/2011 11:25] :
aph:
Perhaps, but its licence seems to be broken, so it can't be used as a general-purpose crypto library for Fedora.
This doesn't seem to prevent it from being used as such, as far as I can see.
Actually, it does: http://lwn.net/Articles/428111/
That's all very theoretical and academic as long as half the distro is linking against OpenSSL anyway. We are where we are, and that will take time to change. Meanwhile, those of us living in the real world need to stick with the pragmatic approach of using what is there and works.
I don't think it's merely theoretical. If there's one lesson we should have learned over the past few years, it's that licences really do matter.
It seems to me (and always has, thinking about it) that the argument of BSD vs. GPL vs. LGPL vs. whatever-other-open-source-licence has always been a passtime for people who lack either the ability or the inclination to get on with real productive work. Flame me for that statement all you want, but that's just what it's always looked like to me.
OK, so that's how it looks to you.
I don't see that this particular licensing "issue" is stopping OpenSSL shipping with every major distro, with a lot of packages linking against it. Hardlining on this issue seems like it blows things way out of proportion.
The point is not that one licence is better than the other, but that some licences are incompatible. The "old" BSD licence with the advertising clause is broken because it prevents a library from being used by some packages. There is a new BSD licence that fixes the problem.
It looks as though Fedora is going the NSS route anyway, so presumably an OpenSSL-compatible library could be built on top of NSS, and the problem would go away.
Aha: I see http://fedoraproject.org/wiki/Nss_compat_ossl
I'm not against NSS - really I'm not. But there are other considerations to be taken into account.
- Does NSS have any kind of support for hardware crypto offload? If so,
I haven't found any references to it (but maybe my google-foo is weak today).
Interesting question; not sure.
- More abstraction (a OpenSSL->NSS shim library), means more bloat,
more context switching and less performance. Is that really the way forward? I mean _really_?
For bulk crypto operations an extra call via a shim probably doesn't matter. For some signature operations it might.
It seems like a clean solution from the point of view of application developers, though.
- Volume of supported commonly used software - if NSS has a clear
advantage in terms of support base, then so be it, let's all put our weight behind it. But my perception is that this is not the case. Everything I touch upon on a daily basis seems to be linked against OpenSSL rather than NSS.
Well, that's not true for everyone, and certainly not for users of GPG.
But the real question is whether one group of Fedora developers is determinedly going to push NSS and the other OpenSSL. That is not a route to happiness.
Andrew.
Andrew Haley wrote:
I'm not against NSS - really I'm not. But there are other considerations to be taken into account.
- Does NSS have any kind of support for hardware crypto offload? If so,
I haven't found any references to it (but maybe my google-foo is weak today).
Interesting question; not sure.
I think it is an important one to answer, and sooner rather than later. This is particularly important to the ARM community since a lot of (most?) ARMs seem to have a crypto co-processor of some description (Freescales, Kirkwood and Tegra definitely seem to have this, I haven't checked others, but since these are the three classes of devices I own, that's 3 out of 3 - I don't think it's luck/coincidence).
This is particularly important for server applications. ARM is getting some traction in the server market. ZT make a really nice (if expensive) multi-ARM server. I have seriously discussed using ShevaPlugs/GuruPlugs specifically for large scale-out low-power SSL offload with clients. Crypto offload support is already important, and it's importance is only going to go up.
- More abstraction (a OpenSSL->NSS shim library), means more bloat,
more context switching and less performance. Is that really the way forward? I mean _really_?
For bulk crypto operations an extra call via a shim probably doesn't matter. For some signature operations it might.
It seems like a clean solution from the point of view of application developers, though.
The other thing that needs to be considered is added complexity and security. I would imagine that since there is an abstraction layer, it introduces additional scope for exploits (buffer overruns, stack smashing, etc.) Is this shim library going to also be FIPS certified? If not then the improved security aspect of NSS vs. OpenSSL comes a lot closer to pure marketing rhetoric (maybe that's where it's at at the moment anyway, I don't claim to be an expert on the subject).
- Volume of supported commonly used software - if NSS has a clear
advantage in terms of support base, then so be it, let's all put our weight behind it. But my perception is that this is not the case. Everything I touch upon on a daily basis seems to be linked against OpenSSL rather than NSS.
Well, that's not true for everyone, and certainly not for users of GPG.
I thought it was mentioned on this thread recently that GPG brings it's own implementation anyway. Did I misunderstand?
But the real question is whether one group of Fedora developers is determinedly going to push NSS and the other OpenSSL. That is not a route to happiness.
It's not, but losing crypto offload and/or a performance drop-off and bloat due to shimming isn't a happy solution, either. If we can address those (the latter by sending patches to build against NSS upstream so shimming isn't required), then it'd be a great idea.
But purely in terms of standardizing on a single crypto library - has anyone actually performed an exhaustive analysis on how much would need to be changed to go either way? The wiki page that has been referenced a few times seems distinctly non-exhaustive. Maybe my perception is non-representative here, but as I said, all the things I get my hands dirty on a regular basis are linked against OpenSSL rather than NSS. The pragmatist in me says that maybe that makes OpenSSL a better target to standardize on, but I would like to see an exhaustive analysis / empirical evidence showing otherwise.
Gordan
On 05/25/2011 12:06 PM, Gordan Bobic wrote:
Andrew Haley wrote:
I'm not against NSS - really I'm not. But there are other considerations to be taken into account.
- Does NSS have any kind of support for hardware crypto offload? If so,
I haven't found any references to it (but maybe my google-foo is weak today).
Interesting question; not sure.
I think it is an important one to answer, and sooner rather than later. This is particularly important to the ARM community since a lot of (most?) ARMs seem to have a crypto co-processor of some description (Freescales, Kirkwood and Tegra definitely seem to have this, I haven't checked others, but since these are the three classes of devices I own, that's 3 out of 3 - I don't think it's luck/coincidence).
This is particularly important for server applications. ARM is getting some traction in the server market. ZT make a really nice (if expensive) multi-ARM server. I have seriously discussed using ShevaPlugs/GuruPlugs specifically for large scale-out low-power SSL offload with clients. Crypto offload support is already important, and it's importance is only going to go up.
Definitely, which is why it's important to focus on the right library going forward.
- More abstraction (a OpenSSL->NSS shim library), means more bloat,
more context switching and less performance. Is that really the way forward? I mean _really_?
For bulk crypto operations an extra call via a shim probably doesn't matter. For some signature operations it might.
It seems like a clean solution from the point of view of application developers, though.
The other thing that needs to be considered is added complexity and security. I would imagine that since there is an abstraction layer, it introduces additional scope for exploits (buffer overruns, stack smashing, etc.) Is this shim library going to also be FIPS certified? If not then the improved security aspect of NSS vs. OpenSSL comes a lot closer to pure marketing rhetoric (maybe that's where it's at at the moment anyway, I don't claim to be an expert on the subject).
Perhaps, but all this is just speculation AFAICS. If a shim is unnecessary then it should be avoided, obviously.
- Volume of supported commonly used software - if NSS has a clear
advantage in terms of support base, then so be it, let's all put our weight behind it. But my perception is that this is not the case. Everything I touch upon on a daily basis seems to be linked against OpenSSL rather than NSS.
Well, that's not true for everyone, and certainly not for users of GPG.
I thought it was mentioned on this thread recently that GPG brings it's own implementation anyway. Did I misunderstand?
It does, but I was responding to the claim that OpenSSL was being used for everything you touch on a daily basis. If that was not what you implied, I apologize. But if most crypto apps are using home-baked library code, then it doesn't matter which crypto library we move to because most apps will need to be ported anyway.
But the real question is whether one group of Fedora developers is determinedly going to push NSS and the other OpenSSL. That is not a route to happiness.
It's not, but losing crypto offload and/or a performance drop-off and bloat due to shimming isn't a happy solution, either.
It's a sight more happy than two groups of Fedora developers fighting over the One True Crypto Library. IMO. :-)
If we can address those (the latter by sending patches to build against NSS upstream so shimming isn't required), then it'd be a great idea.
Right.
But purely in terms of standardizing on a single crypto library - has anyone actually performed an exhaustive analysis on how much would need to be changed to go either way? The wiki page that has been referenced a few times seems distinctly non-exhaustive. Maybe my perception is non-representative here, but as I said, all the things I get my hands dirty on a regular basis are linked against OpenSSL rather than NSS. The pragmatist in me says that maybe that makes OpenSSL a better target to standardize on, but I would like to see an exhaustive analysis / empirical evidence showing otherwise.
I think it's a mistake to characterize one set of developers as more pragmatic than the other. If we end up with a highly-performant crypto library that some Fedora packages can't be linked against for legal reasons, that would be -- pragmatically speaking -- a Very Bad Thing.
Andrew.
Andrew Haley wrote:
- Volume of supported commonly used software - if NSS has a clear
advantage in terms of support base, then so be it, let's all put our weight behind it. But my perception is that this is not the case. Everything I touch upon on a daily basis seems to be linked against OpenSSL rather than NSS.
Well, that's not true for everyone, and certainly not for users of GPG.
I thought it was mentioned on this thread recently that GPG brings it's own implementation anyway. Did I misunderstand?
It does, but I was responding to the claim that OpenSSL was being used for everything you touch on a daily basis. If that was not what you implied, I apologize.
Well, you could just infer that I don't use GPG on a daily basis. :)
But if most crypto apps are using home-baked library code, then it doesn't matter which crypto library we move to because most apps will need to be ported anyway.
As far as I can tell, GPG is rather an exception. Most things don't bother re-inventing the wheel.
But the real question is whether one group of Fedora developers is determinedly going to push NSS and the other OpenSSL. That is not a route to happiness.
It's not, but losing crypto offload and/or a performance drop-off and bloat due to shimming isn't a happy solution, either.
It's a sight more happy than two groups of Fedora developers fighting over the One True Crypto Library. IMO. :-)
I don't really see either as an option. So the key point seems to be getting NSS working with the kernel's crypto offload features (whether it's cryptodev or netlink is irrelevant, as long as it works).
But purely in terms of standardizing on a single crypto library - has anyone actually performed an exhaustive analysis on how much would need to be changed to go either way? The wiki page that has been referenced a few times seems distinctly non-exhaustive. Maybe my perception is non-representative here, but as I said, all the things I get my hands dirty on a regular basis are linked against OpenSSL rather than NSS. The pragmatist in me says that maybe that makes OpenSSL a better target to standardize on, but I would like to see an exhaustive analysis / empirical evidence showing otherwise.
I think it's a mistake to characterize one set of developers as more pragmatic than the other. If we end up with a highly-performant crypto library that some Fedora packages can't be linked against for legal reasons, that would be -- pragmatically speaking -- a Very Bad Thing.
I agree. My concern is that this is looking like glibc vs. some-other-libc argument all over again, and years later that is still not resolved (all that come out of that, AFAICT is that dietlibc now ships as standard); and some things still don't work properly built against glibc (e.g. util-vserver). I can easily see all this remaining unresolved come Fedora 25 in 5 years' time. Unless we start outright banning things that don't conform, that is - and you wouldn't end up with a usable distro if you outright excluded OpenSSL dependants.
Gordan
I think this whole thread is getting a little out of hand. It was mentioned initially about using other libraries instead of built in routines in gpg.
I'm not against NSS - really I'm not. But there are other considerations to be taken into account.
- Does NSS have any kind of support for hardware crypto offload? If so,
I haven't found any references to it (but maybe my google-foo is weak today).
There's lots of reference to HW offload on NSS. For one its explicitly mentioned in referenced Fedora consolidation [1], in the NSS FAQ [2] and wikipedia [3] :-)
So its a matter of seeing how that integrates and fits with the the linux kernel crypto api as that would be the obvious place to add the support as if the HW is there it will get used and it would also get access to any cpu acceleration that is not crypto specific (I've read about using NEON for this for example).
[1] http://fedoraproject.org/wiki/FedoraCryptoConsolidation#Technology_selection [2] https://developer.mozilla.org/en/NSS_FAQ [3] http://en.wikipedia.org/wiki/Network_Security_Services#Hardware_support
Interesting question; not sure.
I think it is an important one to answer, and sooner rather than later. This is particularly important to the ARM community since a lot of (most?) ARMs seem to have a crypto co-processor of some description (Freescales, Kirkwood and Tegra definitely seem to have this, I haven't checked others, but since these are the three classes of devices I own, that's 3 out of 3 - I don't think it's luck/coincidence).
Remember this needs to be upstream in Fedora and other projects and then Fedora ARM will get it by default. Work upstream, propose it as a Fedora 16 feature and do the work.
I don't think its particularly important to ARM. It certainly helps on ARM due to low processing but its relevant on all platforms and most platforms have some form of crypto offload built in to the core CPU now days. Hell even the AMD geode processor used in the XO has it.
This is particularly important for server applications. ARM is getting some traction in the server market. ZT make a really nice (if expensive) multi-ARM server. I have seriously discussed using ShevaPlugs/GuruPlugs specifically for large scale-out low-power SSL offload with clients. Crypto offload support is already important, and it's importance is only going to go up.
Working in hosting myself with massive amounts of SSL offload we're not exactly looking at arm. There's lots of other issues here, and none of them are specific to ARM processors.
- More abstraction (a OpenSSL->NSS shim library), means more bloat,
more context switching and less performance. Is that really the way forward? I mean _really_?
For bulk crypto operations an extra call via a shim probably doesn't matter. For some signature operations it might.
It seems like a clean solution from the point of view of application developers, though.
The other thing that needs to be considered is added complexity and security. I would imagine that since there is an abstraction layer, it introduces additional scope for exploits (buffer overruns, stack smashing, etc.) Is this shim library going to also be FIPS certified? If not then the improved security aspect of NSS vs. OpenSSL comes a lot closer to pure marketing rhetoric (maybe that's where it's at at the moment anyway, I don't claim to be an expert on the subject).
Read the details mentioned on the advantage of NSS over OpenSSL for FIPS certification on the consolidation wiki.
- Volume of supported commonly used software - if NSS has a clear
advantage in terms of support base, then so be it, let's all put our weight behind it. But my perception is that this is not the case. Everything I touch upon on a daily basis seems to be linked against OpenSSL rather than NSS.
Well, that's not true for everyone, and certainly not for users of GPG.
I thought it was mentioned on this thread recently that GPG brings it's own implementation anyway. Did I misunderstand?
Yes, that's why we ended up in the discussion of types of libary!
But the real question is whether one group of Fedora developers is determinedly going to push NSS and the other OpenSSL. That is not a route to happiness.
It's not, but losing crypto offload and/or a performance drop-off and bloat due to shimming isn't a happy solution, either. If we can address those (the latter by sending patches to build against NSS upstream so shimming isn't required), then it'd be a great idea.
NSS supports HW. See above. All things need to be dealt with upstream. Whether it be Fedora, kernel, NSS or other apps.
But purely in terms of standardizing on a single crypto library - has anyone actually performed an exhaustive analysis on how much would need to be changed to go either way? The wiki page that has been referenced a few times seems distinctly non-exhaustive. Maybe my perception is non-representative here, but as I said, all the things I get my hands dirty on a regular basis are linked against OpenSSL rather than NSS. The pragmatist in me says that maybe that makes OpenSSL a better target to standardize on, but I would like to see an exhaustive analysis / empirical evidence showing otherwise.
The consolidation proposal was (I believe) instigated by the team(s) in Red Hat that deal with crypto and the various certifications required by an OS and applications that require things like FIPS cert, HW offload by customers that run platforms that require large encryption capabilities etc so I'm sure they've done a lot of work in that direction and made the proposal for a reason. I remember there was a lot of discussion on devel@ when it was proposed. I suggest you dig out those discussions.
As such support for crypto HW support in what ever libraries is better reported as a bug against the associated components. This list is about supporting ARM as a secondary architecture with in Fedora not about specific HW support in libraries. We will always use upstreams implementation of that. We have more than enough to do to support Fedora as a distro on ARM :-)
Peter
Peter Robinson wrote:
Interesting question; not sure.
I think it is an important one to answer, and sooner rather than later. This is particularly important to the ARM community since a lot of (most?) ARMs seem to have a crypto co-processor of some description (Freescales, Kirkwood and Tegra definitely seem to have this, I haven't checked others, but since these are the three classes of devices I own, that's 3 out of 3 - I don't think it's luck/coincidence).
Remember this needs to be upstream in Fedora and other projects and then Fedora ARM will get it by default. Work upstream, propose it as a Fedora 16 feature and do the work.
I don't think its particularly important to ARM. It certainly helps on ARM due to low processing but its relevant on all platforms and most platforms have some form of crypto offload built in to the core CPU now days. Hell even the AMD geode processor used in the XO has it.
Don't confuse crypto offload with crypto instructions. The new x86 stuff has crypto instructions that make AES churn faster, but that doesn't leave the CPU free to do other things while this is happening.
My big concern is that the pursued solution will be one size fits x86, as often happens (also why the same packages need ARM patches in consecutive releases).
- More abstraction (a OpenSSL->NSS shim library), means more bloat,
more context switching and less performance. Is that really the way forward? I mean _really_?
For bulk crypto operations an extra call via a shim probably doesn't matter. For some signature operations it might.
It seems like a clean solution from the point of view of application developers, though.
The other thing that needs to be considered is added complexity and security. I would imagine that since there is an abstraction layer, it introduces additional scope for exploits (buffer overruns, stack smashing, etc.) Is this shim library going to also be FIPS certified? If not then the improved security aspect of NSS vs. OpenSSL comes a lot closer to pure marketing rhetoric (maybe that's where it's at at the moment anyway, I don't claim to be an expert on the subject).
Read the details mentioned on the advantage of NSS over OpenSSL for FIPS certification on the consolidation wiki.
Are you talking about this sentence?
"NSS, in contrast, allows all applications to inherit the NSS FIPS validation status by following some simple rules detailed in the NSS security policy document."
I find it's a bit vague and opaque. I don't see how you could make the certification hereditary.
But the real question is whether one group of Fedora developers is determinedly going to push NSS and the other OpenSSL. That is not a route to happiness.
It's not, but losing crypto offload and/or a performance drop-off and bloat due to shimming isn't a happy solution, either. If we can address those (the latter by sending patches to build against NSS upstream so shimming isn't required), then it'd be a great idea.
NSS supports HW. See above. All things need to be dealt with upstream. Whether it be Fedora, kernel, NSS or other apps.
Fair enough. I won't hold my breath and give up my OpenSSL quite yet, then. :)
Gordan
Quoting Gordan Bobic gordan@bobich.net:
Peter Robinson wrote:
Interesting question; not sure.
I think it is an important one to answer, and sooner rather than later. This is particularly important to the ARM community since a lot of (most?) ARMs seem to have a crypto co-processor of some description (Freescales, Kirkwood and Tegra definitely seem to have this, I haven't checked others, but since these are the three classes of devices I own, that's 3 out of 3 - I don't think it's luck/coincidence).
Remember this needs to be upstream in Fedora and other projects and then Fedora ARM will get it by default. Work upstream, propose it as a Fedora 16 feature and do the work.
I don't think its particularly important to ARM. It certainly helps on ARM due to low processing but its relevant on all platforms and most platforms have some form of crypto offload built in to the core CPU now days. Hell even the AMD geode processor used in the XO has it.
Don't confuse crypto offload with crypto instructions. The new x86 stuff has crypto instructions that make AES churn faster, but that doesn't leave the CPU free to do other things while this is happening.
My big concern is that the pursued solution will be one size fits x86, as often happens (also why the same packages need ARM patches in consecutive releases).
From what I was reading in the kernel notes last night, the kernel crypto bits are optimized for multiple cores (multithreaded) and SEE3.1 enabled.
Given the kernel support for crypto, shouldn't this be in the kernel also?
- More abstraction (a OpenSSL->NSS shim library), means more bloat,
more context switching and less performance. Is that really the way forward? I mean _really_?
For bulk crypto operations an extra call via a shim probably doesn't matter. For some signature operations it might.
It seems like a clean solution from the point of view of application developers, though.
The other thing that needs to be considered is added complexity and security. I would imagine that since there is an abstraction layer, it introduces additional scope for exploits (buffer overruns, stack smashing, etc.) Is this shim library going to also be FIPS certified? If not then the improved security aspect of NSS vs. OpenSSL comes a lot closer to pure marketing rhetoric (maybe that's where it's at at the moment anyway, I don't claim to be an expert on the subject).
Read the details mentioned on the advantage of NSS over OpenSSL for FIPS certification on the consolidation wiki.
Are you talking about this sentence?
"NSS, in contrast, allows all applications to inherit the NSS FIPS validation status by following some simple rules detailed in the NSS security policy document."
I find it's a bit vague and opaque. I don't see how you could make the certification hereditary.
Think of something like DNSSEC, which hands your machine a certificate, then add something for your actual credentials, like another Cert on a smartcard, kerberos and/or OpenID, then add say VPN to the equation..
If you start looking at this scenario, then most traffic is going to be encrypted between all servers and all clients thus increasingly important that hardware support is enabled for crypto. And this is why crypto accelerators are pretty standard. It speeds up the client systems.
My question is, is everyone using the same crypto hardware accelerator or are they all different (which would be my assumption.) on the ARM Platform.
This whole scenario is a similar to the FPU, SIMD/MIMD, GPU Processing, Crypto issues.. I -wish- there was an easy moduler way to say, okay we have this function, lets use it.
The whole /dev/crypto if it is the right way to go.. reminds me of the /dev/rnd or /dev/urnd argument from way back when we used prng.
To me this means you need /dev/crypto (or something similar) for -all- devices regardless of hardware support or not, and all implementations need to use it. The optimizations only have to be done once, (but can be overridden, as some algorithms are faster then others at specific tasks for the same type of encryption), and the scheduling is taken care of at the kernel level or at a library level but it has a unified interface so NSS, OpenSSL, etc. can all use it efficiently and it needs to be extensible to allow for future expansion.
Is that what we are aiming for?
omalleys@msu.edu wrote:
Quoting Gordan Bobic gordan@bobich.net:
Peter Robinson wrote:
Interesting question; not sure.
I think it is an important one to answer, and sooner rather than later. This is particularly important to the ARM community since a lot of (most?) ARMs seem to have a crypto co-processor of some description (Freescales, Kirkwood and Tegra definitely seem to have this, I haven't checked others, but since these are the three classes of devices I own, that's 3 out of 3 - I don't think it's luck/coincidence).
Remember this needs to be upstream in Fedora and other projects and then Fedora ARM will get it by default. Work upstream, propose it as a Fedora 16 feature and do the work.
I don't think its particularly important to ARM. It certainly helps on ARM due to low processing but its relevant on all platforms and most platforms have some form of crypto offload built in to the core CPU now days. Hell even the AMD geode processor used in the XO has it.
Don't confuse crypto offload with crypto instructions. The new x86 stuff has crypto instructions that make AES churn faster, but that doesn't leave the CPU free to do other things while this is happening.
My big concern is that the pursued solution will be one size fits x86, as often happens (also why the same packages need ARM patches in consecutive releases).
From what I was reading in the kernel notes last night, the kernel crypto bits are optimized for multiple cores (multithreaded) and SEE3.1 enabled.
Given the kernel support for crypto, shouldn't this be in the kernel also?
- More abstraction (a OpenSSL->NSS shim library), means more bloat,
more context switching and less performance. Is that really the way forward? I mean _really_?
For bulk crypto operations an extra call via a shim probably doesn't matter. For some signature operations it might.
It seems like a clean solution from the point of view of application developers, though.
The other thing that needs to be considered is added complexity and security. I would imagine that since there is an abstraction layer, it introduces additional scope for exploits (buffer overruns, stack smashing, etc.) Is this shim library going to also be FIPS certified? If not then the improved security aspect of NSS vs. OpenSSL comes a lot closer to pure marketing rhetoric (maybe that's where it's at at the moment anyway, I don't claim to be an expert on the subject).
Read the details mentioned on the advantage of NSS over OpenSSL for FIPS certification on the consolidation wiki.
Are you talking about this sentence?
"NSS, in contrast, allows all applications to inherit the NSS FIPS validation status by following some simple rules detailed in the NSS security policy document."
I find it's a bit vague and opaque. I don't see how you could make the certification hereditary.
Think of something like DNSSEC, which hands your machine a certificate, then add something for your actual credentials, like another Cert on a smartcard, kerberos and/or OpenID, then add say VPN to the equation..
If you start looking at this scenario, then most traffic is going to be encrypted between all servers and all clients thus increasingly important that hardware support is enabled for crypto. And this is why crypto accelerators are pretty standard. It speeds up the client systems.
My question is, is everyone using the same crypto hardware accelerator or are they all different (which would be my assumption.) on the ARM Platform.
This whole scenario is a similar to the FPU, SIMD/MIMD, GPU Processing, Crypto issues.. I -wish- there was an easy moduler way to say, okay we have this function, lets use it.
The whole /dev/crypto if it is the right way to go.. reminds me of the /dev/rnd or /dev/urnd argument from way back when we used prng.
To me this means you need /dev/crypto (or something similar) for -all- devices regardless of hardware support or not, and all implementations need to use it. The optimizations only have to be done once, (but can be overridden, as some algorithms are faster then others at specific tasks for the same type of encryption), and the scheduling is taken care of at the kernel level or at a library level but it has a unified interface so NSS, OpenSSL, etc. can all use it efficiently and it needs to be extensible to allow for future expansion.
Is that what we are aiming for?
It's an interesting approach, but I think it would be inefficient on hardware without hardware crypto acceleration. I don't think there is any inherent benefit in having all crypto done in kernelspace. Pushing it through the kernel is only useful if the kernel can hand it off to some underlying hardware and not worry about it until it gets back an interrupt saying the data is crypted. If there's no underlying hardware, context switching to the kernel will just make things slower.
I like the uniformity of the idea (and how thin this would make the userspace crypto libs), but all that skinniness of the userspace layer would mean bloat in the kernel, which is worse. It would also make things non-portable. So, sadly, I think the userspace will have to continue to bring it's own implementation for where hardware isn't there to help.
Unless I'm misunderstanding what you were describing?
Gordan
I think this was being described? http://fedoraproject.org/wiki/Features/CryptographyInKernel
Adam
On Wed, May 25, 2011 at 10:45, Gordan Bobic gordan@bobich.net wrote:
omalleys@msu.edu wrote:
Quoting Gordan Bobic gordan@bobich.net:
Peter Robinson wrote:
Interesting question; not sure.
I think it is an important one to answer, and sooner rather than later. This is particularly important to the ARM community since a lot of (most?) ARMs seem to have a crypto co-processor of some description (Freescales, Kirkwood and Tegra definitely seem to have this, I haven't checked others, but since these are the three classes of devices I own, that's 3 out of 3 - I don't think it's luck/coincidence).
Remember this needs to be upstream in Fedora and other projects and then Fedora ARM will get it by default. Work upstream, propose it as a Fedora 16 feature and do the work.
I don't think its particularly important to ARM. It certainly helps on ARM due to low processing but its relevant on all platforms and most platforms have some form of crypto offload built in to the core CPU now days. Hell even the AMD geode processor used in the XO has it.
Don't confuse crypto offload with crypto instructions. The new x86 stuff has crypto instructions that make AES churn faster, but that doesn't leave the CPU free to do other things while this is happening.
My big concern is that the pursued solution will be one size fits x86, as often happens (also why the same packages need ARM patches in consecutive releases).
From what I was reading in the kernel notes last night, the kernel crypto bits are optimized for multiple cores (multithreaded) and SEE3.1 enabled.
Given the kernel support for crypto, shouldn't this be in the kernel also?
> 2) More abstraction (a OpenSSL->NSS shim library), means more bloat, > more context switching and less performance. Is that really the way > forward? I mean _really_? For bulk crypto operations an extra call via a shim probably doesn't matter. For some signature operations it might.
It seems like a clean solution from the point of view of application developers, though.
The other thing that needs to be considered is added complexity and security. I would imagine that since there is an abstraction layer, it introduces additional scope for exploits (buffer overruns, stack smashing, etc.) Is this shim library going to also be FIPS certified? If not then the improved security aspect of NSS vs. OpenSSL comes a lot closer to pure marketing rhetoric (maybe that's where it's at at the moment anyway, I don't claim to be an expert on the subject).
Read the details mentioned on the advantage of NSS over OpenSSL for FIPS certification on the consolidation wiki.
Are you talking about this sentence?
"NSS, in contrast, allows all applications to inherit the NSS FIPS validation status by following some simple rules detailed in the NSS security policy document."
I find it's a bit vague and opaque. I don't see how you could make the certification hereditary.
Think of something like DNSSEC, which hands your machine a certificate, then add something for your actual credentials, like another Cert on a smartcard, kerberos and/or OpenID, then add say VPN to the equation..
If you start looking at this scenario, then most traffic is going to be encrypted between all servers and all clients thus increasingly important that hardware support is enabled for crypto. And this is why crypto accelerators are pretty standard. It speeds up the client systems.
My question is, is everyone using the same crypto hardware accelerator or are they all different (which would be my assumption.) on the ARM Platform.
This whole scenario is a similar to the FPU, SIMD/MIMD, GPU Processing, Crypto issues.. I -wish- there was an easy moduler way to say, okay we have this function, lets use it.
The whole /dev/crypto if it is the right way to go.. reminds me of the /dev/rnd or /dev/urnd argument from way back when we used prng.
To me this means you need /dev/crypto (or something similar) for -all- devices regardless of hardware support or not, and all implementations need to use it. The optimizations only have to be done once, (but can be overridden, as some algorithms are faster then others at specific tasks for the same type of encryption), and the scheduling is taken care of at the kernel level or at a library level but it has a unified interface so NSS, OpenSSL, etc. can all use it efficiently and it needs to be extensible to allow for future expansion.
Is that what we are aiming for?
It's an interesting approach, but I think it would be inefficient on hardware without hardware crypto acceleration. I don't think there is any inherent benefit in having all crypto done in kernelspace. Pushing it through the kernel is only useful if the kernel can hand it off to some underlying hardware and not worry about it until it gets back an interrupt saying the data is crypted. If there's no underlying hardware, context switching to the kernel will just make things slower.
I like the uniformity of the idea (and how thin this would make the userspace crypto libs), but all that skinniness of the userspace layer would mean bloat in the kernel, which is worse. It would also make things non-portable. So, sadly, I think the userspace will have to continue to bring it's own implementation for where hardware isn't there to help.
Unless I'm misunderstanding what you were describing?
Gordan _______________________________________________ arm mailing list arm@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/arm
Adam Goode wrote:
I think this was being described? http://fedoraproject.org/wiki/Features/CryptographyInKernel
Most interesting. So there really is a plan to move all crypto into the kernel? I can see the benefit from the security point of view, but I expected that putting it all in the kernel may be a step too far in terms of bloat. It should certainly simplify things for Linux-only userspace crypto library implementations.
Is the /dev/crypto device node cryptodev compatible (same node name)?
Gordan
On Wed, May 25, 2011 at 5:31 AM, Gordan Bobic gordan@bobich.net wrote:
It seems to me (and always has, thinking about it) that the argument of BSD vs. GPL vs. LGPL vs. whatever-other-open-source-licence has always been a passtime for people who lack either the ability or the inclination to get on with real productive work. Flame me for that statement all you want, but that's just what it's always looked like to me.
I think you're confusing ideologues who feel obligated to fight their moral crusade anywhere they can, with people who have concerns with real world legal issues.
I don't see that this particular licensing "issue" is stopping OpenSSL shipping with every major distro, with a lot of packages linking against it. Hardlining on this issue seems like it blows things way out of proportion.
Unfortunately, what may very well be a "minor insignificant technicality" can still get your ass sued out of existence in the Real World.
OpenSSL persists because of momentum. The world around us has changed, Open Source WON its battle for legal and practical legitimacy, and what were once handwave-able legal technicalities can no longer be ignored as they once were.
Also, I suspect there's a "hypocrisy" factor at work here. Having a history of picking and choosing which licences you follow to the letter just because you're on the "same side" can be used against you in other more hostile copyright enforcement claims. This is why the FSF has to take a hard line on these things.
(IANAL)
On 05/24/2011 06:32 PM, Stephen John Smoogen wrote:
On Tue, May 24, 2011 at 11:20, Gordan Bobicgordan@bobich.net wrote:
On 05/24/2011 06:11 PM, Andrew Haley wrote:
On 05/23/2011 04:12 PM, Gordan Bobic wrote:
omalleys@msu.edu wrote:
My question, is how hard is this to implement the hardware support non-openssl programs.
Not particularly hard if you're writing your own crypto implementation anyway, but there's a lot to be said for just linking against OpenSSL. It's probably safer to link against the library that has a lot of eyes on it than it is to implement your own.
OpenAFS could use this as it can use a lot of DES encryption, but it uses its own DES implementation. It also happens to be the only one I can think of off the top of my head that uses its own implementation. It would be nice to have.
gpg seems to use its own AES implementation that's slower than SSL's. It would certainly be nice to fix that to use acceleration.
Sounds like it might be a good idea to post a feature request to the upstream bugzilla. Have you checked if there is a build option to make it link against OpenSSL instead of using the bundled crypto stack?
There may be a license incompatibility. OpenSSL has an advertising clause in it I believe which makes it incompatible with various GPL unless an exception is given.
I didn't think that matters on dynamic linking. Does it? And we are specifically talking about dynamic linking to get the features of the system OpenSSL install.
Gordan
On Tue, May 24, 2011 at 6:11 PM, Andrew Haley aph@redhat.com wrote:
On 05/23/2011 04:12 PM, Gordan Bobic wrote:
omalleys@msu.edu wrote:
My question, is how hard is this to implement the hardware support non-openssl programs.
Not particularly hard if you're writing your own crypto implementation anyway, but there's a lot to be said for just linking against OpenSSL. It's probably safer to link against the library that has a lot of eyes on it than it is to implement your own.
OpenAFS could use this as it can use a lot of DES encryption, but it uses its own DES implementation. It also happens to be the only one I can think of off the top of my head that uses its own implementation. It would be nice to have.
gpg seems to use its own AES implementation that's slower than SSL's. It would certainly be nice to fix that to use acceleration.
It would be better to use nss as it has the option of all the various fips certifications which would be useful for gpg.
Alternatively I would think it would be better to use the HW crytpo user interface directly so you get HW acceleration if it avail or fallback if its not. I'd personally prefer not to use openssl for gpg as its not the most secure beast.
Peter
On 05/24/2011 07:05 PM, Peter Robinson wrote:
On Tue, May 24, 2011 at 6:11 PM, Andrew Haleyaph@redhat.com wrote:
On 05/23/2011 04:12 PM, Gordan Bobic wrote:
omalleys@msu.edu wrote:
My question, is how hard is this to implement the hardware support non-openssl programs.
Not particularly hard if you're writing your own crypto implementation anyway, but there's a lot to be said for just linking against OpenSSL. It's probably safer to link against the library that has a lot of eyes on it than it is to implement your own.
OpenAFS could use this as it can use a lot of DES encryption, but it uses its own DES implementation. It also happens to be the only one I can think of off the top of my head that uses its own implementation. It would be nice to have.
gpg seems to use its own AES implementation that's slower than SSL's. It would certainly be nice to fix that to use acceleration.
It would be better to use nss as it has the option of all the various fips certifications which would be useful for gpg.
Just out of interest, what is the "fips" option to configure on OpenSSL for?
Alternatively I would think it would be better to use the HW crytpo user interface directly so you get HW acceleration if it avail or fallback if its not.
Sure, just as OpenSSL does. The point here was that if it can be built to link against OpenSSL, it doesn't have to modify it's bundled crypto implementation for options with all possible crypto engines.
I'd personally prefer not to use openssl for gpg as its not the most secure beast.
The issue here seems to be philosophical. The simple fact is that we trust so much to OpenSSL we might as well save ourselves some memory and effort of reimplementing the wheel and maintaining that reimplemented wheel. Considering we already trust ssh and https in almost all instances to OpenSSL, I think the issue is pretty academic.
One other thing to consider is that the reason OpenSSL gets cryptanalyzed so much is specifically because it is so popular. It also has a lot of eyes on it making sure it is tight and stays that way. IMO, using something else is bordering on security through obscurity - and that shouldn't be encouraged.
Gordan
On Tue, May 24, 2011 at 7:19 PM, Gordan Bobic gordan@bobich.net wrote:
On 05/24/2011 07:05 PM, Peter Robinson wrote:
On Tue, May 24, 2011 at 6:11 PM, Andrew Haleyaph@redhat.com wrote:
On 05/23/2011 04:12 PM, Gordan Bobic wrote:
omalleys@msu.edu wrote:
My question, is how hard is this to implement the hardware support non-openssl programs.
Not particularly hard if you're writing your own crypto implementation anyway, but there's a lot to be said for just linking against OpenSSL. It's probably safer to link against the library that has a lot of eyes on it than it is to implement your own.
OpenAFS could use this as it can use a lot of DES encryption, but it uses its own DES implementation. It also happens to be the only one I can think of off the top of my head that uses its own implementation. It would be nice to have.
gpg seems to use its own AES implementation that's slower than SSL's. It would certainly be nice to fix that to use acceleration.
It would be better to use nss as it has the option of all the various fips certifications which would be useful for gpg.
Just out of interest, what is the "fips" option to configure on OpenSSL for?
Alternatively I would think it would be better to use the HW crytpo user interface directly so you get HW acceleration if it avail or fallback if its not.
Sure, just as OpenSSL does. The point here was that if it can be built to link against OpenSSL, it doesn't have to modify it's bundled crypto implementation for options with all possible crypto engines.
I'd personally prefer not to use openssl for gpg as its not the most secure beast.
The issue here seems to be philosophical. The simple fact is that we trust so much to OpenSSL we might as well save ourselves some memory and effort of reimplementing the wheel and maintaining that reimplemented wheel. Considering we already trust ssh and https in almost all instances to OpenSSL, I think the issue is pretty academic.
One other thing to consider is that the reason OpenSSL gets cryptanalyzed so much is specifically because it is so popular. It also has a lot of eyes on it making sure it is tight and stays that way. IMO, using something else is bordering on security through obscurity - and that shouldn't be encouraged.
NSS gets a lot of review as well as its used in firefox and a lot of other enterprise products (from RH and others). FIPS is one of the certifications and reviews. There's some detail here [1] on the difference between the FIPS differences between NSS and openssl.
Peter
[1] http://fedoraproject.org/wiki/FedoraCryptoConsolidation#FIPS_140
omalleys@msu.edu writes:
Good question. Although I thought dkms support was recently added like F13.
Nah, DKMS support has been in Fedora for a while! It's certainly in F12!
Installing: dkms noarch 2.1.0.1-1.fc12 fedora 95 k
-derek
On Mon, May 23, 2011 at 3:15 PM, Gordan Bobic gordan@bobich.net wrote:
omalleys@msu.edu wrote:
Quoting Gordan Bobic gordan@bobich.net:
On 05/22/2011 09:17 AM, Peter Robinson wrote:
On Sun, May 22, 2011 at 2:11 AM, Gordan Bobicgordan@bobich.net wrote:
Hi,
In case anyone is interested, I got this working on F13. It required building the cryptodev kernel module and rebuilding the standard F13 OpenSSL package with three additional parameters (the cryptodev support is already in the standard OpenSSL package sources, it just isn't enabled in the default build).
More details available here: http://www.altechnative.net/?p=174
Any chance we can have cryptodev enabled in the standard package build? I cannot see any drawbacks to having it available - when cryptodev device isn't there, it will simply fall back to the software implementation. (Note: required cryptodev header file provided by the external kernel driver).
We use upstream Fedora mainline packages. File a bug and once its enabled in Fedora it will come to the ARM platform too.
That just rocks, Thanks!!
Yeah, it's pretty awesome. It makes the Sheevaplug catch up with the Atom that is 466MHz faster and 4x more power-hungry.
What I'm pondering now is something like a dkms package for the cryptodev kernel module, but I seem to remember reading somewhere that dkms is a non-Fedora RHEL thing. What do you guys think would be the best way to approach it, especially since we don't have "standard" kernels at the moment?
From my understanding, these are the options in their assumed preferred order:
1) integration in the upstream kernel - Without this, is it not likely that a kernel in a standard Fedora repository will have cryptodev by default - How likely the inclusion is depends on both upstream for cryptodev and the linux kernel
2) kmod packages that provide the .ko file(s) for a series of kernel releases - obsoleted standard as modules should be included upstream - http://fedoraproject.org/wiki/Obsolete/KernelModules - compatible with a series of built kernels (kABI-compatible) - there is no Fedora ARM kernel package that can be compatible
3) DKMS from http://linux.dell.com/dkms/dkms.html - compiles modules on boot if the module is not available - compiles against the running kernel - several issues, like the need of kernel-headers and a suitable compiler on the system
So, 1) is not an option yet from my understanding. 2) requires a packaged kernel and all users should use that same kernel, which is not the case at the moment either. This leaves 3) as only solution...
HTH, Niels
On Sun, May 22, 2011 at 02:11:16AM +0100, Gordan Bobic wrote:
Hi,
In case anyone is interested, I got this working on F13. It required building the cryptodev kernel module and rebuilding the standard F13 OpenSSL package with three additional parameters (the cryptodev support is already in the standard OpenSSL package sources, it just isn't enabled in the default build).
More details available here: http://www.altechnative.net/?p=174
Do you know how would someone configure all to make openssh take advantage of the cryptodev?
Thank you!
On 05/28/2011 09:52 AM, Lluís Batlle i Rossell wrote:
On Sun, May 22, 2011 at 02:11:16AM +0100, Gordan Bobic wrote:
Hi,
In case anyone is interested, I got this working on F13. It required building the cryptodev kernel module and rebuilding the standard F13 OpenSSL package with three additional parameters (the cryptodev support is already in the standard OpenSSL package sources, it just isn't enabled in the default build).
More details available here: http://www.altechnative.net/?p=174
Do you know how would someone configure all to make openssh take advantage of the cryptodev?
Once OpenSSL uses it, OpenSSH will use it, too, since it is linked against OpenSSL. If you lsmod | grep cryptodev, you will see that for every ssh session you have running, the reference count will go up by one.
Gordan
-
Adam Goode -
Andrew Burgess -
Andrew Haley -
Callum Lerwick -
Derek Atkins -
Emmanuel Seyman -
Gordan Bobic -
Lluís Batlle i Rossell -
Niels de Vos -
omalleys＠msu.edu -
Peter Robinson -
Stephen John Smoogen