Hi,
I'm currently packaging a libraries that are nearly identical, except one version is sse3 optimized, and the other is not. (license also changes to GPL from BSD). The performance difference is significant.
What is the correct way to handle this? They both provide the same libraries and headers. Should I use an ExclusiveArch: x86_64 on the sse3 version, and an ExcludeArch: x86_64 on the unoptimized version? Or perhaps even use a conflicts? (Assuming there are x86_64 processors that don't support SSE3? SSE3 seems to have been introduced with Pentium 4 Prescott)
Thanks in advance for your replies, Matt
Hello, Matt.
On Tuesday, 19 July 2016 at 00:06, Matthew Chan wrote:
I'm currently packaging a libraries that are nearly identical, except one version is sse3 optimized, and the other is not. (license also changes to GPL from BSD). The performance difference is significant.
What is the correct way to handle this? They both provide the same libraries and headers. Should I use an ExclusiveArch: x86_64 on the sse3 version, and an ExcludeArch: x86_64 on the unoptimized version? Or perhaps even use a conflicts? (Assuming there are x86_64 processors that don't support SSE3? SSE3 seems to have been introduced with Pentium 4 Prescott)
In theory, the code shipped in Fedora should work on any i686-class CPU for x86 32bit builds and any x86_64 CPU. So if you want to enable any kind of SIMD unconditionally, you can only rely on SSE2 presence in x86_64 ABI. Anything else should be detected and used at runtime only.
Having said that, it's unlikely that someone is using your particular package on such old hardware. What package are you talking about, specifically?
There was a similar question in the devel list a couple of months back, but with no replies: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/...
Regards, Dominik
Hi Dominik,
I'm still packaging it (no stable copr build yet. I'm using https://copr.fedorainfracloud.org/coprs/talcite/qcint/ and https://copr.fedorainfracloud.org/coprs/talcite/libcint/), but it essentially calculates gaussian integrals for quantum chemistry software. It does something similar to Libint, except it uses a different algorithm. It's a part of a broader push on my part to have PySCF included into the repos, so I can finally get HORTON into there (also quantum chemistry packages).
And you're correct, it's almost inconceivable that someone would calculate on such old hardware. The jobs are rather computationally intensive (bordering on HPC).
Would it be safe just to go with the ExclusiveArch/ExcludeArch route then?
Thanks! Matt
On Tue, 19 Jul 2016 at 00:38 Dominik 'Rathann' Mierzejewski < dominik@greysector.net> wrote:
Hello, Matt.
On Tuesday, 19 July 2016 at 00:06, Matthew Chan wrote:
I'm currently packaging a libraries that are nearly identical, except one version is sse3 optimized, and the other is not. (license also changes to GPL from BSD). The performance difference is significant.
What is the correct way to handle this? They both provide the same
libraries
and headers. Should I use an ExclusiveArch: x86_64 on the sse3 version,
and
an ExcludeArch: x86_64 on the unoptimized version? Or perhaps even use a conflicts? (Assuming there are x86_64 processors that don't support SSE3? SSE3 seems to have been introduced with Pentium 4 Prescott)
In theory, the code shipped in Fedora should work on any i686-class CPU for x86 32bit builds and any x86_64 CPU. So if you want to enable any kind of SIMD unconditionally, you can only rely on SSE2 presence in x86_64 ABI. Anything else should be detected and used at runtime only.
Having said that, it's unlikely that someone is using your particular package on such old hardware. What package are you talking about, specifically?
There was a similar question in the devel list a couple of months back, but with no replies:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/...
Regards, Dominik -- Fedora http://fedoraproject.org/wiki/User:Rathann RPMFusion http://rpmfusion.org "Faith manages." -- Delenn to Lennier in Babylon 5:"Confessions and Lamentations" -- packaging mailing list packaging@lists.fedoraproject.org
https://lists.fedoraproject.org/admin/lists/packaging@lists.fedoraproject.or...
On Tuesday, 19 July 2016 at 01:08, Matthew Chan wrote:
Hi Dominik,
I'm still packaging it (no stable copr build yet. I'm using https://copr.fedorainfracloud.org/coprs/talcite/qcint/ and https://copr.fedorainfracloud.org/coprs/talcite/libcint/), but it essentially calculates gaussian integrals for quantum chemistry software. It does something similar to Libint, except it uses a different algorithm. It's a part of a broader push on my part to have PySCF included into the repos, so I can finally get HORTON into there (also quantum chemistry packages).
Very nice! Thank you for these (upcoming) contributions in advance. Feel free to contact me directly if you have trouble finding a reviewer for your packages. You might also want to join our SciTech SIG (https://fedoraproject.org/wiki/Category:SciTech_SIG) and mark your review requests as blocking FE-SCITECH tracker bug.
And you're correct, it's almost inconceivable that someone would calculate on such old hardware. The jobs are rather computationally intensive (bordering on HPC).
Would it be safe just to go with the ExclusiveArch/ExcludeArch route then?
That's probably a practical option. Personally, I still try to build all my packages for all primary Fedora architectures (and secondaries, if I happen to have some time). This has already helped me expose some bugs in the code, for example in GROMACS and those bugs are fixed already by upstream. I would encourage you to keep building at least the i686 and the armv7hl versions as well. You would probably be the only user/tester of these non-x86_64 builds.
Regards, Dominik
On 07/18/2016 04:33 PM, Dominik 'Rathann' Mierzejewski wrote:
On Tuesday, 19 July 2016 at 01:08, Matthew Chan wrote:
Hi Dominik,
I'm still packaging it (no stable copr build yet. I'm using https://copr.fedorainfracloud.org/coprs/talcite/qcint/ and https://copr.fedorainfracloud.org/coprs/talcite/libcint/), but it essentially calculates gaussian integrals for quantum chemistry software. It does something similar to Libint, except it uses a different algorithm. It's a part of a broader push on my part to have PySCF included into the repos, so I can finally get HORTON into there (also quantum chemistry packages).
Very nice! Thank you for these (upcoming) contributions in advance. Feel free to contact me directly if you have trouble finding a reviewer for your packages. You might also want to join our SciTech SIG (https://fedoraproject.org/wiki/Category:SciTech_SIG) and mark your review requests as blocking FE-SCITECH tracker bug.
I'd be also willing to review these packages, since these packages are very much in my field.
And you're correct, it's almost inconceivable that someone would calculate on such old hardware. The jobs are rather computationally intensive (bordering on HPC).
Would it be safe just to go with the ExclusiveArch/ExcludeArch route then?
That's probably a practical option. Personally, I still try to build all my packages for all primary Fedora architectures (and secondaries, if I happen to have some time). This has already helped me expose some bugs in the code, for example in GROMACS and those bugs are fixed already by upstream. I would encourage you to keep building at least the i686 and the armv7hl versions as well. You would probably be the only user/tester of these non-x86_64 builds.
I'd rather suggest the Conflicts: route. libcint is the general version that works on any platform, while qcint is a drop-in(?) replacement that only works if SSE3 instructions are available (not all x86_64 machines are supported!). [Naturally, qcint is ExclusiveArch: x86_64.]
A speedup of 5-50% for the integrals isn't always noteworthy, since the integrals may not be dominant for the runtime. Also, many people still have older computers out there, and like to run smaller calculations on their desktop.
Okay, the bugs have been filed. I'm going to work on PySCF now.
libcint: https://bugzilla.redhat.com/show_bug.cgi?id=1357728 qcint: https://bugzilla.redhat.com/show_bug.cgi?id=1357724
I'm happy to have two people offer to review the packages for me! Susi and Dominik, either one of you would be great. I'll let you two decide who wants to do it more?
Thanks! Matt
On Tue, 19 Jul 2016 at 02:30 Susi Lehtola jussilehtola@fedoraproject.org wrote:
On 07/18/2016 04:33 PM, Dominik 'Rathann' Mierzejewski wrote:
On Tuesday, 19 July 2016 at 01:08, Matthew Chan wrote:
Hi Dominik,
I'm still packaging it (no stable copr build yet. I'm using https://copr.fedorainfracloud.org/coprs/talcite/qcint/ and https://copr.fedorainfracloud.org/coprs/talcite/libcint/), but it essentially calculates gaussian integrals for quantum chemistry
software.
It does something similar to Libint, except it uses a different
algorithm. It's
a part of a broader push on my part to have PySCF included into the
repos,
so I can finally get HORTON into there (also quantum chemistry
packages).
Very nice! Thank you for these (upcoming) contributions in advance. Feel free to contact me directly if you have trouble finding a reviewer for your packages. You might also want to join our SciTech SIG (https://fedoraproject.org/wiki/Category:SciTech_SIG) and mark your review requests as blocking FE-SCITECH tracker bug.
I'd be also willing to review these packages, since these packages are very much in my field.
And you're correct, it's almost inconceivable that someone would
calculate
on such old hardware. The jobs are rather computationally intensive (bordering on HPC).
Would it be safe just to go with the ExclusiveArch/ExcludeArch route
then?
That's probably a practical option. Personally, I still try to build all my packages for all primary Fedora architectures (and secondaries, if I happen to have some time). This has already helped me expose some bugs in the code, for example in GROMACS and those bugs are fixed already by upstream. I would encourage you to keep building at least the i686 and
the
armv7hl versions as well. You would probably be the only user/tester of
these
non-x86_64 builds.
I'd rather suggest the Conflicts: route. libcint is the general version that works on any platform, while qcint is a drop-in(?) replacement that only works if SSE3 instructions are available (not all x86_64 machines are supported!). [Naturally, qcint is ExclusiveArch: x86_64.]
A speedup of 5-50% for the integrals isn't always noteworthy, since the integrals may not be dominant for the runtime. Also, many people still have older computers out there, and like to run smaller calculations on their desktop. -- Susi Lehtola Fedora Project Contributor jussilehtola@fedoraproject.org -- packaging mailing list packaging@lists.fedoraproject.org
https://lists.fedoraproject.org/admin/lists/packaging@lists.fedoraproject.or...
packaging@lists.fedoraproject.org
-
Dominik 'Rathann' Mierzejewski -
Matthew Chan -
Susi Lehtola