Hi,
at the moment golang packages are built only on primary architectures (defined by %go_arches macro) due to golang compiler architecture support. For secondary architectures gcc-go is available. In order to provide an easy way for packagers to package golang projects for secondary architectures as well, go-srpm-macros packages is introduced. It provides basic macros that can be used within spec files. Currently golang package ships /usr/lib/rpm/macros.d/macros.golang file, which defines %gopath and %go_arches macros. As secondary architectures has no golang package these macros are not available. Thus I am proposing to move these macros to go-srpm-macros package and provide additional ones:
# Define arches for PA and SA %golang_arches %{ix86} x86_64 %{arm} %gccgo_arches %{power64} s390x aarch64 %go_arches %{golang_arches} %{gccgo_arches}
# Where to set GOPATH for builds %gopath %{_datadir}/gocode
# Minimal version of gcc providing gcc-go %gccgo_min_vers 5.0.0
# Define commands for building %golang_build go build -compiler gc %gcc_go_build go build -compiler gccgo -gccgoflags "$RPM_OPT_FLAGS"
# Define commands for testing %golang_test go test -compiler gc %gcc_go_test go test -compiler gccgo -gccgoflags "$RPM_OPT_FLAGS"
Meaning of %golang_arches and %gccgo_arches is obvious. %go_arches is extended for secondary architectures. Spec files using %go_arches macros will not get touched by this change as it takes effect only on secondary architectures. %golang_build, resp. %golang_test and %gcc_go_build, resp. %gcc_go_test macros provides an easy way to run go build, resp. go test commands for golang and gcc-go compiler without typing the minimal list of options. The reason to define %golang_build and %gcc_go_build instead of %go_build for both compilers is to cover a case when gcc-go and golang have a different set of options. So packagers are aware of which compiler/command they use for building. The same holds for the 'go test' command.
Recommended use in spec file: 1) To choose the correct compiler: %ifarch %{golang_arches} BuildRequires: golang %else BuildRequires: gcc-go >= %{gccgo_min_vers} %endif
2) To choose the correct command for building and testing: %ifarch %{golang_arches} %{golang_build} -o bin/binary %{import_path}/binary %{golang_test} %{import_path}/binary %else %{gcc_go_build} -o bin/binary %{import_path}/binary %{gcc_go_test} %{import_path}/binary %endif
At the same time packaging guidelines for golang will be extended to provide information about secondary architectures.
What are the steps to accomplish this task? 1) finish review of go-srpm-macros [1] 2) add go-srpm-macros as a run-time dependency of redhat-rpm-config for f21-f23 and el6 3) remove /usr/lib/rpm/macros.d/macros.golang from golang 4) update packaging guidelines
I will fill all necessary bugs/tickets if needed and update packaging guidelines.
Florian, I would like to make go-srpm-macros a run-time dependency of redhat-rpm-config. Thus I believe go-srpm-macros needs an extra caution so it does not break minimal buildroot. Can you check out the macros.go-srpm file in the package [1]? The file define only one line macros, no %if nor %ifarch.
[1] https://bugzilla.redhat.com/show_bug.cgi?id=1241156
Any feedback and questions are welcomed and appreciated.
Thanks to all involved in helping us.
Kind Regards Jan Chaloupka
On 2015-07-09, Jan Chaloupka jchaloup@redhat.com wrote:
# Define arches for PA and SA %golang_arches %{ix86} x86_64 %{arm}
[...]
Recommended use in spec file:
- To choose the correct compiler:
%ifarch %{golang_arches} BuildRequires: golang %else BuildRequires: gcc-go >= %{gccgo_min_vers} %endif
This will not work. A source package is built on random architecture, thus using %ifarch to define BuildRequire will provide random results.
(And maybe while building a source package, the RPM architecture is redefined to `noarch' value.)
-- Petr
On 07/09/2015 12:49 PM, Petr Pisar wrote:
On 2015-07-09, Jan Chaloupka jchaloup@redhat.com wrote:
# Define arches for PA and SA %golang_arches %{ix86} x86_64 %{arm}
[...]
Recommended use in spec file:
- To choose the correct compiler:
%ifarch %{golang_arches} BuildRequires: golang %else BuildRequires: gcc-go >= %{gccgo_min_vers} %endif
This will not work. A source package is built on random architecture, thus using %ifarch to define BuildRequire will provide random results.
Yes. SRPM is generated on random architecture. However spec file in generated SRPM is untouched. Once rpm is being built correct architectures is chosen based on BuildArch/ExclusiveArch tag. The above piece of a code is again reevaluated. But this time %ifarch is tested based on the chosen architecture. Thus the correct compiler is taken.
Even if the rpm is noarch, %ifarch x86_64 still works if it is built on x86_64. %ifarch noarch is false. Or maybe koji works differently?
(And maybe while building a source package, the RPM architecture is redefined to `noarch' value.)
-- Petr
On Thu, 9 Jul 2015 10:49:01 +0000 (UTC) Petr Pisar ppisar@redhat.com wrote:
On 2015-07-09, Jan Chaloupka jchaloup@redhat.com wrote:
# Define arches for PA and SA %golang_arches %{ix86} x86_64 %{arm}
[...]
Recommended use in spec file:
- To choose the correct compiler:
%ifarch %{golang_arches} BuildRequires: golang %else BuildRequires: gcc-go >= %{gccgo_min_vers} %endif
This will not work. A source package is built on random architecture, thus using %ifarch to define BuildRequire will provide random results.
(And maybe while building a source package, the RPM architecture is redefined to `noarch' value.)
I don't have any real analysis at hand, but we use %ifarch-ed BuildRequires regularly and it works as expected. Secondary arches do builds from imported srpms that are created on primary arches. A problem is when Source or Patch are %ifarch-ed and this is forbidden by the Guidelines.
And I think you are right with the "noarch" set as the architecture when srpm is being created in the BuildSRPMFromSCM task.
Dan
On 07/09/2015 03:07 PM, Dan Horák wrote:
On Thu, 9 Jul 2015 10:49:01 +0000 (UTC) Petr Pisar ppisar@redhat.com wrote:
On 2015-07-09, Jan Chaloupka jchaloup@redhat.com wrote:
# Define arches for PA and SA %golang_arches %{ix86} x86_64 %{arm}
[...]
Recommended use in spec file:
- To choose the correct compiler:
%ifarch %{golang_arches} BuildRequires: golang %else BuildRequires: gcc-go >= %{gccgo_min_vers} %endif
This will not work. A source package is built on random architecture, thus using %ifarch to define BuildRequire will provide random results.
(And maybe while building a source package, the RPM architecture is redefined to `noarch' value.)
I don't have any real analysis at hand, but we use %ifarch-ed BuildRequires regularly and it works as expected. Secondary arches do builds from imported srpms that are created on primary arches. A problem is when Source or Patch are %ifarch-ed and this is forbidden by the Guidelines.
Talking to Dan Mach (CC'ing) he confirmed the srpm is rebuilt on a concrete architecture before rpm is built. So the workflow is: 1) build srpm on random architecture 2) once rpm is about to be built on a concrete architecture (noarch or arch-dep), rebuild srpm on the architecture 3) build rpm from the rebuilt srpm.
And I think you are right with the "noarch" set as the architecture when srpm is being created in the BuildSRPMFromSCM task.
Dan
On 2015-07-09, Jan Chaloupka jchaloup@redhat.com wrote:
On 07/09/2015 03:07 PM, Dan Horák wrote:
On Thu, 9 Jul 2015 10:49:01 +0000 (UTC) Petr Pisar ppisar@redhat.com wrote:
On 2015-07-09, Jan Chaloupka jchaloup@redhat.com wrote:
# Define arches for PA and SA %golang_arches %{ix86} x86_64 %{arm}
[...]
Recommended use in spec file:
- To choose the correct compiler:
%ifarch %{golang_arches} BuildRequires: golang %else BuildRequires: gcc-go >= %{gccgo_min_vers} %endif
This will not work. A source package is built on random architecture, thus using %ifarch to define BuildRequire will provide random results.
(And maybe while building a source package, the RPM architecture is redefined to `noarch' value.)
I don't have any real analysis at hand, but we use %ifarch-ed BuildRequires regularly and it works as expected. Secondary arches do builds from imported srpms that are created on primary arches. A problem is when Source or Patch are %ifarch-ed and this is forbidden by the Guidelines.
Talking to Dan Mach (CC'ing) he confirmed the srpm is rebuilt on a concrete architecture before rpm is built. So the workflow is:
- build srpm on random architecture
- once rpm is about to be built on a concrete architecture (noarch or
arch-dep), rebuild srpm on the architecture 3) build rpm from the rebuilt srpm.
That's good to know. But then I don't understand this guide lines rule https://fedoraproject.org/wiki/Packaging:Guidelines?rd=Packaging/Guidelines#BuildRequires_and_.25.7B_isa.7D:
You MUST NOT use arched BuildRequires. The arch ends up in the built SRPM but SRPMs need to be architecture independent. For instance, if you did this:
# Example of what *not* to do BuildRequires: python%{?_isa} >= 2.7
Then the SRPM that is built in Fedora would have one of these Requirements depending on what builder the SRPM was created on:
python(x86-32) >= 2.7 # or python(x86-64) >= 2.7
This would prevent yum-builddep or similar tools that use the SRPM's requirements from operating correctly.
Is the only issue that SRPM repository which is populated from packages built on random architecture (the (1) step)) will get random data?
Then the original question would suffer from the same problem.
Then either we should ban it too or claim that SRPM repository is not reliable and remove the ban from the guide lines.
-- Petr
Dne 9.7.2015 v 11:18 Jan Chaloupka napsal(a):
Recommended use in spec file:
- To choose the correct compiler:
%ifarch %{golang_arches} BuildRequires: golang %else BuildRequires: gcc-go >= %{gccgo_min_vers} %endif
- To choose the correct command for building and testing:
%ifarch %{golang_arches} %{golang_build} -o bin/binary %{import_path}/binary %{golang_test} %{import_path}/binary %else %{gcc_go_build} -o bin/binary %{import_path}/binary %{gcc_go_test} %{import_path}/binary %endif
Why are you not using the %{?golang_arches} and similar (i.e. with the "?" at the beginning). Why the GO compilers does not provide some virtual provide, which ensures that the compiler is available, without even checking for architecture? This would avoid the need of requiring go-srpm-macros by redhat-rpm-config and it could be build require just by Go packages.
This is definitely step backward.
Vít
On 07/24/2015 03:36 PM, Vít Ondruch wrote:
Dne 9.7.2015 v 11:18 Jan Chaloupka napsal(a):
Recommended use in spec file:
- To choose the correct compiler:
%ifarch %{golang_arches} BuildRequires: golang %else BuildRequires: gcc-go >= %{gccgo_min_vers} %endif
- To choose the correct command for building and testing:
%ifarch %{golang_arches} %{golang_build} -o bin/binary %{import_path}/binary %{golang_test} %{import_path}/binary %else %{gcc_go_build} -o bin/binary %{import_path}/binary %{gcc_go_test} %{import_path}/binary %endif
Why are you not using the %{?golang_arches} and similar (i.e. with the "?" at the beginning).
Yes, you are right. 0%{?golang_arches} will do better. This was a heads up. Still needs some polishing.
%ifarch 0%{?gccgo_arches} BuildRequires: gcc-go >= %{gccgo_min_vers} %else BuildRequires: golang %endif
Why the GO compilers does not provide some virtual provide, which ensures that the compiler is available, without even checking for architecture? This would avoid the need of requiring go-srpm-macros by redhat-rpm-config and it could be build require just by Go packages.
It is a question of maintainability. Golang does not support ppc64 at the moment. Supported by gcc-go. In future this can change. Then you would have to update both golang and gcc components. It means report a bug on golang and gcc. For gcc this is not a high priority issue. It would take some time. Plus gcc would have to add ifarch to its spec file as gcc-go and golang has common architectures.
go-srpm-macros is one point of change. For virtual provides you would need at least two. go-srpm-macros is needed anyway otherwise you duplicate all macros which can get out of sync.
This is definitely step backward.
Sometimes you have to go back so you can go forward.
Vít
Dne 26.7.2015 v 14:03 Jan Chaloupka napsal(a):
On 07/24/2015 03:36 PM, Vít Ondruch wrote:
Dne 9.7.2015 v 11:18 Jan Chaloupka napsal(a):
Recommended use in spec file:
- To choose the correct compiler:
%ifarch %{golang_arches} BuildRequires: golang %else BuildRequires: gcc-go >= %{gccgo_min_vers} %endif
- To choose the correct command for building and testing:
%ifarch %{golang_arches} %{golang_build} -o bin/binary %{import_path}/binary %{golang_test} %{import_path}/binary %else %{gcc_go_build} -o bin/binary %{import_path}/binary %{gcc_go_test} %{import_path}/binary %endif
Why are you not using the %{?golang_arches} and similar (i.e. with the "?" at the beginning).
Yes, you are right. 0%{?golang_arches} will do better. This was a heads up. Still needs some polishing.
%ifarch 0%{?gccgo_arches} BuildRequires: gcc-go >= %{gccgo_min_vers} %else BuildRequires: golang %endif
Why the GO compilers does not provide some virtual provide, which ensures that the compiler is available, without even checking for architecture? This would avoid the need of requiring go-srpm-macros by redhat-rpm-config and it could be build require just by Go packages.
It is a question of maintainability. Golang does not support ppc64 at the moment. Supported by gcc-go. In future this can change. Then you would have to update both golang and gcc components. It means report a bug on golang and gcc. For gcc this is not a high priority issue. It would take some time. Plus gcc would have to add ifarch to its spec file as gcc-go and golang has common architectures.
If you need that change in those packages, it is just one time change which will happen only once. If you were able to convince RPM maintainer to introduce this unneeded dependency, I'm pretty sure you can convince golang/gcc-go maintainers to introduce the virtual provide where it belongs.
go-srpm-macros is one point of change. For virtual provides you would need at least two. go-srpm-macros is needed anyway otherwise you duplicate all macros which can get out of sync.
I don't ming go-srpm-macros. That is perfectly fine, if it is BR of package which needs it. But I don't want to have go-srpm-macros installed on my system, because I'm quite sure I'm not going to build any Go package any time soon and hence this is just cruft.
Vít
On 07/27/2015 10:12 AM, Vít Ondruch wrote:
Dne 26.7.2015 v 14:03 Jan Chaloupka napsal(a):
On 07/24/2015 03:36 PM, Vít Ondruch wrote:
Dne 9.7.2015 v 11:18 Jan Chaloupka napsal(a):
Recommended use in spec file:
- To choose the correct compiler:
%ifarch %{golang_arches} BuildRequires: golang %else BuildRequires: gcc-go >= %{gccgo_min_vers} %endif
- To choose the correct command for building and testing:
%ifarch %{golang_arches} %{golang_build} -o bin/binary %{import_path}/binary %{golang_test} %{import_path}/binary %else %{gcc_go_build} -o bin/binary %{import_path}/binary %{gcc_go_test} %{import_path}/binary %endif
Why are you not using the %{?golang_arches} and similar (i.e. with the "?" at the beginning).
Yes, you are right. 0%{?golang_arches} will do better. This was a heads up. Still needs some polishing.
%ifarch 0%{?gccgo_arches} BuildRequires: gcc-go >= %{gccgo_min_vers} %else BuildRequires: golang %endif
Why the GO compilers does not provide some virtual provide, which ensures that the compiler is available, without even checking for architecture? This would avoid the need of requiring go-srpm-macros by redhat-rpm-config and it could be build require just by Go packages.
It is a question of maintainability. Golang does not support ppc64 at the moment. Supported by gcc-go. In future this can change. Then you would have to update both golang and gcc components. It means report a bug on golang and gcc. For gcc this is not a high priority issue. It would take some time. Plus gcc would have to add ifarch to its spec file as gcc-go and golang has common architectures.
If you need that change in those packages, it is just one time change which will happen only once.
And every time golang or gcc-go changes its list of supported architectures.
If you were able to convince RPM maintainer to introduce this unneeded dependency, I'm pretty sure you can convince golang/gcc-go maintainers to introduce the virtual provide where it belongs.
go-srpm-macros is one point of change. For virtual provides you would need at least two. go-srpm-macros is needed anyway otherwise you duplicate all macros which can get out of sync.
I don't ming go-srpm-macros. That is perfectly fine, if it is BR of package which needs it. But I don't want to have go-srpm-macros installed on my system, because I'm quite sure I'm not going to build any Go package any time soon and hence this is just cruft.
You can say the same about perl-srpm-macros, ocaml-srpm-macros or other *-srpm-macros package redhat-rpm-config has as a runtime dependency.
go-srpm-macros provides only macros.go-srpm file which takes about 1.1 KB of your memory.
Vít
Dne 27.7.2015 v 12:00 Jan Chaloupka napsal(a):
On 07/27/2015 10:12 AM, Vít Ondruch wrote:
Dne 26.7.2015 v 14:03 Jan Chaloupka napsal(a):
On 07/24/2015 03:36 PM, Vít Ondruch wrote:
Dne 9.7.2015 v 11:18 Jan Chaloupka napsal(a):
Recommended use in spec file:
- To choose the correct compiler:
%ifarch %{golang_arches} BuildRequires: golang %else BuildRequires: gcc-go >= %{gccgo_min_vers} %endif
- To choose the correct command for building and testing:
%ifarch %{golang_arches} %{golang_build} -o bin/binary %{import_path}/binary %{golang_test} %{import_path}/binary %else %{gcc_go_build} -o bin/binary %{import_path}/binary %{gcc_go_test} %{import_path}/binary %endif
Why are you not using the %{?golang_arches} and similar (i.e. with the "?" at the beginning).
Yes, you are right. 0%{?golang_arches} will do better. This was a heads up. Still needs some polishing.
%ifarch 0%{?gccgo_arches} BuildRequires: gcc-go >= %{gccgo_min_vers} %else BuildRequires: golang %endif
Why the GO compilers does not provide some virtual provide, which ensures that the compiler is available, without even checking for architecture? This would avoid the need of requiring go-srpm-macros by redhat-rpm-config and it could be build require just by Go packages.
It is a question of maintainability. Golang does not support ppc64 at the moment. Supported by gcc-go. In future this can change. Then you would have to update both golang and gcc components. It means report a bug on golang and gcc. For gcc this is not a high priority issue. It would take some time. Plus gcc would have to add ifarch to its spec file as gcc-go and golang has common architectures.
If you need that change in those packages, it is just one time change which will happen only once.
And every time golang or gcc-go changes its list of supported architectures.
And how often this happens?
If you were able to convince RPM maintainer to introduce this unneeded dependency, I'm pretty sure you can convince golang/gcc-go maintainers to introduce the virtual provide where it belongs.
go-srpm-macros is one point of change. For virtual provides you would need at least two. go-srpm-macros is needed anyway otherwise you duplicate all macros which can get out of sync.
I don't ming go-srpm-macros. That is perfectly fine, if it is BR of package which needs it. But I don't want to have go-srpm-macros installed on my system, because I'm quite sure I'm not going to build any Go package any time soon and hence this is just cruft.
You can say the same about perl-srpm-macros, ocaml-srpm-macros or other *-srpm-macros package redhat-rpm-config has as a runtime dependency.
Yes, and I say that about them. You can ask Perl maintainers at minimum. The worst thing is that you already take them as an excuse to not do the right thing :/
go-srpm-macros provides only macros.go-srpm file which takes about 1.1 KB of your memory.
That is just one metrics you deliberately chose, ignoring the others.
Vít
On 2015-07-27, Vít Ondruch vondruch@redhat.com wrote:
Dne 27.7.2015 v 12:00 Jan Chaloupka napsal(a):
You can say the same about perl-srpm-macros, ocaml-srpm-macros or other *-srpm-macros package redhat-rpm-config has as a runtime dependency.
Yes, and I say that about them. You can ask Perl maintainers at minimum. The worst thing is that you already take them as an excuse to not do the right thing :/
Sometimes you need macros in minimal build root to influence resulting source pacakges. Then the macros will be always available for everyone anytime regardless being deliery by redhat-rpm-config or any other package.
My reasoning for having specific macros in specific package instead of in redhat-rpm-config is that it separates responsibility and accountibility. If something goes wrong, locating the real cause by comparing build root listing is easier than diffing redhat-rpm-config files. Also letting tens of people commit into redhat-rpm-config does not sound for me correct and safe.
-- Petr
Dne 27.7.2015 v 13:51 Petr Pisar napsal(a):
On 2015-07-27, Vít Ondruch vondruch@redhat.com wrote:
Dne 27.7.2015 v 12:00 Jan Chaloupka napsal(a):
You can say the same about perl-srpm-macros, ocaml-srpm-macros or other *-srpm-macros package redhat-rpm-config has as a runtime dependency.
Yes, and I say that about them. You can ask Perl maintainers at minimum. The worst thing is that you already take them as an excuse to not do the right thing :/
Sometimes you need macros in minimal build root to influence resulting source pacakges.
My initial proposal removes the need for this macros being available for building SRPM I have not tested it though). I don't dispute the rest you said :)
Vít
Then the macros will be always available for everyone anytime regardless being deliery by redhat-rpm-config or any other package.
My reasoning for having specific macros in specific package instead of in redhat-rpm-config is that it separates responsibility and accountibility. If something goes wrong, locating the real cause by comparing build root listing is easier than diffing redhat-rpm-config files. Also letting tens of people commit into redhat-rpm-config does not sound for me correct and safe.
-- Petr
On Monday, July 27, 2015 10:12:03 AM Vít Ondruch wrote:
Dne 26.7.2015 v 14:03 Jan Chaloupka napsal(a):
On 07/24/2015 03:36 PM, Vít Ondruch wrote:
Dne 9.7.2015 v 11:18 Jan Chaloupka napsal(a):
Recommended use in spec file:
- To choose the correct compiler:
%ifarch %{golang_arches} BuildRequires: golang %else BuildRequires: gcc-go >= %{gccgo_min_vers} %endif
- To choose the correct command for building and testing:
%ifarch %{golang_arches} %{golang_build} -o bin/binary %{import_path}/binary %{golang_test} %{import_path}/binary %else %{gcc_go_build} -o bin/binary %{import_path}/binary %{gcc_go_test} %{import_path}/binary %endif
Why are you not using the %{?golang_arches} and similar (i.e. with the "?" at the beginning).
Yes, you are right. 0%{?golang_arches} will do better. This was a heads up. Still needs some polishing.
%ifarch 0%{?gccgo_arches} BuildRequires: gcc-go >= %{gccgo_min_vers} %else BuildRequires: golang %endif
Why the GO compilers does not provide some virtual provide, which ensures that the compiler is available, without even checking for architecture? This would avoid the need of requiring go-srpm-macros by redhat-rpm-config and it could be build require just by Go packages.
It is a question of maintainability. Golang does not support ppc64 at the moment. Supported by gcc-go. In future this can change. Then you would have to update both golang and gcc components. It means report a bug on golang and gcc. For gcc this is not a high priority issue. It would take some time. Plus gcc would have to add ifarch to its spec file as gcc-go and golang has common architectures.
If you need that change in those packages, it is just one time change which will happen only once. If you were able to convince RPM maintainer to introduce this unneeded dependency, I'm pretty sure you can convince golang/gcc-go maintainers to introduce the virtual provide where it belongs.
go-srpm-macros is one point of change. For virtual provides you would need at least two. go-srpm-macros is needed anyway otherwise you duplicate all macros which can get out of sync.
I don't ming go-srpm-macros. That is perfectly fine, if it is BR of package which needs it. But I don't want to have go-srpm-macros installed on my system, because I'm quite sure I'm not going to build any Go package any time soon and hence this is just cruft.
the very nature of the srpm-macros packages precludes the ability of it being added as a BuildRequire the package defines macros that need to be installed for srpm creation to set macros and define BuildRequires correctly. any other macro that is defined belongs elsewhere in a packag that is added as a BuildRequires.
Dennis
-
Dan Horák -
Dennis Gilmore -
Jan Chaloupka -
Petr Pisar -
Vít Ondruch