packaging February 2018

packaging@lists.fedoraproject.org

12 participants
15 discussions

Fixing wrong Bootstrapping part in Guidelines
by Jun Aruga 23 Oct '18

23 Oct '18

Dear Packagers who are using Boostrapping logic for the cyclical dependency Need your help to fix wrong Bootstrapping part in Guidelines. This mail is long. Sorry for that in advance. You may be building the cyclical dependency packages by using a variable such as _with_bootstrap, need_bootstrap, bootstrap, enable_test, and etc.. For example, you may build with below ways for that, if you will use mock command. ``` $ mock -r fedora-rawhide-x86_64 --with=bootstrap *.src.rpm => _with_bootstrap can be used as --with=bootstrap $ mock -r fedora-rawhide-x86_64 --define '_with_bootstrap 1' *.src.rpm $ mock -r fedora-rawhide-x86_64 --define 'need_bootstrap 1' *.src.rpm $ mock -r fedora-rawhide-x86_64 --define 'enable_test 1' *.src.rpm ... ``` Here is a document page to unify the Bootstrap logic. You may know it. https://fedoraproject.org/wiki/Packaging:Guidelines#Bootstrapping However I found that below part in the page is wrong. ``` %{!?_with_bootstrap: %global bootstrap 1} ``` Because .. If _with_bootstrap is not set from outside, bootstrap is 1 => bootstrap is True/Enabled if _with_bootstrap is set as 1 from outside, bootstrap's value is not set. => the value is empty if it is not declared in advance. It's a kind of 0. bootstrap is False/Disabled. This situation is opposite meaning of "_with_bootstrap". Below way not using negative operator `!?` is correct. ``` %{?_with_bootstrap: %global bootstrap 1} ``` The reason why I wrote this here is I found that had already been reported 2 years ago for packaging committee, however it was closed without fixing. https://pagure.io/packaging-committee/issue/509 I am not sure that why it is not admitted. You may feel that it does not matter because you may edit the Bootstrapping logic in the RPM spec file manually. But in my case, I am one of the people who use the Bootsrapping logic actively. There are 89 RPM packages that constitutes Ruby on Rails 5.0. To build Ruby on Rails 5.0 completely from scratch, I have to build the packages total 103 times considering bootstrap. I am trying to build those packages automatically by a tool [1] with a configuration file [2] for Ruby on Rails. It is important to fix it due to that. Fortunately today another guy Vit created new ticket for that. So, if YOU like this proposal, please comment in below page of the ticket or reply here. It is helpful for us to move this huge rock. I really want to fix it. "I like it." comment please. => https://pagure.io/packaging-committee/issue/684 Thank you for your help. [1] https://github.com/sclorg/rpm-list-builder [2] https://github.com/sclorg/rhscl-rebuild-recipes/blob/master/ror.yml Kind regards, Jun Aruga

10 27

Proposed Fedora packaging guideline: More Go packaging
by nicolas.mailhot＠laposte.net 04 Mar '18

04 Mar '18

Hi, I am proposing for inclusion a set of rpm technical files aimed at automating the packaging of forge-hosted projects. - Packaging draft: https://fedoraproject.org/wiki/More_Go_packaging - https://pagure.io/packaging-committee/issue/734 - go-srpm-macros RFE with the technical files: https://bugzilla.redhat.com/show_bug.cgi?id=1526721 This proposal is integrated with and depends on the https://fedoraproject.org/wiki/Forge-hosted_projects_packaging_automation draft It builds on the hard work of the Go SIG and reuses the rpm automation of https://fedoraproject.org/wiki/PackagingDrafts/Go when it exists, and produces compatible packages. What it does: - drastically shorter spec files, up to 90% in some cases, often removing hundreds of lines per spec. - simple, packager-friendly spec syntax - automated package naming derived from the native identifier (import path). No more packages names without any relation with current upstream naming. - working Go autoprovides. No forgotten provides anymore. - working Go autorequires. No forgotten requires anymore. - strict automated directory ownership (used by autorequires and autoprovides). - centralized computation of source URLs (via Forge-hosted projects packaging automation). No more packages lacking guidelines. No more broken guidelines no one notices. - easy switch between commits, tags and releases (via Forge-hosted projects packaging automation). No more packages stuck on commits when upstream starts releasing. - guidelines-compliant automated snapshot naming, including snapshot timestamps (via Forge-hosted projects packaging automation). No more packages stuck in 2014. - guidelines-compliant bootstraping. - systematic use of the Go switches defined by the Go maintainer. Easy to do changes followed by a mass rebuild. - flexibility, do the right thing transparently by default, leave room for special cases and overrides. - no bundling (a.k.a. vendoring) due to the pain of packaging one more Go dependency. - centralized Go macros that can be audited and enhanced over time. - aggressive leverage of upstream unit tests to detect quickly broken code. Please consult packaging draft for full information. The proposal has been tested in Rawhide and EL7 over a set of ~ 140 Go packages. This set is a mix of current Fedora packages, bumped to a more recent version, rewrites of Fedora packages, and completely new packages. I hope posting the second part of the automation will answer some questions people had on the https://fedoraproject.org/wiki/Forge-hosted_projects_packaging_automation draft Regards, -- Nicolas Mailhot

8 34

Summary/Minutes from today's FPC Meeting (2018-02-21 18:00 - 18:55 UTC)
by James Antill 21 Feb '18

21 Feb '18

====================== #fedora-meeting-2: fpc ====================== Meeting started by geppetto at 18:00:16 UTC. The full logs are available at https://meetbot.fedoraproject.org/fedora-meeting-2/2018-02-21/fpc.2018- 02-21-18.00.log.html . Meeting summary --------------- * Roll Call (geppetto, 18:00:16) * Schedule (geppetto, 18:10:41) * LINK: https://lists.fedoraproject.org/archives/list/packaging@lists.fedor aproject.org/message/OASC4FXNZQD3GONPZD3GBUZO4WA7SPVY/ (geppetto, 18:10:43) * #741 Clarify the PyPI URL template (geppetto, 18:10:55) * #745 Don't use %{__python}, %{python_sitelib/arch} unredefined (geppetto, 18:34:03) * ACTION: Don't use %{__python}, etc. (+1:4, 0:0, -1:0) (geppetto, 18:37:59) * #751 Packaging:Guidelines#Noarch_with_Unported... is outdated (geppetto, 18:38:20) * Open Floor (geppetto, 18:48:11) * ACTION: Don't use %{__python}, etc. (+1:5, 0:0, -1:0) (geppetto, 18:48:32) * ACTION: Clarify the PyPI URL template (+1:5, 0:0, -1:0) (geppetto, 18:52:44) * ACTION: Rathann Going to investigate koji noarch/exclusive arch bugfix is actually fixed. (geppetto, 18:53:40) Meeting ended at 18:54:06 UTC. Action Items ------------ * Don't use %{__python}, etc. (+1:4, 0:0, -1:0) * Don't use %{__python}, etc. (+1:5, 0:0, -1:0) * Clarify the PyPI URL template (+1:5, 0:0, -1:0) * Rathann Going to investigate koji noarch/exclusive arch bugfix is actually fixed. Action Items, by person ----------------------- * Rathann * Rathann Going to investigate koji noarch/exclusive arch bugfix is actually fixed. * **UNASSIGNED** * Don't use %{__python}, etc. (+1:4, 0:0, -1:0) * Don't use %{__python}, etc. (+1:5, 0:0, -1:0) * Clarify the PyPI URL template (+1:5, 0:0, -1:0) People Present (lines said) --------------------------- * geppetto (47) * Rathann (19) * mbooth (16) * zodbot (12) * limburgher (8) * orionp (3) * lorbus (1) Generated by `MeetBot`_ 0.1.4 .. _`MeetBot`: http://wiki.debian.org/MeetBot

1 0

Schedule for Wednesday's FPC Meeting (2018-02-21 18:00 UTC)
by James Antill 20 Feb '18

20 Feb '18

Following is the list of topics that will be discussed in the FPC meeting Thursday at 2018-02-21 18:00 UTC in #fedora-meeting-2 on irc.freenode.net. Local time information (via. uitime): ================= Day: Wednesday ================= 2018-02-21 10:00 PST US/Pacific 2018-02-21 13:00 EST --> US/Eastern <-- 2018-02-21 18:00 GMT Europe/London 2018-02-21 18:00 UTC UTC 2018-02-21 19:00 CET Europe/Berlin 2018-02-21 19:00 CET Europe/Paris 2018-02-21 23:30 IST Asia/Calcutta --------------- New Day: Thursday ---------------- 2018-02-22 02:00 HKT Asia/Hong_Kong 2018-02-22 02:00 +08 Asia/Singapore 2018-02-22 03:00 JST Asia/Tokyo 2018-02-22 04:00 AEST Australia/Brisbane Links to all tickets below can be found at: https://pagure.io/packaging-committee/issues?status=Open&tags=meeting = Followups = #topic #691 noarch *sub*packages with arch-specific dependencies .fpc 691 https://pagure.io/packaging-committee/issue/691 #topic #693 Wiki:Packaging:RPMMacros .fpc 693 https://pagure.io/packaging-committee/issue/693 #topic #694 Packaging guidelines for application independence .fpc 694 https://pagure.io/packaging-committee/issue/694 #topic #702 C/C++ guidelines should say using clang needs an exception .fpc 702 https://pagure.io/packaging-committee/issue/702 #topic #708 Allocating a static uid and gid for openvswitch .fpc 708 https://pagure.io/packaging-committee/issue/708 #topic #710 Ruby packaging guidelines update .fpc 710 https://pagure.io/packaging-committee/issue/710 #topic #713 Forward-looking conditionals by default .fpc 713 https://pagure.io/packaging-committee/issue/713 #topic #714 let's kill file deps! .fpc 714 https://pagure.io/packaging-committee/issue/714 #topic #715 Separately building package documentation .fpc 715 https://pagure.io/packaging-committee/issue/715 #topic #719 Simplify packaging of forge-hosted projects .fpc 719 https://pagure.io/packaging-committee/issue/719 #topic #723 Guidelines for handling deprecated dependencies during review .fpc 723 https://pagure.io/packaging-committee/issue/723 #topic #726 Review for SELinux Independent Policy packaging Draft .fpc 726 https://pagure.io/packaging-committee/issue/726 #topic #738 Mangling shebangs .fpc 738 https://pagure.io/packaging-committee/issue/738 #topic #743 Add link to C/C++ build flag docs. in redhat-rpm-config .fpc 743 https://pagure.io/packaging-committee/issue/743 = New business = #topic #741 Clarify the PyPI URL template .fpc 741 https://pagure.io/pack aging-committee/issue/741 #topic #745 Don't use %{__python}, %{python_sitelib/arch} unredefined .fpc 745 https://pagure.io/packaging-committee/issue/745 #topic #751 Packaging:Guidelines#Noarch_with_Unported... is outdated .fpc 751 https://pagure.io/packaging-committee/issue/751 = Open Floor = For more complete details, please visit each individual ticket. The report of the agenda items can be found at: https://pagure.io/packaging-committee/issues?status=Open&tags=meeting If you would like to add something to this agenda, you can reply to this e-mail, file a new ticket at https://pagure.io/packaging-committee , e-mail me directly, or bring it up at the end of the meeting, during the open floor topic. Note that added topics may be deferred until the following meeting.

3 2

Schedule for Thursday's FPC Meeting (2018-02-15 17:00 UTC)
by James Antill 15 Feb '18

15 Feb '18

Following is the list of topics that will be discussed in the FPC meeting Thursday at 2018-02-15 17:00 UTC in #fedora-meeting-1 on irc.freenode.net. Local time information (via. uitime): ================= Day: Thursday ================== 2018-02-15 09:00 PST US/Pacific 2018-02-15 12:00 EST --> US/Eastern <-- 2018-02-15 17:00 GMT Europe/London 2018-02-15 17:00 UTC UTC 2018-02-15 18:00 CET Europe/Berlin 2018-02-15 18:00 CET Europe/Paris 2018-02-15 22:30 IST Asia/Calcutta ---------------- New Day: Friday ----------------- 2018-02-16 01:00 HKT Asia/Hong_Kong 2018-02-16 01:00 +08 Asia/Singapore 2018-02-16 02:00 JST Asia/Tokyo 2018-02-16 03:00 AEST Australia/Brisbane Links to all tickets below can be found at: https://pagure.io/packaging-committee/issues?status=Open&tags=meeting = Followups = #topic #691 noarch *sub*packages with arch-specific dependencies .fpc 691 https://pagure.io/packaging-committee/issue/691 #topic #693 Wiki:Packaging:RPMMacros .fpc 693 https://pagure.io/packaging-committee/issue/693 #topic #694 Packaging guidelines for application independence .fpc 694 https://pagure.io/packaging-committee/issue/694 #topic #702 C/C++ guidelines should say using clang needs an exception .fpc 702 https://pagure.io/packaging-committee/issue/702 #topic #708 Allocating a static uid and gid for openvswitch .fpc 708 https://pagure.io/packaging-committee/issue/708 #topic #710 Ruby packaging guidelines update .fpc 710 https://pagure.io/packaging-committee/issue/710 #topic #713 Forward-looking conditionals by default .fpc 713 https://pagure.io/packaging-committee/issue/713 #topic #714 let's kill file deps! .fpc 714 https://pagure.io/packaging-committee/issue/714 #topic #715 Separately building package documentation .fpc 715 https://pagure.io/packaging-committee/issue/715 #topic #719 Simplify packaging of forge-hosted projects .fpc 719 https://pagure.io/packaging-committee/issue/719 #topic #723 Guidelines for handling deprecated dependencies during review .fpc 723 https://pagure.io/packaging-committee/issue/723 #topic #726 Review for SELinux Independent Policy packaging Draft .fpc 726 https://pagure.io/packaging-committee/issue/726 #topic #743 Add link to C/C++ build flag documentation in redhat-rpm- config .fpc 743 https://pagure.io/packaging-committee/issue/743 = New business = #topic #741 Clarify the PyPI URL template .fpc 741 https://pagure.io/packaging-committee/issue/741 = Open Floor = For more complete details, please visit each individual ticket. The report of the agenda items can be found at: https://pagure.io/packaging-committee/issues?status=Open&tags=meeting If you would like to add something to this agenda, you can reply to this e-mail, file a new ticket at https://pagure.io/packaging-committee , e-mail me directly, or bring it up at the end of the meeting, during the open floor topic. Note that added topics may be deferred until the following meeting.

1 0

Maintaining Package Branches (on COPR)
by Evan Klitzke 12 Feb '18

12 Feb '18

Hi, I have a couple of packages on Fedora COPR, and right now I am just building the same version of the RPM for all the Fedora releases I support. Upstream for one of these projects has released a release candidate, and I would like to package the release candidate for Rawhide only. Is it possible to set this up in an automated way? If so, how? The way I intuitively expect this to work is based on y experience with Debian where I would use different git branches in my package repo for different Debian releases. I didn't see anything like this in the COPR build options, but I'm pretty new to the Fedora packaging ecosystem so this might just be PEBCAK? And how is this typically managed for the "real" Fedora package repos? Thanks, Evan

2 1

Re: Proposed Fedora packaging guideline: More Go packaging
by nicolas.mailhot＠laposte.net 11 Feb '18

11 Feb '18

De: "Jan Chaloupka" Hi Jan Apologies for the delayed answer, I was beating the PR into shape to address review comments. > Let's keep moving forward, > improving the infrastructure for other folks and pushing new ideas and > improvements into practical solutions. Let's move forward, I'm sick on sitting on a private spec stash and I see people trying to package stuff already in my list with the old guidelines. On 12/17/2017 08:11 AM, nicolas.mailhot wrote: >> It builds on the hard work of the Go SIG and reuses the rpm automation ofhttps://fedoraproject.org/wiki/PackagingDrafts/Go when it exists, and produces compatible packages. > Can you describe what you mean by "compatible packages"? That mostly means the resulting packages can use "old-style" packages and "old-style" packages can use "new-style" packages. The effective result is similar enough there is no need for a rupture. Now, as I wrote before "old-style" packages will often be too old or not rigorous enough to be useful to "new-style" packages. > I mentioned a list of things that you did not answer fully. Most important to me: > - your macros do not generate build-time dependencies, which I see as > one of the drawbacks. Generating BuildRequires dynamically needs changes in rpm tooling (mostly mock, I think). So it can not be done today if I'm not wrong. If you have a tool that outputs BuildRequires outside rpm, you can still use it. That's not an absolute showstopper as the go compiler is pretty explicit on what's missing, so BuildRequires are easily added during the first %build or %check run (though that's tedious and annoying). > Do you plan to ignore them? Generating BuildRequires would be quite easy if rpm tooling provided an entry point at the end of %prep. So, that's a mid-term objective and requires sending an RFE mock-side first. Requires generation is automatic and complete, so a Go -devel package will pull in all it needs without manual intervention (except for version constrains, that needs some maturing go dep side first) > - support for more subpackages. You describe how to specify which files > go to which packages > (https://fedoraproject.org/wiki/More_Go_packaging#Separate_code_packages) > but you don't say how list of provided packages and a list of > dependencies are generated for the subpackages. It is fully automated and transparent, rpm does not care why you split your Go code, it will compute provides and requires for every directory installed and owned under %{gopath}/src/%{goipath}, and attribute the result to the (sub)package owning the the directory. So just choosing what to install in each %goinstall invocation is sufficient to compute the corresponding requires/provides and attach them to the correct subpackage. If you don't want your package to pull a particular dep, just don't install the code that calls it. > - reproducible evaluation of macros. Either for running automatic > analysis over spec files or for debugging purposes. I would like the > macros to be built in top of binaries I can run separately. That would be quite difficult to do as most macros depend on rpm variables and other context defined during the build run, and can themselves alter rpm state dynamically via variable setting and other calls. rpm is not a dumb archive format it includes all kinds of package building facilities that the macros take advantage of, it ships with an embedded lua engine, and so on. For "simple" macros that only read a specific variable set, and only perform "simple" actions (or set a specific set of variables) you could certainly rewrite them as separate scripts or binaries (for example the naming code is pretty trivial to rewrite in any language, and takes only one input and one output). But the more extensive the integration with rpm is, the more rpm glue you'd need to interface the result, and in some cases the glue is likely to be a significant part of the existing code. Also, you'd lose quite a lot of flexibility, and complexify your build root requirements. I'm not saying it can not be done, or it is not worthwhile to do, but it's a lot of analysis and rewriting work and I'm not sure it's worth it (I'm certainly not interested to do it myself). Note that autodeps are already effectively separate scripts, you just need to pass them a few arguments and pipe them Go code directory names to see them working. It may feel weird but that's how the autodep rpm API is defined. Other ecosystems packaged as rpms sometimes use separate binaries there, sometimes with light shell gluing. You have my blessing to rewrite them in Go if you want to. >> - automated package naming derived from the native identifier (import path). No more packages names without any relation with current upstream naming. > Can you provide a list of package names that diverge from this convention? For example, all the golang/x/foo stuff, which is a pity as it is used right and left. > For historical reasons there are some packages that does not fall into > the naming schema. Either because a go project got migrated to a > different repo I understand, but the packages should be renamed, and not carry on the previous previous previous upstream name. > or because it make/made sense to combine some projects > together and ship them in a single rpm. Combining does not really work from a lifecycle POW, won't be easy to do with the new automation, or with native Go tooling. You really need to give up on it, sorry, it has never been a good idea for Go or for other languages, all the tooling expects *one* project not many, even when you can somewhat force it it never ends well. >> - working Go autoprovides. No forgotten provides anymore. > Sometimes it make sense to skip some provided package. E.g. some > packages provide Windows specific code Then don't install the windows code and it won't pull corresponding deps. >> - working Go autorequires. No forgotten requires anymore. > Depending on how you generate the list you can non-intentionally require > more than is needed. Which causes installation of unneeded trees of rpms. Again, the solution is to not install problem code, instead of shipping non-working code to others. > E.g. for building purposes there is no need to install dependencies of tests I don't install test code so it does not pollute other projects with its dependencies > So the claim is not completely valid. Yes it is. You're just too used to installing everything without filtering, and trying to workaround the resulting problems. > (Valid for both requires/provides): Sometimes you only need to > install/provide a subset of a Go project. If you only install part of a project, rpm will only generate requires/provides for the partial install. Note, however, than this is not the direction go dep is going towards. A lot of projects are going to need to structure their code better in a go dep world (for exactly the same reasons rpm requires better structuring: tools are binary, there is no convenient gray messy zone like when dealing with humans) > There are not many projects that imports every package another project > provides. > So to save time packaging dependencies of unimported packages (and their > following maintenance), Again, just don't install unneeded code, it won't pull any unneeded dep. Much simpler for everyone involved. >> - strict automated directory ownership (used by autorequires and autoprovides). >> - centralized computation of source URLs (via Forge-hosted projects packaging automation). No more packages lacking guidelines. No more broken guidelines no one notices. > In other words to use %gourl macro? Actually, you need %gosource more. %gourl is a pretty side effect. >> - easy switch between commits, tags and releases (via Forge-hosted projects packaging automation). No more packages stuck on commits when upstream starts releasing. > The issue is not about upstream releasing, it's about projects that > actually import snapshots of other projects instead of particular > releases. Given by the nature of the Go land Try to convert one old-style Go spec from commit to release, or the reverse, you'll see it's not quite so simple there is a lot of tricky boilerplate to change. I've seen quite a few packages, that used commits that corresponded to releases, because redoing the spec as a release was too much hassle. >> - guidelines-compliant automated snapshot naming, including snapshot timestamps (via Forge-hosted projects packaging automation). No more packages stuck in 2014. > The issue is not that a package is stuck and not updated for more than 6 > months. The issue is API backwards incompatibility that does not allow > to easily update to a newer version. And a lack of man power to perform > an update. You can hope for more manpower or you can make changes less time-consuming. Both result in more work done, one is less realistic than the other. >> - no bundling (a.k.a. vendoring) due to the pain of packaging one more Go dependency. > Can you more elaborate on that? Cause this is unavoidable no matter how > great any Go guidelines get. > Choice of bundeling is not about how many more Go dependencies you need > to package. > It's due to API backwards incompatibility reasons. You can address API incompatibilities with compat packages, instead of hiding them in bundled libs >> - aggressive leverage of upstream unit tests to detect quickly broken code. > Will not work in general due to API backwards incompatibility issues. I have some doubt that code that fails unit tests is actually something that can be used (and if it can, not be used why are you shipping it). Sure, the unit test may be wrong, but that's not the usual case. Better to detect, check if another package does not need updating, and report upstream otherwise. We had quite a lot of complex builds that were failing, and got unstuck when someone tried to run unit tests, and updated the components they were pointing to. >> no reliance on external utilities to compute code requirements. No >> more dependencies that do not match the shipped Go code. > Please, rephrase or remove the point as for the CGO it is useful to have > external utility that can list all imported C header files and try to > find a set of packages that provide the header files (and corresponding > libraries). I'm not saying other things are not useful, but I *have* seen quite a few Fedora packages that were declaring garbage deps, because the external tool was run on another code state and the requirements were never corrected. > I am surprised you are able to do that without any API issues. Have you > tried to de-bundle and build Kubernetes, Docker, Etcd, Prometheus and > other big projects from the same set of Go packages? At some point you > have/had to switch to compat packages or bundling. Given the state of upstream projects debundling absolutely requires some compat packages. Which is why I documented how to create them easily, even if Jakub disapproves. I'll take compat packages over stealth bundling any day. >> gives up on many conventions of current Fedora Go packaging, as they >> were an obstacle to the target packaging efficiency. > (Still part of the Limitations): Can you make a list of conventions you > consider as obstacles? I would like to comment on each of them. All the very complex layers upon layers of variables, ifdefing and so on that will completely drain your energy before doing any serious change and often differ in subtle ways from the conventions of the rest of the distro. I tried to keep the amount of boilerplate, variables and conditionals minimal, and to apply official Fedora patterns. >> https://fedoraproject.org/wiki/More_Go_packaging#Excluding_testing_in_speci… >> %gochecks . transport/http transport/grpc transport option internal >> integration-tests/storage examples > The excluding mechanism is not intuitive, opt-out is not intuitive but if you don't do opt-out people will select a few directories to test, give up as soon as the list is too long, and certainly never refresh on update opt-out is more robust in presence of humain limitations. And the default should be that everything is tested. As soon as you integrate the opt-out nature of the macro, it's really easy to understand and use. And you certainly do not need to inflict yourself and others one or more exclusion switches that will need to be used every time. >> https://fedoraproject.org/wiki/More_Go_packaging#Handling_API_changes >> API changes may require the creation of Go code compatibility packages >I would not recommended doing that unless you have an army of packagers. As you wrote yourself, it's that or bundling. And bundling is the same misery without the help of rpm tooling and without putting the result in a package that can be reused in other projects that encountered the same API break. >> a packager, that identifies an API change in his Go package, MUST >> notify golang(a)fedoraproject.org at least a week before pushing his >> changes to any release (and those releases SHOULD include >> fedora-devel). This grace period can be waived by FPC for security >> reasons. > From the point of view of updating popular projects like Etcd, > Kubernetes, Prometheus, etc. this is highly impractical. Usually, there > are more dependencies to update. If I will have to wait one week to > update etcd, than another week to push its update to stable branch, then > I will not really be effective in updating. I'm not wed to this rule but removing it means instant breakage for all the stuff that relied on the old version. Which is not nice in a community project. Now if all Fedora Go packagers can live with it, I won't push it more. >> a packager, that identifies an unannounced API change in another Go >> package, MUST notify golang(a)fedoraproject.org at once. >> those notices SHOULD be copied to the maintainers of packages affected >> by the change, when they are known. > This is going to be a regular thing unless a tooling detecting this > changes is available. Is it better to silently wait for the rush of mass rebuild breakage? Not That I would't appreciate some tooling (automated rebuild of dependant packages + notification of breakage would be nice) >> a packager, that identifies that the code he packages, is broken by >> API changes in another project, SHOULD notify politely the upstream of >> the broken project of the API change, if he can not ascertain it is >> already aware of the change. > Some upstream Go projects are responsive, some are not. A lot of Go > projects are forked, abandoned or inactive. Or the upstream is a single > person that does not have time to deal with distribution specific > guidelines. But I appreciate the effort to spread the good practices. > However, sometimes the API change is intentional cause there is no other > way. Note that the guideline write to notify the project that was broken by the change, not the reverse. For non-dead projects they'll have to deal with the breakage sooner or later, better notify them as soon as possible so they can plan how to manage the break. > Usual workflow of updating Go packages: Pretty much what I understood > ## What we need > - automatic updates of Go packages (each time there is a new release or > on demand to a specific commit) That should be quite easy to do with the new style of packaging as long as there is no huge project restructuring. The hard part is compat packages. BuildRequires could also be annoying as long as they are not fully automated (but, it is quite easy to scrap the build log for 'could not find foo package' messages and add the corresponding BR if already packaged) > - no build-time dependencies (we can not discover missing dependencies > before installation) You can not discover deps before the package is fully built but once built it does declare all the stuff it needs and it is possible to check that against repo metadata. > - always generating all provided packages that are available (I don't > want to provide all packages always, e.g. need just a subset to build my > project, the remaining provided packages need new Go projects which adds > unnecessary complexity) More accurately, you can not package Go code, and publish it to others, without the dep it needs. You may choose to package more deps, or to publish less code. You can rely on the code other packaged having the required dependencies (at least at the import level, version constrain level need progress upstream on the go dep front) > - as jcajka pointed out, it is not easy to debug the lua based scripts Yes debugging lua is not the most pleasant thing, on the other hand the language is so limited, once code is written and worked once it is very unlikely to break in the future due to unrelated updates. It is much more robust towards changes than shell or other macro code. > - combination of bundled and un-bundled dependencies (sometimes I just > need to use one or two bundled dependencies because without generating a > tune of compat packages) That certainly works today (for whatever definition of "works" that encompasses bundling in general) > In all cases, the new approach is strongly focused on specfile > simplification. That was its primary objective. > Even if we create a lot of compat packages we still need a mechanism > that can tell us when a given compat package is no longer needed. Sure, however it is trivial to switch from compat to tip in buildrequires and try the result. > I am working on a tooling (in my free time) that will provide that > information (https://github.com/gofed/symbols-extractor, feel free to > contribute to the project with ideas who we need to detect or extract > from the code to make more spec file pieces generic and macro-replaceable). Thank you, added to the long list of things I need to look at. > At the end, even if we do "create as many compat Go packages as needed" > or "let's use vendored packages because we can", > we will still need to maintain entire ecosystem in some reasonable way. They are two faces of the same problem as long as upostreams are not careful API-wise and we lack the manpower to fix every Go project every time its deps change. > From my point of view, the new Go packaging guidelines make it easier > for fresh Go packager to get familiar with Go packaging practices and to > quickly package his/her Go project. I hope that will help new people to join, there are two few Fedora Go packagers for the amount of code that would be interesting to package and maintain. > However, from the point of view of entire Go ecosystem (or any micro > ecosystem = I have more projects building from the same dependencies), > it is harder to tune individual packages for the mutual benefit. I'm not sure why you feel that way, the new guidelines make dependency breakage evident, they do not create it. > So please, pay at least > some respect to all who spent their time and dreamless nights coming > with quick solutions. I don't think I've ever criticized anyone? I do criticize the current status but that has nothing to do with the people involved, given the amount of needed work and the ridiculously small Fedora Go packager pool. I can't justify all the proposed the changes, or the huge amount of work they required, or the amount of work it will take to convert the existing package pool, by pretending the current status is good. I quite like Fedora but I have better ways to spend my work or free time if there is nothing to fix. > I also suggest to remove every comparison with the Go guidelines draft > and the current state of all Go spec files. Why exactly should FPC spend a lot of gruelling review time on the new guidelines if they improve nothing compared to the current state? Their time is not free either. The pros/cons section is just there to explain FPC why we are sucking their precious time, I don't see the point or use of keeping it post-approval. I must say, I don't feel FPC very willing to look at Go in Fedora, if they're not promised some changes. I understand it is not pleasant to see some of those things written down, but not writing them will not cause them to cease to exist. Getting this stuff implemented, approved, and applied, *will* cause a lot of them to cease to exist. And convincing FPC there are significant gains to be had is a way to get the approval faster, and make all of this a thing of the past. And of course, it's just a step, someone else will find other things to fix, and his solution will be better than this one, because it will build on this one, and that will be perfectly fine, and I won't insist this state can not be improved. > There are other parts of the new guidelines I could contradict. However, > doing that we can end up in endless discussion, annoying each other and > burning time that we can use to do important things instead. Improving documentation is useful, but I'm quite sick of it right now, let's move to other things :) > But please, keep in mind, you are not the only one involved in this effort. I certainly hope so, there would be little point spending all this time answering questions and writing wiki pages otherwise! > We only do what we can in the time given :). That is so true… I don't think I can continue stretching it very long either. > Let's combine the results, our experience. Let's open a google document > (or different collaboration solution if you have one in mind) where we > collect all our requirements. > > Let's ask other Go (or wannabe) packagers > what are their requirement, what they see as drawbacks of the current > and new guidelines, who is willing to help with the packaging and > maintenance, let's ask more questions. Well you won't entice me to write even more things near term, and given how few people replied to this thread, even though it was copied everywhere, I'm not sure a lot of people would be interested either. I'd say they probably need to see some visible advancement to commit their efforts there. I certainly need some visible advancement to continue. I'd be delighted to be wrong of course. > I am happy to take the tooling side. More tooling is always good! However there will be a need for a general package baseline refresh or it will keep not interesting many people. > As I mentioned in the ticket > (https://pagure.io/packaging-committee/issue/382), I shifted my focus on > building tooling that do a lot of analysis and detection which are out > of scope of any guidelines. Even if we agree on the final form of the > guidelines, we still need tooling that will help us with the packaging. > E.g. detection of API backwards incompatibilities, spec file generators > (enhancing and improving over time), detection of go packages that are > no longer needed, running CI/CD, etc. The nice thing about fixing one problem, is that it makes fixing other problems possible. Best regards -- Nicolas

1 1

Summary/Minutes from today's FPC Meeting (2018-02-07 18:00 - 19:00 UTC)
by James Antill 07 Feb '18

07 Feb '18

====================== #fedora-meeting-2: fpc ====================== Meeting started by geppetto at 18:01:32 UTC. The full logs are available at https://meetbot.fedoraproject.org/fedora-meeting-2/2018-02-07/fpc.2018- 02-07-18.01.log.html . Meeting summary --------------- * Roll Call (geppetto, 18:01:32) * Schedule (geppetto, 18:05:37) * LINK: https://lists.fedoraproject.org/archives/list/packaging@lists.fedor aproject.org/message/NJL2IBBF6HN622BZ5AVI4E5TDZC7SZ5A/ (geppetto, 18:05:38) * #743 Add link to C/C++ build flag docs. in redhat-rpm-config (geppetto, 18:05:52) * ACTION: mboddu Will help florian get a real draft we can vote on, probably merging that docs. into the C/C++ page. (geppetto, 18:20:49) * #749 Add note about disabling BRP scripts to guidelines (geppetto, 18:21:06) * ACTION: Add note about disabling BRP scripts to guidelines (+1:5, 0:0, -1:0) (geppetto, 19:00:24) * Open Floor (geppetto, 19:00:44) Meeting ended at 19:02:20 UTC. Action Items ------------ * mboddu Will help florian get a real draft we can vote on, probably merging that docs. into the C/C++ page. * Add note about disabling BRP scripts to guidelines (+1:5, 0:0, -1:0) Action Items, by person ----------------------- * **UNASSIGNED** * mboddu Will help florian get a real draft we can vote on, probably merging that docs. into the C/C++ page. * Add note about disabling BRP scripts to guidelines (+1:5, 0:0, -1:0) People Present (lines said) --------------------------- * geppetto (59) * tibbs (46) * Rathann (22) * mbooth (14) * zodbot (11) * ignatenkobrain (7) * limburgher (4) * orionp (1) Generated by `MeetBot`_ 0.1.4 .. _`MeetBot`: http://wiki.debian.org/MeetBot

1 0

Schedule for Wednesday's FPC Meeting (2018-02-07 18:00 UTC)
by James Antill 06 Feb '18

06 Feb '18

Following is the list of topics that will be discussed in the FPC meeting Thursday at 2018-02-07 18:00 UTC in #fedora-meeting-2 on irc.freenode.net. Local time information (via. uitime): ================= Day: Wednesday ================= 2018-02-07 10:00 PST US/Pacific 2018-02-07 13:00 EST --> US/Eastern <-- 2018-02-07 18:00 GMT Europe/London 2018-02-07 18:00 UTC UTC 2018-02-07 19:00 CET Europe/Berlin 2018-02-07 19:00 CET Europe/Paris 2018-02-07 23:30 IST Asia/Calcutta --------------- New Day: Thursday ---------------- 2018-02-08 02:00 HKT Asia/Hong_Kong 2018-02-08 02:00 +08 Asia/Singapore 2018-02-08 03:00 JST Asia/Tokyo 2018-02-08 04:00 AEST Australia/Brisbane Links to all tickets below can be found at: https://pagure.io/packaging-committee/issues?status=Open&tags=meeting = Followups = #topic #691 noarch *sub*packages with arch-specific dependencies .fpc 691 https://pagure.io/packaging-committee/issue/691 #topic #693 Wiki:Packaging:RPMMacros .fpc 693 https://pagure.io/packaging-committee/issue/693 #topic #694 Packaging guidelines for application independence .fpc 694 https://pagure.io/packaging-committee/issue/694 #topic #702 C/C++ guidelines should say using clang needs an exception .fpc 702 https://pagure.io/packaging-committee/issue/702 #topic #708 Allocating a static uid and gid for openvswitch .fpc 708 https://pagure.io/packaging-committee/issue/708 #topic #710 Ruby packaging guidelines update .fpc 710 https://pagure.io/packaging-committee/issue/710 #topic #713 Forward-looking conditionals by default .fpc 713 https://pagure.io/packaging-committee/issue/713 #topic #714 let's kill file deps! .fpc 714 https://pagure.io/packaging-committee/issue/714 #topic #715 Separately building package documentation .fpc 715 https://pagure.io/packaging-committee/issue/715 #topic #719 Simplify packaging of forge-hosted projects .fpc 719 https://pagure.io/packaging-committee/issue/719 #topic #723 Guidelines for handling deprecated dependencies during review .fpc 723 https://pagure.io/packaging-committee/issue/723 #topic #726 Review for SELinux Independent Policy packaging Draft .fpc 726 https://pagure.io/packaging-committee/issue/726 = New business = #topic #743 Add link to C/C++ build flag docs. in redhat-rpm-config .fpc 743 https://pagure.io/packaging-committee/issue/743 #topic #749 Add note about disabling BRP scripts to guidelines .fpc 749 https://pagure.io/packaging-committee/issue/749 = Open Floor = For more complete details, please visit each individual ticket. The report of the agenda items can be found at: https://pagure.io/packaging-committee/issues?status=Open&tags=meeting If you would like to add something to this agenda, you can reply to this e-mail, file a new ticket at https://pagure.io/packaging-committee , e-mail me directly, or bring it up at the end of the meeting, during the open floor topic. Note that added topics may be deferred until the following meeting.

1 0

Re: <DKIM> Re: Proposed Fedora packaging guideline: More Go packaging
by nicolas.mailhot＠laposte.net 06 Feb '18

06 Feb '18

----- Mail original ----- De: "Owen Taylor" Hi Owen, > Is there a guide for Fedora packagers about how to handle unbundling for > golang packages? The draft guidelines don't seem to go into any details. I don't think there is, nor that it is necessarily needed. The posted guidelines should be sufficient technically, there are no magic I know of I didn't document, the rest is just a lot of work (but see ↓) > I've looked at packaging a few golang packages unbundled, and have > immediately run into: > > A) lots of unpackaged dependencies > B) dependencies that are packaged in Fedora with different, often much > older versions Yes the state of Go packages in Fedora is pretty sad right now. I wouldn't expect anyone to be able to package anything but the most trivial app in unbundled mode. Too many common parts are missing, when they are not missing they are too old, trying to update an existing Go package is an exercise in frustration (too much package-specific shell code, that is difficult to understand and does not really work with new code versions), and trying to update or add missing parts just reveals more breakage and work to be done. However, accepting to package complex Go apps in bundled mode is how Fedora got to this state in the first place. In plain speak, bundling (vendoring in Go linguo) just does not scale. You need an awful lot of manpower to audit the hundreds of other projects each app bundles, bundling removes all the package tooling that may help you to do so, and the result is not shared with any other package, or with other versions of the same package, so you get zero positive network effects. Worse, big bundled apps that do not actually try to work in unbundled mode, do not actually test the code they export (they are bundling, remember) so the result is toxic to small Go packages that try to work with them. So bundling parts is a direct road to bundling everything, bundling everything is a direct road to bundling everything blindly, bundling everything blindly is error-prone and dangerous (because upstreams are only human and do make lots of mistakes), and pretty much removes any value Fedora can add to end users, to other parts of Fedora that would like to integrate with Go software, or to the upstream projects themselves (no Fedora QA, no stream of Fedora-originated fixes, no Fedora pressure to stabilize parts when upstream is lost in tunnel effect mode and does not realize that it is wasting everyone's time starting with its own). Therefore, trying to get all this it a better state, requires the following steps IMHO: 1. completely refactor our Go packaging style it's less painful to update Go packages and they do not need a packager with deep knowledge of package-specific shell glue. That takes documentation, and factoring out common Go packaging functions in shared rpm code (macros and autodeps) → that's what I posted 2. using the new documentation and tooling to clean up years of Fedora technical debt, and create a new set of up-to-date Go packages that can serve as new baseline → I have hundreds of specs that I'm waiting for step 1 to complete to submit. They won't constitute a full baseline by themselves, they're not all 100% done, it's too much work to do alone but working with others requires the common macros and documentation to be merged and adopted. This step is going to be painful I'm afraid, Fedora dug itself a deep hole, leaving it is going to be hard. 3. hopefully, the result will be streamlined enough it won't be too painful to keep up to date, having an up to date baseline will help attract new Go packagers like you, everyone will benefit and be happy. We had to package some ostree bits for example to create this baseline, and I'm pretty sure ostree people within Fedora would prefer to maintain those themselves, if the rest of the Fedora go universe didn't make it too hard > B) more > intimidating. It seems like to package the new package, I have to get the > maintainer of the library to upgrade the version, The guidelines and automation aim at making upgrading easy, and avoid one package or packager blocking others > and someone needs to > rebuild everything that depends on the rebuilt library and test that the > rebuilt packages work. I hope that normalizing Fedora Go packages means it will be possible to automate QA tests such as "try to rebuild all the packages that depend on golang(foo) every time you see the package providing golang(foo) changing, and report what broke" That would be expensive computing-side but a lot less expensive and long than expecting each Go packager to do it himself with his own means. And that is certainly not overkill, given how lax Go projects are about maintaining API stability. And then in case of breakage, revert or create a compat package. That's why there is a long chapter dedicated to compat package creation in the proposed guidelines. Regards, -- Nicolas Mailhot

2 11

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

packaging February 2018