packaging June 2022

packaging@lists.fedoraproject.org

16 participants
9 discussions

RPM-level auto release and changelog bumping - Fedora 33 System-Wide Change proposal
by Ben Cotton 13 Aug '22

13 Aug '22

https://fedoraproject.org/wiki/Changes/rpm_level_auto_release_and_changelog… == Summary == redhat-rpm-config will be updated so users of the auto framework get automated release and changelog bumping. == Owner == * Name: [[User:nim| Nicolas Mailhot]] * Email: <nicolas.mailhot at laposte.net> == Detailed Description == This is a system-wide change because all packages build with redhat-rpm-config, but it only concerns packages that opted to use this part of redhat-rpm-config (auto framework). The change will make those packages auto-bump and auto-changelog at the rpm level, in an infrastructure-independent way. == Benefit to Fedora == Autobumping removes a huge packager shore and makes timestamping in changelogs more reliable. == Scope == * Proposal owners: The feature is coded and works at the rpm level. Unfortunately, mock filters away the srpms containing the bump state, so it does not work in upper layers. * Other developers: The feature requires buy-in by mock developers (and probably koji developers) to lift the restrictions that block it above the rpm level. Also, it requires a mechanism to pass the user name and email that will be used in bumped changelogs (defining two variables in ~/.rpmmacros is sufficient at rpm level) * Mock issue: https://github.com/rpm-software-management/mock/issues/599 * Release engineering: https://pagure.io/releng/issue/9567 * Policies and guidelines: maybe eventually if things work out on the technical level * FPC issue: https://pagure.io/packaging-committee/issue/998 * Trademark approval: N/A (not needed for this Change) == Upgrade/compatibility impact == This is a pure build tooling update, it changes how things are built not what is built. == How To Test == A redhat-rpm-config packages with the changes and some example packages are available in https://copr.fedorainfracloud.org/coprs/nim/refactoring-forge-patches-auto-… Since the mock/copr layer is currently blocking the feature, you need to install the redhat-rpm-config and forge macro packages available in this repo locally. Afterwards you can take any of the example packages in the repo and rebuild them with rpmbuild -ba to your heart content, and see the releases bump and the changelogs being updated accordingly. To get beautiful changelogs, you also need to add <pre> %buildsys_name Your name %buildsys_email Your email </pre> in ~/.rpmmacros == User Experience == N/A Packager experience change only == Dependencies == The change is a spin-off of https://fedoraproject.org/wiki/Changes/Patches_in_Forge_macros_-_Auto_macro… Therefore, it depends on the success of that other change and will probably need rebasing if the code in this other change evolves during the redhat-rpm-config merge. It also depends on mock / copr/ koji buy-in and changes, that may add their own requirements. == Contingency Plan == There is no contingency plan because the change will happen or not at all. == Documentation == There is as much documentation as the average redhat-rpm-config change (ie comments in the macro files themselves) == Release Notes == N/A Packager productivity change only -- Ben Cotton He / Him / His Senior Program Manager, Fedora & CentOS Stream Red Hat TZ=America/Indiana/Indianapolis

12 23

What way of opting out form a particular Python shebang flag do you prefer?
by Miro Hrončok 19 Jul '22

19 Jul '22

Hello Pythonistas, packagers. In the context of this change: https://fedoraproject.org/wiki/Changes/PythonSafePath Python shebangs will have be: #! /usr/bin/python3 -sP In order to remove certain flags, packagers have the following tool: # Unset -s on python shebang - ensure that extensions installed with pip # to user locations are seen and properly loaded %global py3_shebang_flags %(echo %py3_shebang_flags | sed s/s//) Or: # Don't add -P to Python shebang # This package only works when /usr/bin is in sys.path %global py3_shebang_flags %(echo %py3_shebang_flags | sed s/P//) In the implementation PR, Maxwell suggested a different approach: https://src.fedoraproject.org/rpms/python-rpm-macros/pull-request/141#comme… Basically, packagers would do something like this: # Unset -s on python shebang - ensure that extensions installed with pip # to user locations are seen and properly loaded %global _python3_shebang_nousersite %{nil} Or: # Don't add -P to Python shebang # This package only works when /usr/bin is in sys.path %global _python3_shebang_safepath %{nil} The macro names are not set in stone, it could even be %_python3_shebang_s and %_python3_shebang_P. The previous sed-based way would still work and packages that already use it would not need to change immediately. Do you consider the macro based approach better (worth it)? And if so, do you prefer actual flag letters in the macro names, or the verbose names? Thanks for your input. -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok

4 4

license of the binary policy
by Jilayne Lovejoy 13 Jul '22

13 Jul '22

Hi Fedora legal and packaging, I'm cross-posting this, as I think it's relevant to both groups. The current policy for filling out the license field of the spec file (as described at https://docs.fedoraproject.org/en-US/packaging-guidelines/LicensingGuidelin… ) states, "The License: field refers to the licenses of the contents of the binary rpm. When in doubt, ask." As we consider how to improve documentation related to Fedora licensing, it would be helpful to hear people's thoughts on the following: 1) how do you (package maintainers) interpret this policy in practice? 2) what further information/documentation about this policy would be helpful? 3) should this policy be different, and if so, how? 4) any other related thoughts or observations Thanks! Jilayne

11 35

Shell Completions
by Maxwell G 29 Jun '22

29 Jun '22

Hi everyone, Currently, packaging shell completions is a bit of a hassle. You have to memorize a bunch of long directory names (`%{_datadir}/bash-completion/ completions`, `%{_datadir}/fish/vendor_completions.d`, and `%{_datadir}/zsh/ site-functions`), and repeat them multiple times in the specfile. For some reason, the bash completion directories are owned by filesystem, but zsh and fish's shell completions directories are not. Therefore, every package needs to copy ``` %files [...] %{_datadir}/bash-completion/completions/%{name} %dir %{_datadir}/fish %dir %{_datadir}/fish/vendor_completions.d %{_datadir}/fish/vendor_completions.d/%{name}.fish %dir %{_datadir}/zsh %dir %{_datadir}/zsh/site-functions %{_datadir}/zsh/site-functions/_%{name} ``` or own each shell's respective root directory and not bother with `%dir` and explicit paths. I don't like doing the latter, as I prefer to be explicit and follow the spirit of https://docs.fedoraproject.org/en-US/packaging-guidelines/#_explicit_lists. I was thinking of creating an sh-completions-packaging package (name suggestions welcome) that provides a subpackage containing macros for each of those directories that would be part of the default buildroot and a filesystem subpackage that owns these directories so packages can Require it and not have to copy this long boiler plate. I would appreciate any feedback you have, particularly regarding naming of the packages and macros. For the macros, would `%{_bashcompdir}`, `%{_zshcompdir}`, and `%{_fishcompdir}` be appropriate macro names? I'm a bit unclear about macro namespacing conventions. Even if you don't like this idea, I think the guidelines could at least use some clarification in this area. I have seen or dealt with multiple packages that install shell completions to the wrong directories, don't properly own the directories, or have other problems. -- Thanks, Maxwell G (@gotmax23) Pronouns: He/Him/His

3 10

Re: [RFC] Build tag in RPM: from NVR to NVRB
by Miro Hrončok 24 Jun '22

24 Jun '22

On 23. 06. 22 14:24, Aleksandra Fedorova wrote: >> 3) If every Fedora packager can rebuild anything without a commit, what do we >> do prevent accidental builds? > I think each rebuild should be treated as a new package, thus it would > require a new bodhi update, testing, and signing. Which means it will > be less likely to accidentally ship it. > > But as I mentioned in the post, right now we'd like to propose the > refactoring, and not a change of the development workflow. Thus I > would initially restrict the rebuild possibility to the admin group > which handles mass-rebuilds and other admin tasks. Then I would > gradually open it up case by case, each case through a separate > conversation. > > Saying that, probably the first case, which I would consider is: is > there a problem of an accidental rebuild of a merged code for Fedora > Rawhide? What would be the reasons for us to_not_ allow it? A specific example is this workflow: Imagine 200 packages need to be rebuilt with boost 1.99. 1) I build boost 1.99 in a side tag 2) I commit a bump to 200 packages 3) I submit a side tag build for 200 packages 4) I repeat (3) until it seems futile This solves the dependency order issues quite well. I also don't need to think about that much and scripting it is trivial. If the build previously succeeded, it won't do anything. If it failed, it will try again. I realize this isn't a very clever workflow. In fact, that isn't my exact workflow. Really, in (4) I exclude the packages that already rebuilt successfully. But I don't need to be that careful when checking. Is my list of finished builds slightly out of date? No problem. Are some builds still running and I try to submit them again? No problem. Nothing happens. With the proposed build ID think, every workflow like this would either need to be extremely more robust, or we would end up with 20 useless concurrent libreoffice builds very soon. -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok

3 2

Re: [RFC] Build tag in RPM: from NVR to NVRB
by Miro Hrončok 24 Jun '22

24 Jun '22

On 23. 06. 22 14:24, Aleksandra Fedorova wrote: >> 2) A packager rebuilds packages from Fedora dist-git in a Copr repo, what will >> be the value of the build tag? How will the Copr build sort over the official >> Fedora builds? (This is essentially the same question but the answer might differ.) > Similar as above, the questions would be how does it work now, do we > want a change or do we want to keep the current setup? Examples from before the Python 3.11 side tag was merged. Fedora rawhide: python3-tomli-2.0.1-2.fc37 (built with Python 3.10) Python 3.11 copr: python3-tomli-2.0.1-2.fc37 (built with Python 3.11) The NEVRs are identical. Other builds in the Python 3.11 copr install the 3.11 build. Now when we managed to not pick up a certain bump: Fedora rawhide: python3-tomli-2.0.1-2.fc37 (built with Python 3.10) Python 3.11 copr: python3-tomli-2.0.1-1.fc37 (built with Python 3.11) The rawhide's NEVR is higher. Other builds in the Python 3.11 copr install try to install the 3.10 build and that conflicts with other stuff in the copr and the dependency resolution fails. We certainly don't want to regress in that regard. -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok

2 2

Re: [RFC] Build tag in RPM: from NVR to NVRB
by Miroslav Suchý 23 Jun '22

23 Jun '22

Dne 23. 06. 22 v 14:24 Aleksandra Fedorova napsal(a): >> 1) An user rebuilds a package from Fedora dist-git in local mock, what will be >> the value of the build tag? How will the local build sort over the official >> Fedora builds? > Afaik currently, if you do a local build, you need to bump an NVR to > ensure the upgrade path is set. Then if the base distribution upgrades > or rebuilds the package, you would need to bump your version again if > you like to preserve the upgrade path. > > This approach would work exactly the same way after the change, as > bumping the Release tag in the sources makes build-related versioning > irrelevant. > > We can agree that by default Build tag in mock environment is unset. > Local builds have no build id, and to make clean upgrades bump of > Release tag is required. > > But there are some new possibilities: > > If you want to create a local rebuild without Release bump, you would > be able to pass the build tag value manually and set to whatever you > want. For example you can add 1 to existing build tag. Or you could would be able => most people will not do that. > set it to 0. Or to 99999999. Any final guidelines should recommend the semantics of the number. At least. It can be monotonous sequence (1, 2, 3...) or a date used in SOA in DNS (2022062301). Or anything else. Just document it. > The latter is less likely though. > > If I were to create local builds which are always higher or always > lower than builds from the base system, I would probably play with the > dist-tag, not build id: > > $ rpmdev-vercmp 0:1.2.3-12.fc37.15 0:1.2.3-12.localfc37.1 > 0:1.2.3-12.fc37.15 < 0:1.2.3-12.localfc37.1 > > There is also a possibility to set default local build tag to be the > local timestamp. But I am not sure if it is useful for a local > development > >> 2) A packager rebuilds packages from Fedora dist-git in a Copr repo, what will >> be the value of the build tag? How will the Copr build sort over the official >> Fedora builds? (This is essentially the same question but the answer might differ.) > Similar as above, the questions would be how does it work now, do we > want a change or do we want to keep the current setup? While for local build users may set the tag to some value, Copr should set it. And Copr developers needs to know to what value it will be. I, personally, do not participate in this thread as I am in neutral position. At the end, I would appreciate issue created against Mock and Copr where will be clear guidance what and how it should be set. Miroslav

1 0

Re: [RFC] Build tag in RPM: from NVR to NVRB
by Neal Gompa 23 Jun '22

23 Jun '22

On Thu, Jun 23, 2022 at 8:25 AM Aleksandra Fedorova <alpha(a)bookwar.info> wrote: > > Hi, Miro, > > On Thu, Jun 23, 2022 at 12:15 PM Miro Hrončok <mhroncok(a)redhat.com> wrote: > > > > On 18. 06. 22 13:05, Aleksandra Fedorova wrote: > > > Hi, all, > > > > > > I'd like to discuss how we can add Build tag in the RPM. > > > > > > As one of the key points is to turn it into a common standard for rpm > > > packages across the ecosystem, the conversation is currently opened > > > upstream [1] and in RHEL Engineering. And I'd like to get Fedora > > > community on board. > > > > > > This is not a finalized proposal and I think it is not ready yet to be > > > a Fedora Change. > > > But I want to start a conversation and ask for opinions. There are > > > also some open questions which need help, especially the requirements > > > around build reason. And alternative suggestions are welcome as well. > > > > > > I've posted long version at > > > https://discussion.fedoraproject.org/t/rfc-build-tag-in-rpms/39954 > > > > > > You can comment there (simply login with your FAS id) or here on the > > > mailing list. > > > And I am going to update that post with the new feedback. > > > > > > Shorter version: > > > ================================================ > > > We'd like to introduce Build Number/Tag/Id in the rpm metadata. > > > > > > By this change we would like to: > > > * Provide a possibility to change build environment and rebuild rpm > > > packages without changing their content: neither sources nor spec > > > files. > > > * Set a common standard for the RPM-based ecosystem, which can be used > > > not just within Fedora, but also by Remixes, downstreams, SIGs and > > > other distributions. > > > > > > The most visible impact of the proposal would be the filename change: > > > > > > Current: dnf-4.9.0-12.fc36.noarch.rpm > > > Proposed: dnf-4.9.0-12.fc36.34897715.noarch.rpm > > > > > > It can be implemented in two steps: > > > > > > 1) Now → Compatibility mode > > > > > > * Introduce Build tag in the rpm metadata > > > * Introduce “build reason” to be added to rpm changelog as a top entry > > > * Enable passing Build tag value to the build in Koji. The value of > > > the Build tag will be set from Koji build id. > > > * Introduce macro in Release field of the rpm spec files, which adds > > > Build tag after the usual disttag > > > Release: 12.%{?dist}%{?build:.%{build}} > > > * Introduce option to pass “build reason” to a Koji build via koji cli > > > and fedpkg/centpkg tooling. > > > > > > 2) Compatibility mode → Final > > > > > > * Implement support for the upgrade path on the rpm side in a > > > compatible way. So that NV(R’=R+B) and NVRB are treated the same. > > > * Remove %{build} part from Release tag and use independent Build tag > > > for versioning. > > > ================================================ > > > > > > [1] https://github.com/rpm-software-management/rpm/discussions/2031 > > > > I have couple concerns/questions. > > > > When modularity was introduced, local builds (outside of MSB) were side tracked > > not to be part of the MVP. Implementing this later proved out to be rather > > complicated. So let's focus on the following gotchas from the start here > > instead please: > > I think these are very valid questions. I didn't think about them at > first, but then added at the last moment under the name: > "Cross-distribution upgrade paths" in the doc. But I like how you > mention Copr and local builds as especially important cases of this, > so I will adjust the wording. > > I don't have a ready to go answer, because I am not sure about the > goal. How do we want this to work? > > Here are just some considerations. > > > > > 1) An user rebuilds a package from Fedora dist-git in local mock, what will be > > the value of the build tag? How will the local build sort over the official > > Fedora builds? > > Afaik currently, if you do a local build, you need to bump an NVR to > ensure the upgrade path is set. Then if the base distribution upgrades > or rebuilds the package, you would need to bump your version again if > you like to preserve the upgrade path. > > This approach would work exactly the same way after the change, as > bumping the Release tag in the sources makes build-related versioning > irrelevant. > > We can agree that by default Build tag in mock environment is unset. > Local builds have no build id, and to make clean upgrades bump of > Release tag is required. > > But there are some new possibilities: > > If you want to create a local rebuild without Release bump, you would > be able to pass the build tag value manually and set to whatever you > want. For example you can add 1 to existing build tag. Or you could > set it to 0. Or to 99999999. > > The latter is less likely though. > > If I were to create local builds which are always higher or always > lower than builds from the base system, I would probably play with the > dist-tag, not build id: > > $ rpmdev-vercmp 0:1.2.3-12.fc37.15 0:1.2.3-12.localfc37.1 > 0:1.2.3-12.fc37.15 < 0:1.2.3-12.localfc37.1 > > There is also a possibility to set default local build tag to be the > local timestamp. But I am not sure if it is useful for a local > development. > I think it makes sense to just set it to zero by default. > > 2) A packager rebuilds packages from Fedora dist-git in a Copr repo, what will > > be the value of the build tag? How will the Copr build sort over the official > > Fedora builds? (This is essentially the same question but the answer might differ.) > > Similar as above, the questions would be how does it work now, do we > want a change or do we want to keep the current setup? > > CC @Miroslav Suchý > I think we'll want to introduce sticky vendors to the Fedora ecosystem to deal with this. Otherwise we'll have EVRB games between sources. Users generally want packages coming from a particular source and following that track makes sense. DNF has the capability to do this if you set "allow_vendor_change=False" in dnf.conf. See: https://dnf.readthedocs.io/en/latest/conf_ref.html#allow-vendor-change-label > > 3) If every Fedora packager can rebuild anything without a commit, what do we > > do prevent accidental builds? > > I think each rebuild should be treated as a new package, thus it would > require a new bodhi update, testing, and signing. Which means it will > be less likely to accidentally ship it. > > But as I mentioned in the post, right now we'd like to propose the > refactoring, and not a change of the development workflow. Thus I > would initially restrict the rebuild possibility to the admin group > which handles mass-rebuilds and other admin tasks. Then I would > gradually open it up case by case, each case through a separate > conversation. > > Saying that, probably the first case, which I would consider is: is > there a problem of an accidental rebuild of a merged code for Fedora > Rawhide? What would be the reasons for us to _not_ allow it? > The most compelling reason would be storage. Koji keeps the binaries of every successful build. We need some policy changes to keep this from turning into even more of a problem than it is now. > > > > As for your open questions, I think that build reason is useful as a changelog > > message (and should be exposed to users). > > > > I also agree with others in this thread that exposing the build ID so > > user-visibly isn't great. > > > > I am not convinced about it. Technically we can configure filename to > not show build-id at all. > But then we will have a confusion, that upgrade path is not visible > from a filename. And as it was mentioned elsewhere [1] we already have > Epoch tag which works this way, and users hate it for this exact > reason. > > I think that cleaner tags and filenames is a good goal. > But I think the requirement "filename should contain all information > which is required to calculate the upgrade path between two builds" is > valid and probably should be included in the definition of that > "cleanliness". > > [1] https://github.com/rpm-software-management/rpm/discussions/2031#discussionc… > It's important for a non-obvious reason: it's pretty much the only way to ensure when repodata is made, that multiple builds can live safely in the same repository. -- 真実はいつも一つ！/ Always, there's only one truth!

1 0

Re: [RFC] Build tag in RPM: from NVR to NVRB
by Miro Hrončok 23 Jun '22

23 Jun '22

On 18. 06. 22 13:05, Aleksandra Fedorova wrote: > Hi, all, > > I'd like to discuss how we can add Build tag in the RPM. > > As one of the key points is to turn it into a common standard for rpm > packages across the ecosystem, the conversation is currently opened > upstream [1] and in RHEL Engineering. And I'd like to get Fedora > community on board. > > This is not a finalized proposal and I think it is not ready yet to be > a Fedora Change. > But I want to start a conversation and ask for opinions. There are > also some open questions which need help, especially the requirements > around build reason. And alternative suggestions are welcome as well. > > I've posted long version at > https://discussion.fedoraproject.org/t/rfc-build-tag-in-rpms/39954 > > You can comment there (simply login with your FAS id) or here on the > mailing list. > And I am going to update that post with the new feedback. > > Shorter version: > ================================================ > We'd like to introduce Build Number/Tag/Id in the rpm metadata. > > By this change we would like to: > * Provide a possibility to change build environment and rebuild rpm > packages without changing their content: neither sources nor spec > files. > * Set a common standard for the RPM-based ecosystem, which can be used > not just within Fedora, but also by Remixes, downstreams, SIGs and > other distributions. > > The most visible impact of the proposal would be the filename change: > > Current: dnf-4.9.0-12.fc36.noarch.rpm > Proposed: dnf-4.9.0-12.fc36.34897715.noarch.rpm > > It can be implemented in two steps: > > 1) Now → Compatibility mode > > * Introduce Build tag in the rpm metadata > * Introduce “build reason” to be added to rpm changelog as a top entry > * Enable passing Build tag value to the build in Koji. The value of > the Build tag will be set from Koji build id. > * Introduce macro in Release field of the rpm spec files, which adds > Build tag after the usual disttag > Release: 12.%{?dist}%{?build:.%{build}} > * Introduce option to pass “build reason” to a Koji build via koji cli > and fedpkg/centpkg tooling. > > 2) Compatibility mode → Final > > * Implement support for the upgrade path on the rpm side in a > compatible way. So that NV(R’=R+B) and NVRB are treated the same. > * Remove %{build} part from Release tag and use independent Build tag > for versioning. > ================================================ > > [1] https://github.com/rpm-software-management/rpm/discussions/2031 I have couple concerns/questions. When modularity was introduced, local builds (outside of MSB) were side tracked not to be part of the MVP. Implementing this later proved out to be rather complicated. So let's focus on the following gotchas from the start here instead please: 1) An user rebuilds a package from Fedora dist-git in local mock, what will be the value of the build tag? How will the local build sort over the official Fedora builds? 2) A packager rebuilds packages from Fedora dist-git in a Copr repo, what will be the value of the build tag? How will the Copr build sort over the official Fedora builds? (This is essentially the same question but the answer might differ.) 3) If every Fedora packager can rebuild anything without a commit, what do we do prevent accidental builds? As for your open questions, I think that build reason is useful as a changelog message (and should be exposed to users). I also agree with others in this thread that exposing the build ID so user-visibly isn't great. -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

packaging June 2022