On Mon, Mar 2, 2020 at 8:46 AM clime clime@fedoraproject.org wrote:
On Mon, 2 Mar 2020 at 12:05, Nicolas Mailhot via devel devel@lists.fedoraproject.org wrote:
If you don’t keep things decentralized you’ll be in a word of pain when the scm or buildsys needs to be changed for another implementation (not to mention, that’s not a good way to collaborate with other distros). That will happen eventually. Web apps are not eternal.
Full decentralization likely means that everyone has its own git repo and we can only sync by mailing list. I think src.fp.o is a good point where things can be done and where we can agree on what the packages that Fedora produce are (meaning we need a canonical source of package sources, otherwise it would be more complex to put a distribution together).
This is not true. Pagure accepts PRs from arbitrary Git servers just fine via the remote PR feature, so we do support decentralized workflows without resorting to sending patches via email or Bugzilla.
Ad. Zbyszek:
What about doing <name>-<version>-<dist>.<commits-since-version-bump>? This means that upgrade path not affected by the number of commits or builds in the older release.
I was thinking how to properly implement this into rpkg-util and then finally, it clicked for me.
First, I added the prefix parameter for git_release macro (below git_dir_release is used instead, which is the recommended form). Hence now, one can specify:
Release: {{{ git_dir_release prefix="0%{?dist}" }}}
which would produce release strings like:
0.fc32.1 0.fc32.2 0.fc32.3
for each tag in f32 branch. The leading zero is not used here for anything but without it, we would get NVRs like somepkg-1.0-.fc32.1 - i.e. dash would be followed by immediate dot, which is not actually invalid but it is strange.
Then it came to me that instead of %{dist} we can simply use branch name and hence (finally) drop "c" from .fcXY:
Release: {{{ git_dir_release prefix="$GIT_BRANCH" }}}
("$GIT_BRANCH" is a macro variable that gives you name of the currently checked out branch)
This will not work for cases when people put some special stuff into %{dist} like Nicolas showed: https://src.fedoraproject.org/rpms/fedora-release/blob/master/f/fedora-relea... but it would work for simple cases and fallback would be possible.
Finally, one can just alias `git_dir_release prefix="$GIT_BRANCH"` with `git_dir_release_branched` (https://pagure.io/rpkg-util/blob/master/f/macros/macros.d/git.bash#_464) and hence get the following:
Release: {{{ git_dir_release_branched }}}
which will be bumping release with respect to the latest tag on the current branch as well as the commits stacked on top of that tag (it will be also bumping release for uncommitted work if your working tree is dirty but i don't want to show it here now because NVRs are then quite long).
I've prepared a test project for this new macro: https://pagure.io/hello_rpkg_release. You need the latest "rpkg" and "rpkg-macros" from https://copr.fedorainfracloud.org/coprs/clime/rpkg-util/ for it to work. But I also just dumped the results here: https://pagure.io/hello_rpkg_release/tree/result . You can see what NVRs are generated (by `rpkg nvr` command) at each particular point for the given branch (i.e. hello_rpkg_release-1.14.0-f32.2 for the second tagged commit in f32 branch) -- they are written down after the "-->" arrow. I mention there three NVRs in total:
hello_rpkg_release-1.14.0-master.1 (the first tagged commit master branch hello_rpkg_release-1.14.0-f32.2 (the second tagged commit in the f32 branch) hello_rpkg_release-1.14.0-f31.1.git.1.34684da9 (untagged commit on top of the first tagged commit in the f31 branch)
Any feedback welcome.
On Mon, 2 Mar 2020 at 14:51, Neal Gompa ngompa13@gmail.com wrote:
On Mon, Mar 2, 2020 at 8:46 AM clime clime@fedoraproject.org wrote:
On Mon, 2 Mar 2020 at 12:05, Nicolas Mailhot via devel devel@lists.fedoraproject.org wrote:
If you don’t keep things decentralized you’ll be in a word of pain when the scm or buildsys needs to be changed for another implementation (not to mention, that’s not a good way to collaborate with other distros). That will happen eventually. Web apps are not eternal.
Full decentralization likely means that everyone has its own git repo and we can only sync by mailing list. I think src.fp.o is a good point where things can be done and where we can agree on what the packages that Fedora produce are (meaning we need a canonical source of package sources, otherwise it would be more complex to put a distribution together).
This is not true. Pagure accepts PRs from arbitrary Git servers just fine via the remote PR feature, so we do support decentralized workflows without resorting to sending patches via email or Bugzilla.
-- 真実はいつも一つ!/ Always, there's only one truth! _______________________________________________ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
On Tue, 3 Mar 2020 at 00:28, clime clime@fedoraproject.org wrote:
Ad. Zbyszek:
What about doing <name>-<version>-<dist>.<commits-since-version-bump>? This means that upgrade path not affected by the number of commits or builds in the older release.
I was thinking how to properly implement this into rpkg-util and then finally, it clicked for me.
First, I added the prefix parameter for git_release macro (below git_dir_release is used instead, which is the recommended form). Hence now, one can specify:
Release: {{{ git_dir_release prefix="0%{?dist}" }}}
which would produce release strings like:
0.fc32.1 0.fc32.2 0.fc32.3
Actually, that wouldn't work because prefix needs to be static, not dependent on rpm macros (we would be searching for tags that contain literally '0%{?dist}' in the last release part after the dash when generating the current release based on past tag names). Only the below versions that depend just on Git would work. I was hoping I can get a fallback for cases like Nicolas or the ruby packages have but that doesn't seem to be possible (at least to me at the moment).
for each tag in f32 branch. The leading zero is not used here for anything but without it, we would get NVRs like somepkg-1.0-.fc32.1 - i.e. dash would be followed by immediate dot, which is not actually invalid but it is strange.
Then it came to me that instead of %{dist} we can simply use branch name and hence (finally) drop "c" from .fcXY:
Release: {{{ git_dir_release prefix="$GIT_BRANCH" }}}
("$GIT_BRANCH" is a macro variable that gives you name of the currently checked out branch)
This will not work for cases when people put some special stuff into %{dist} like Nicolas showed: https://src.fedoraproject.org/rpms/fedora-release/blob/master/f/fedora-relea... but it would work for simple cases and fallback would be possible.
See above, regrettably.
Finally, one can just alias `git_dir_release prefix="$GIT_BRANCH"` with `git_dir_release_branched` (https://pagure.io/rpkg-util/blob/master/f/macros/macros.d/git.bash#_464) and hence get the following:
Release: {{{ git_dir_release_branched }}}
which will be bumping release with respect to the latest tag on the current branch as well as the commits stacked on top of that tag (it will be also bumping release for uncommitted work if your working tree is dirty but i don't want to show it here now because NVRs are then quite long).
I've prepared a test project for this new macro: https://pagure.io/hello_rpkg_release. You need the latest "rpkg" and "rpkg-macros" from https://copr.fedorainfracloud.org/coprs/clime/rpkg-util/ for it to work. But I also just dumped the results here: https://pagure.io/hello_rpkg_release/tree/result . You can see what NVRs are generated (by `rpkg nvr` command) at each particular point for the given branch (i.e. hello_rpkg_release-1.14.0-f32.2 for the second tagged commit in f32 branch) -- they are written down after the "-->" arrow. I mention there three NVRs in total:
hello_rpkg_release-1.14.0-master.1 (the first tagged commit master branch hello_rpkg_release-1.14.0-f32.2 (the second tagged commit in the f32 branch) hello_rpkg_release-1.14.0-f31.1.git.1.34684da9 (untagged commit on top of the first tagged commit in the f31 branch)
Any feedback welcome.
On Mon, 2 Mar 2020 at 14:51, Neal Gompa ngompa13@gmail.com wrote:
On Mon, Mar 2, 2020 at 8:46 AM clime clime@fedoraproject.org wrote:
On Mon, 2 Mar 2020 at 12:05, Nicolas Mailhot via devel devel@lists.fedoraproject.org wrote:
If you don’t keep things decentralized you’ll be in a word of pain when the scm or buildsys needs to be changed for another implementation (not to mention, that’s not a good way to collaborate with other distros). That will happen eventually. Web apps are not eternal.
Full decentralization likely means that everyone has its own git repo and we can only sync by mailing list. I think src.fp.o is a good point where things can be done and where we can agree on what the packages that Fedora produce are (meaning we need a canonical source of package sources, otherwise it would be more complex to put a distribution together).
This is not true. Pagure accepts PRs from arbitrary Git servers just fine via the remote PR feature, so we do support decentralized workflows without resorting to sending patches via email or Bugzilla.
-- 真実はいつも一つ!/ Always, there's only one truth! _______________________________________________ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Le 2020-03-03 07:03, clime a écrit :
Actually, that wouldn't work because prefix needs to be static, not dependent on rpm macros
For myself, I would oppose any rpm release process that would not take core rpm mecanisms like macros into account.
Recording builds in changelogs without checking they actually happened is bad engineering.
Simulating rpm behaviour without performing actual spec evaluation in rpm, is also bad engineering. Especially when you know your simulation is horribly simplistic and approximative.
“Who cares if results match most of the time” is terrible workload optimization. You’ll make packagers waste far more time fixing the cases where automation guessed wrong, than you will win when it guesses right. Basic 80/20 rule, the 20% hard cases cost more than the 80% easy cases. Taking care of the 80% easy cases while making the 20% hard cases harder (due to automation mistakes) is not a good deal at all.
Please work on approaches which are reliable by default. Reliability is hard even when you aim for it. When you don’t, it’s not attainable at all.
Reagrds,
On Tue, 3 Mar 2020 at 09:38, Nicolas Mailhot nicolas.mailhot@laposte.net wrote:
Le 2020-03-03 07:03, clime a écrit :
Actually, that wouldn't work because prefix needs to be static, not dependent on rpm macros
For myself, I would oppose any rpm release process that would not take core rpm mecanisms like macros into account.
Being independent of rpm has one huge potential advantage. You can apply the same approach that you apply for rpm spec files to other kinds of spec files. For example the fedora & containers mailing list contains a thread, when person needs to parametrize FROM clause by branch name to correctly reference base image: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/...
The approach I am suggesting would help there and it could help on other places too.
Recording builds in changelogs without checking they actually happened is bad engineering.
It's not about recording builds but instead about recording releases which I see as something that takes place already in Git.
Simulating rpm behaviour without performing actual spec evaluation in rpm, is also bad engineering. Especially when you know your simulation is horribly simplistic and approximative.
I am not trying to simulate rpm behavior. I am just trying to introduce macros that are rendered verbatim into the spec file inside srpm so those srpms can be rebuilt at any time when needed. rpm does not have this feature atm. If we should put it into rpm, it would be already third kind of rpm macro but even if it should go into rpm at some point, it is better to have the approach heavily tested by production use first because here it doesn't really add much complexity. And as I mentioned, independence of rpm means you can use it for non-rpm projects too.
It's also not simplistic: what you see in {{{ }}} tags is interpreted by bash, it can be multi-line and you can hence do anything there you can do with rpm macros. The only problem is, people already have some automation in rpm macros that doesn't require the git context, instead just e.g. the tarball content and this functionality would need to be rewritten by using {{{ }}} if it should play together with the git-based release generation. This cannot be avoided as much as I would like to. If we try to avoid it, we will get a weird hybrid that doesn't work.
It's also not approximative. You can define your own logic for generating release. There should be some pre-made recommended macros but in the end you are not limited by them.
“Who cares if results match most of the time” is terrible workload optimization. You’ll make packagers waste far more time fixing the cases where automation guessed wrong, than you will win when it guesses right. Basic 80/20 rule, the 20% hard cases cost more than the 80% easy cases. Taking care of the 80% easy cases while making the 20% hard cases harder (due to automation mistakes) is not a good deal at all.
Please work on approaches which are reliable by default. Reliability is hard even when you aim for it. When you don’t, it’s not attainable at all.
I think the approach I am suggesting is very reliable.
As much as I enjoy this conversation, I will need to actually try to make something happen. Forgive me if I don't go into long disputes hereafter. If you, however, make a detailed document summarizing the infra changes needed for your approach I will try to comment on the individual aspects of it and also respond with the document with the changes needed for what I am suggesting.
Best regards! clime
Reagrds,
-- Nicolas Mailhot _______________________________________________ packaging mailing list -- packaging@lists.fedoraproject.org To unsubscribe send an email to packaging-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/packaging@lists.fedoraproject....
packaging@lists.fedoraproject.org