On Mon, May 29, 2017 at 12:41:57PM +0200, Miroslav Suchý wrote:

Dne 26.5.2017 v 21:42 Ralph Bean napsal(a):

...

Any feedback before that would be greatly appreciated.

PkgDB handles Koschei and upstream monitoring settings too. How I can do that after the migration?

The Koschei devs have a feature that will let you manage the koschei settings in the koschei web ui.

For upstream monitoring (hotness), we're going to extract the current settings and add them ad a yaml file in dist-git repos for packages that have this turned on. We had issues for this in our backlog but they failed to make it to the timeline in the wiki. I've added them just now.

Does this change somehow affect fedora-packages (aka Moksha) https://apps.fedoraproject.org/packages/ ?

Eh, it does. That's one we missed -- thanks!

https://github.com/fedora-infra/fedora-packages/blob/develop/fedoracommunity...

We'll need to patch that to make requests to the pagure API here too.

So, PkgDB now comes with a big fat warning saying:

"Attention! PkgDB will be replaced during the week of July 10th, 2017. Please read the following for migration instructions: https://fedoraproject.org/wiki/Infrastructure/WhatHappenedToPkgdb"

If I go there I find no "migration instructions" whatsoever, the pararagraph on "How Is PkgDB's Functionality Being Replaced" is ridiculously short and points to a "Pagure over Dist-Git (placeholder link)" which obviously points to nowhere. All these "to be written" in the timeline for June (!) don't make me overly confident in that agenda.

Saying "Do something or else" without telling us "what" nor "what else" is a really good way to put off contributors, but no good way to get everyone in the boat for the move.

Really, I'm all in for weeding out the git branch madness that we have now in dist-git (too many cherry-picks, insane merge directions) and I do hope that entanglich branches from releases will help that purpose.

But please, take a step back and think about how and what you communicate to packagers, that is "infra-users".

Michael

Adam Samalik venit, vidit, dixit 04.06.2017 12:43:

This enables us to have branches that make more sense for individual packages - so we can save work by having just one branch for one version acrsoss releases, or to offer more versions or "streams". A slide [1] from my recent talk demonstrates the possibilities - and also shows why branches are not always just versions. It talks about modules, but it's the same for packages, too.

I like your work, guys! If you want to use any of the graphic from my slides in a documentation or anywhere else, please feel free to do so. I can even tweak it if you like.

[1] https://asamalik.fedorapeople.org/modularity-dorscluc-2017/#/1/2

On Fri, May 26, 2017 at 9:42 PM, Ralph Bean <rbean@redhat.com mailto:rbean@redhat.com> wrote:

Hello,

As part of the Factory 2.0 and Modularity efforts[1], we’ve been
developing a plan to migrate to an “arbitrary” branching model from
our current model of one branch per release (as had been discussed at
Flock and DevConf[2]).

The main motivation behind this is to enable functionality required by
Modularity[3] and to ultimately reduce some package maintenance
burden. For some packages, it makes sense to have only a single branch
that feeds into multiple releases. For other packages, it makes sense
to have multiple branches which correlate with multiple upstream minor
releases. Today, our source branches are tied to the distro release,
via PkgDB.  We want to decouple that and use modules to put it all
back together again.

To make this happen requires significant infrastructure changes.  Our
proposed plan[4] is to decommission PkgDB entirely and to replace it
with a combination of PDC[5] and pagure over dist-git.  (Tangentially,
getting pagure over dist-git to play nicely with PkgDB was a
challenge.  This route gets us to a pull-request interface for spec
files quicker.)

We have brought this Change to FESCo[6][7][8] who expressed general
agreement on the project but also concern that the community may be
caught by off guard by the removal of PkgDB. As part of this change,
we have proposed a timeline[9] that outlines the steps we plan to take
to actually proceed with the migration. Please review that if you have
time and provide feedback. We are most concerned with missing
scripts/tools that may rely on PkgDB’s API. If you can think of any
that we may have overlooked, please let us know and we will add it to
the timeline!

We are meeting again with FESCo next Friday, June 2nd, where a
decision will be made on the Change. Any feedback before that would be
greatly appreciated.

Ralph and Matt,
From the so-called Factory 2.0 team

[1] https://fedoraproject.org/wiki/Infrastructure/Factory2
<https://fedoraproject.org/wiki/Infrastructure/Factory2>
[2] https://youtu.be/5gqccjyjwFk?t=26m27s
<https://youtu.be/5gqccjyjwFk?t=26m27s>
[3] https://docs.pagure.org/modularity/
<https://docs.pagure.org/modularity/>
[4]
https://fedoraproject.org/wiki/Infrastructure/Factory2/Focus/ArbitraryBranching
<https://fedoraproject.org/wiki/Infrastructure/Factory2/Focus/ArbitraryBranching>
[5] https://fedoraproject.org/wiki/Changes/ProductDefinitionCenter
<https://fedoraproject.org/wiki/Changes/ProductDefinitionCenter>
[6] https://fedoraproject.org/wiki/Changes/ArbitraryBranching
<https://fedoraproject.org/wiki/Changes/ArbitraryBranching>
[7]
https://meetbot.fedoraproject.org/teams/fesco/fesco.2017-05-19-16.00.html
<https://meetbot.fedoraproject.org/teams/fesco/fesco.2017-05-19-16.00.html>
[8]
https://meetbot.fedoraproject.org/teams/fesco/fesco.2017-05-26-16.00.html
<https://meetbot.fedoraproject.org/teams/fesco/fesco.2017-05-26-16.00.html>
[9]
https://fedoraproject.org/wiki/Changes/ArbitraryBranching#Timeline
<https://fedoraproject.org/wiki/Changes/ArbitraryBranching#Timeline>

_______________________________________________
devel-announce mailing list --
devel-announce@lists.fedoraproject.org
<mailto:devel-announce@lists.fedoraproject.org>
To unsubscribe send an email to
devel-announce-leave@lists.fedoraproject.org
<mailto:devel-announce-leave@lists.fedoraproject.org>

--

Adam Šamalík

Software Engineer Red Hat

---

devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-leave@lists.fedoraproject.org

On Thu, Jun 08, 2017 at 09:42:45AM -0500, Michael Cronenworth wrote:

This change, which is a pretty radical change, was only brought out in the open last month. It's now being shovelled down our throats after being behind closed doors for who knows how long. This is a dramatic reversal from the previous open, community-driven, changes that Fedora normally sees.

Speaking as someone on both side of doors... this is not something that was developed in secret at all. It's something that was implied by the modularity work — which has been very open — and the change "in the open last month" is all there is to it. There is no puppetmaster behind the curtain.

There are plenty of times when I _do_ need to do work to drag things out to be public, transparent, and community-oriented, but this isn't one of them, and jumping to accusations doesn't help.

On Thu, Jun 08, 2017 at 10:38:11AM -0500, Michael Cronenworth wrote:

Normally I ignore any Modularity discussion. It doesn't interest me, and it doesn't affect any projects I work on. It's my own fault that this change, which does affect me, was not on my radar. I'm not looking to stop the change but only express my concern about the lack of marketing and lack of documentation.

Thanks. Communication is hard. Have you seen the stuff at https://docs.pagure.org/modularity/?

On Thu, Jun 8, 2017 at 7:15 PM, Tom Hughes tom@compton.nu wrote:

On 08/06/17 17:58, Matthew Miller wrote:

...

On Thu, Jun 08, 2017 at 10:38:11AM -0500, Michael Cronenworth wrote:

...

Normally I ignore any Modularity discussion. It doesn't interest me, and it doesn't affect any projects I work on. It's my own fault that this change, which does affect me, was not on my radar. I'm not looking to stop the change but only express my concern about the lack of marketing and lack of documentation.

Thanks. Communication is hard. Have you seen the stuff at https://docs.pagure.org/modularity/?

Speaking for myself I came across that the other day and got as far as clicking through to the documentation page and seeing that just the contents was about four screens long and gave up at that point as it's not something I have much interest in anyway.

Something similar happened in regard to the specific issue here when comment was invited on the arbitrary branching stuff before Fesco discussed it in that I clicked through to the document and found it was a long and detailed list of steps the sysadmins would need to take to roll it out rather than an explanation of what it meant for end users and gave up at that point on trying to understand what it meant beyond moving from pkgdb to pagure over dist-git.

This is actually a good feedback. The landing page should explain the main concepts and the documentation should give more detail... but maybe the landing page is very high-level and the docs too detailed? And there is nothing in between?

I talked to someone about a different way to write stuff - starting with a summary at the top and giving more and more details as you follow reading. But you could theoretically stop reading at almost any point and it should still make sense. Maybe I could give it a shot and rewrite some of the pages - or at least give a higher-level, yet still specific and technical, summary. Hmm.. let me think about it.

Suggestions? Anything I should prioritize?

Tom

-- Tom Hughes (tom@compton.nu) http://compton.nu/

---

devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-leave@lists.fedoraproject.org

On Thu, Jun 08, 2017 at 06:48:27PM +0100, Tom Hughes wrote:

I mean it would probably still be quite daunting for somebody that did want to get into more detail I guess but I think I wound up there following through from some of the other stuff about arbitrary branching and I was mostly just trying to see what changes I (as a packager largely avoiding modularity) might find it impossible to avoid and that was just the point at which I decided I wasn't likely to find anything useful to that goal there and bailed out.

Tom, you work on Node stuff, right? Do you think any of this might be useful for packaging and maintaining that in Fedora?

On Thu, Jun 8, 2017 at 8:13 PM, Tom Hughes tom@compton.nu wrote:

On 08/06/17 18:54, Matthew Miller wrote:

...

On Thu, Jun 08, 2017 at 06:48:27PM +0100, Tom Hughes wrote:

...

I mean it would probably still be quite daunting for somebody that did want to get into more detail I guess but I think I wound up there following through from some of the other stuff about arbitrary branching and I was mostly just trying to see what changes I (as a packager largely avoiding modularity) might find it impossible to avoid and that was just the point at which I decided I wasn't likely to find anything useful to that goal there and bailed out.

Tom, you work on Node stuff, right? Do you think any of this might be useful for packaging and maintaining that in Fedora?

Well from what I understand I don't see how it will help with any of the problems that we have internally with packaging Node stuff.

I mean assuming you're talking about the node interpreter and node libraries (ie modules but I'll avoid that name as it will be confusing here) all being part of a Fedora module then I don't see how it helps at all because the issue with node packaging is the number of dependencies within the node stack and the rate of change of those dependencies.

I can imagine the artificial branching that is coming with modularity could be beneficial for you. It will give you the possibility of having package branches with specific end of life and "support or quality level".

You say that you deal with a large number of dependencies that keep changing. How often these changes happen? What if you could - and this is just my idea - create a package of lower quality in a specific branch (which would take less time), and explicitly say so. You could then use the package just as a dependency of your thing. And when the dependencies change and the package is no longer needed, you can just dump it. That's fine because you said it's just for you.

And I don't want to make it sound like the main selling point of modularity is bad packaging... :-) Another example: You want to add a package to Fedora that you just need as a dependency. Two bad things can happen:

1/ The package is already there, but in a different version. You can't just change it. 2/ You add the package and other people start to use it. That's great until you need to change the version, but can't, because other people started to use it as a dependency and it would break their stuff.

Modularity with artificial branching will solve both problems.

Is any of this relevant to you?

PS: I also call Python modules libraries... I guess nothing we do is necessarily final. And that might include naming things.

Tom

-- Tom Hughes (tom@compton.nu) http://compton.nu/

On Thu, Jun 8, 2017 at 10:49 PM, Randy Barlow bowlofeggs@fedoraproject.org wrote:

On Thu, 2017-06-08 at 22:17 +0200, Adam Samalik wrote:

...

You add the package and other people start to use it. That's great until you need to change the version, but can't, because other people started to use it as a dependency and it would break their stuff.

I recently heard that it will be impossible to install two packages that depend on different versions of a dependency at the same time. For example, if package A depends on C < 2.0 and package B depends on C >= 2.0, it will be impossible to install A and B on the same system. Is this true? If so, I worry that we will see fracturing in Fedora as packages drift apart in their dependencies. If not so, I apologize for the noise ☺

I would say it's "half truth" right now. :-)

There has been no extra work on changing RPMs. If something conflicts now, it will keep conflicting. But!

My immediate answer would be containers. I like using them to run services, I even have my developer environments in containers - so I don't pollute my system. And I even saw people working on "standalone containers" or "system containers" which are basically containers installed by DNF, started with systemd, having all the config files on the local filesystem, tracked by RPM.

I also heard people talk about Flatpaks built out of modules - that might be a solution for desktop apps.

RPMs... Well, if someone has an application on their server that doesn't run in a container, there are still RPMs on a traditional system. But would you install multiple versions stuff on that single system? Or would other things run in containers? And I'm just curious here, because I managed to use containers for basically everything I need. What about other people?

So, not everything will probably be installable as RPMs on the same system at the same time. But I see the world is going to containers, and I'm thinking if not being to install all RPMs next to each other on a single system is still a real problem. Thoughts?

_______________________________________________

devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-leave@lists.fedoraproject.org

On Fri, Jun 09, 2017 at 08:50:58AM +0200, Adam Samalik wrote:

RPMs... Well, if someone has an application on their server that doesn't run in a container, there are still RPMs on a traditional system. But would you install multiple versions stuff on that single system? Or would other things run in containers? And I'm just curious here, because I managed to use containers for basically everything I need. What about other people?

So, not everything will probably be installable as RPMs on the same system at the same time. But I see the world is going to containers, and I'm thinking if not being to install all RPMs next to each other on a single system is still a real problem. Thoughts?

I can totally see the appeal of containers when you have a uniform cluster infrastructure that is shared by wildly different deployments/users with wildly different requirements. That's what things lke k8s do well.

OTOH, you pay for that nice flexibility with a huge increase in complexity. Why would I want to put up with that if I'm not managing a lot of deployments or, heck, when it's just my own development computer?

I may still do it for fun or because the benefits are really worth it in a specific case, but designating it as the default (and maybe only) way to work seems wrong[1]. Not when all that's actually required is a different version of some library.

[1] Especially when considering that our industry as a whole has a very hard time (euphemism!) producing systems that correctly deal with the complexity that's already there.

On Fri, Aug 18, 2017 at 02:13:17PM -0700, Adam Williamson wrote:

There's an important consequence of this that I only realized today.

pkgdb had an API endpoint, 'collections', which was useful as a reliable source of information about available Fedora releases and their status. It still exists now, until pkgdb is entirely turned off:

https://admin.fedoraproject.org/pkgdb/api/collections/

but as pkgdb was made read-only on August 4th, its data is outdated. It shows Fedora 24 as 'active' (rather than 'EOL'), and it has not had Fedora 27 added (as would usually be the case when it branched).

I swear we talked about this somewhere before. I can't find the ticket, though. My suggestion was to simply replace or redirect to a static page until a replacement can be created. I think it was a pagure ticket somewhere, but I'm not sure where.

On Fri, Aug 18, 2017 at 08:06:22PM -0500, Michael Catanzaro wrote:

...

I swear we talked about this somewhere before. I can't find the ticket, though.

Possibly https://pagure.io/fedora-workstation/issue/23

Yes! That was totally it. Thanks.

Adam Williamson wrote:

pkgdb had an API endpoint, 'collections', which was useful as a reliable source of information about available Fedora releases and their status. It still exists now, until pkgdb is entirely turned off:

https://admin.fedoraproject.org/pkgdb/api/collections/

but as pkgdb was made read-only on August 4th, its data is outdated. It shows Fedora 24 as 'active' (rather than 'EOL'), and it has not had Fedora 27 added (as would usually be the case when it branched).

[snip]

Of course, the 'natural' response to this would be to rewrite things that use the pkgdb collections API to use PDC instead. However, there's a problem with that: PDC does not provide sufficient data. Releases in PDC can only be marked as "active" or "inactive". It seems EOL releases are marked "inactive" and both stable and under-development releases are marked "active" - so it is literally not possible to tell from PDC whether a release is stable or Branched (e.g. right now, both Fedora 26 and Fedora 27 are simply 'active').

It's always the same story in Fedora (both on the infrastructure side and in the distro itself): working software gets decommissioned before there is a complete replacement and before consumers of the APIs are updated to use whatever should be the replacement.

And then we are surprised when reviewers call Fedora a "bleeding-edge distro" or a "RHEL beta".

Kevin Kofler