The Git forge decision (was CPE Weekly: 2020-03-28) - infrastructure - Fedora mailing-lists

29 Mar 2020

      On Sat, Mar 28, 2020 at 4:12 PM Aoife Moloney amoloney@redhat.com wrote:
...
### Other Updates
#### GitForge Decision

After evaluating over 300 user stories from multiple stakeholders we

have aligned on a decision for the Gitforge that CPE will operate for
the coming years. We are opting for Gitlab for our dist git and
project hosting and will continue to run pagure.io with community
assistance.
    * Check out our GitForge decision on the Fedora Community blog
https://communityblog.fedoraproject.org/
    * And at the CentOS blog page
https://blog.centos.org/2020/03/git-forge-decision/

Keep an eye out for mails in the coming months to the devel lists as

we plan transitions and next steps with GitLab

We would like to express our sincere thank you to all who

contributed requirements to us!
I'm going to start with the delivery of this decision sucked. If I
hadn't been alerted to look for this by other folks due to my advocacy
and community building work around Pagure, I would *not* have known
that the decision had been made. This is in contrast to the *big deal*
that was made about starting this "decision process". I don't know if
there were folks counting on nobody noticing this or not, but this is
not a good way to deliver effectively a major decision like this. I
also absolutely *refuse* to deal with the fact that this thread was
split into three mailing lists. All three are connected to this
thread, and I will take responses from all of them and follow them
accordingly.
Enough about that, let's talk about the actual decision.
So, you're going with GitLab. It's interesting to note that the
particulars about going to GitLab are not even figured out. It is
curious that "SAAS" is mentioned very prominently. That made me a bit
more curious, so I looked at the "final" feature requirements.
Needless to say, I was extremely disappointed.
To put it bluntly, there are *zero* free and open source solutions
that satisfy your needs. From this, I understood why you said GitLab
and SaaS. You want GitLab Ultimate, the proprietary solution that
includes several extra features on top to satisfy the following
requirements (quoted from your blog post):
...

There is a need for CentOS Stream to integrate with a kernel workflow that is an automated bot driven merging solution (merge trains). This allows for richer CI capabilities and minimizes the need for human interaction
Gitlab allows for project planning capability which could make multiple trackers such as Taiga redundant, allowing for the planning and tracking to reside within the repo. It would enrich the current ticket based solution that Pagure has evolved into for some groups

These two requirements *alone* automatically force us to GitLab
Enterprise Ultimate Edition, as there is no other solution available
that satisfies those requirements in one product. Merge trains *is* a
feature of Pagure when combined with Zuul, but GitLab's project
planning features do not exist in *any* FOSS product, including GitLab
CE (or GitLab Core as they call it now).
There are mentions of other work that CPE team wants to do to better
improve Fedora. Okay, sure. I even agree with some of them. And the
time bomb that is FAS is definitely worth the attention (note that I
am somewhat involved in the development of the Fedora AAA solution and
am also working on trying to develop a community around it so it
doesn't implode like virtually every other project under the Fedora
umbrella, more on *that* a bit later).
However, nobody has given me or anyone else in the Pagure community
(which yes, is more than Pierre-Yves, thank you very much!) any
concrete details of deficiencies they'd see that is not satisfiable by
the community within a year before now. I've spent a little over a day
analyzing the user stories and thinking about the gaps between Pagure
and what the community wants, and I want to give some perspective here
for some of these. I'm happy to accept refutations and other details
to further enhance the color of these stories, of course.
...
5
As a RH Developer
I need the ability to privately comment on a PR
so that confidential information does not leak outside Red Hat
Ignoring the mountainous levels of terrible problems that such a
feature causes us *now* in the Red Hat Bugzilla, Pierre-Yves and I
were literally discussing this with a Mageian who was interested in
this feature for Pagure weeks ago. We had identified the feature as
not difficult to implement but also simultaneously nobody really
*wanted it now*. It surprises me that this is something that should be
considered an important feature to preserve. It's actually a very
*rare* feature, and does not exist in any forge today, presumably
because it's actually a horrifically bad idea and antithetical to open
development. GitLab itself lacks this feature, in all variants.
...
16
As a general user
I want to be able to create a bot/service account
That integrates with the gitforge in the same way as a human does
This is a nice ask, but it's not a reasonably easy one to implement
with our current system. This is a problem I've struggled with at
$DAYJOB with GitLab as well, and frankly, if you have account
integration with a single-sign-on system (which virtually all
large-scale Git forge instances do), you can't really easily have this
without causing major problems. Not the least of which is how you
resolve account namespace collisions. There are also problems with
maintaining and auditing, too. But it's certainly something I'd like
to see in any forge (note that none implement this today either).
...
29
As a General User
I want to access repos fully over https
For environments where SSH is blocked
I would be really curious if the Red Hat Infrastructure Security guys
have changed their opinion on this after four years of blocking the
development of this feature in Pagure. The two major reasons we don't
have this in Pagure are:
* There is a very strong push to not provide a way to bypass the SSO
(ignoring the fact that SSH keys are effectively a bypass, but most
security people are two-faced about this anyway)
* There is a very strong push to not provide LDAP integration so that
the required HTTP(S) Git proxy server would not be able to be
implemented.
Note that for GitLab, unless you configure it to have access to LDAP
or set up personal access tokens, you cannot use HTTP(S) push at all.
Once again, this totally bypasses SSO. If the same rules that applied
to Pagure apply to GitLab, nobody is getting this feature anytime soon
with GitLab. If the attitude toward this feature has finally changed,
I'd love to know.
Also, for those who don't know, Fedora's Dist-Git supports HTTPS push,
and has for almost a year:
https://fedoraproject.org/wiki/Infrastructure/HTTPS-commits
This is done by *forcing* an SSO flow for *fedpkg* itself and
leveraging it as a credential and auth point. Unfortunately, this is
not a general purpose feature because there's not an easy way to do
that, so it's unavailable on pagure.io at this time. This mechanism
for HTTP push is *not* possible with GitLab, and would be quite
difficult to implement due to the crazy architecture internally. But
if more *normal* HTTPS is acceptable (like through auth tokens or LDAP
auth), then this is something that could be relatively quickly added
to Pagure (there was some old code that never got merged two years
ago, I still have it archived somewhere...).
...
31
As a General User
I can request access rights to a repository
So that I can contribute in a low friction manner
I honestly don't know entirely what this means. Are we talking about
having partial commit access (commit access to only some branches)? If
so, there's a PR out for implementing this in Pagure under review:
https://pagure.io/pagure/pull-request/4786
...
34
As a General User
I want a mobile native app
To allow me contribute while away from my desk
I don't have words for this... You folks know that only GitHub.com has
an official mobile app, right? And the experience is not great...
🤦‍♂️
...
42
As a General User
want a GUI to interface with the system as well as a CLI
so new users have an easier way to interface with it
I'd like to point out that most popular GUIs for Git stuff are not
forge specific and do not even do much for forge integration. But
sure...?
...
44
As a General User
I want a temporary file (gist)
So that I can share code easily
There seems to be a misunderstanding here... Gists are not temporary.
They're lightweight *permanent* Git repos (in GitHub). GitLab stores
them (called Snippets) as permanent things in the database (not a Git
repo). Are you trying to conflate pastebin use-cases here? That's
probably a very bad idea...
...
46
As a General User
I want to be notified of CVEs in my code
so that I can stay on top of critical vulnerabilities
There are *zero* FOSS solutions for this in the forge space. This
feature does *not* exist below GitLab Ultimate (the former Gemnasium
service was integrated into it).
...
47
As a General User
I want integrated keyword support
to allow me automate a lot of my actions such as a rebuild / retest
This is not a forge feature, this is a CI service feature. And note,
GitLab CI does not support this.
...
49
As a General User
I want to gain analytics and insights from my code
so that I can have historic context to make decisions moving forward
GitLab Enterprise features...
...
51
As a General User
I would like to track my work in an Agile manner
allowing me centralise all my planning in my forge and gain insights into how I am working.
GitLab Enterprise features, as mentioned earlier.
...
56
As a General User
I want registry integration
so that I can store dependencies
No, you don't. You *really* don't want this. Are you *sure* you know
what you're asking for? You're suggesting three things here:
* OSS solutions for this (including Fedora's own package/container
infrastructure and hosted quay.io) are not good enough
* You are ready to have even more arbitrary data stored in the Git
system that may not follow our legal compliance
* You are willing to pay the cost of having even more data stored
...
57
As a General User
I want the ability to have a private branch
So that I do not need to leave the code tree I am already in
Just so everyone is *crystal clear*: you literally cannot do this. Git
does not have a concept of private branches or private refs within a
repository. You can have private forks, of course. And Pagure supports
those just fine, just like GitLab and GitHub do.
(also note, we *intentionally* do not have private repos turned on...
if we want this, we could just flip a switch...)
...
62
As a General User
I want automerges when specific acceptance criteria are met
So that I do not need manual intervention
So... Mergify then? This is not *currently* a GitLab feature, and
Mergify does not support GitLab, last I checked. Only GitHub.com. It's
been on my bucket for a while to look at extending Mergify to support
Pagure for this, as it's a really nice feature...
I want to take a moment to reflect on something that has been on my
mind for a while now: Fedora has not done a good job being an umbrella
organization. As an organization, we have done a huge disservice to
all Fedora-affiliated projects by not allocating any community
development effort or marketing effort. I know that Matthew Miller
feels Fedora should evolve to being an operating systems factory[1]
and to some extent that isn't a bad thought. But the Fedora Project
was always intended to be more than just the Fedora Linux
distribution. It has always been intended to be a home for Free and
Open Source innovation. In a Hacker News thread last year[2], I had
reflected on how proud I have been to be part of Fedora because we, as
a community, weren't willing to just give up like so many others do.
When FOSS solutions were inadequate, we built better ones. We've
invented things that didn't exist before, and jump-started
conversations about concepts that people didn't think of before. But
there has been a bit of a dark side to this for Fedora. We've rarely
given our projects an opportunity to grow beyond us. Off the top of my
head, the *only* project that technically *started* in Fedora and
became successful was Ansible. And if I'm being totally brutally
honest, it was only successful because the engineers who were
passionate about it had to quit Red Hat and create AnsibleWorks to
push it to success. It was successful *despite* Fedora. Maybe soon
we'll be able to add HyperKitty and Postorius to that, as I've been
seeing deployments of that come online over the past few months. It's
taken a while, but HyperKitty is finally taking off. There was one
person I talked to who told me that HyperKitty made mailing lists
enjoyable and she didn't know the project came from Fedora originally.
Again, seeing success despite Fedora.
When I talk to folks in other communities and show them some of the
infrastructure projects we've developed as part of trying to build
communities around them, I've literally had people tell me that they
wish they had known we made them before, because it solved all their
problems they were struggling with. That's both amazingly uplifting
and terribly depressing at the same time. I'm not even putting in a
lot of effort and we get so much more out of it. It didn't take a lot
for me to get openSUSE interested in our new AAA solution and
contributing. That tells me that we're just not trying. And really,
that's obvious. Even a simple comparison of the Fedora and openSUSE
project landing pages show that Fedora gives zero attention to the
projects that exist under its umbrella. When you look at the openSUSE
landing page, the distribution and major software projects under the
umbrella are all broadcasted there. It makes it so much easier to
discover and generate interest. I'm not saying I love every aspect of
the openSUSE marketing. Far from it! But this is one thing they do
right that we do terribly wrong. And then we sit back and wonder why
our projects fail to generate interest beyond a few folks in Fedora
itself. It's a self-fulfilling prophecy. This is something we need to
fix for *all* our projects: present and future.
In the end, I think the biggest disappointment of this process is that
it feels like it demonstrates that Fedora leadership and management is
not as committed to its mission and vision[3] as I hoped it was. I
realize that I should not be surprised by this. Most of the leadership
and management are no longer the idealistic people they were when they
first became involved. And it's even harder to be idealistic when it's
so easy to give in when you work for "open source company" that
increasingly uses proprietary software to manage its workflow (to be
fair, I think at this point virtually all major companies do this,
which more or less demonstrates the amoral nature of these entities).
Maybe I'm just an old remnant of a bygone era, but I personally remain
somewhat idealistic, even as I have progressed over the years. I also
remain hopeful that other members of the community are of similar
mind. And perhaps this is a bit of a fool's hope, but I hope the CPE
team reconsiders their decision, or at least would be willing to
provide more context on the gaps they felt pushed them over so they
could be prioritized for Pagure development (and maybe we can develop
them fast enough so that we don't have to switch...).
I think this is also an important indicator that Open Source has *not*
won and it is *not* the default. People who keep saying otherwise need
to seriously look hard at the landscape and realize that we have a
long way to go before Open Source becomes the true default. It
behooves us to become cognizant of this and push for freedom whenever
and wherever we can.
As for me? Well, I will do my best to try to help develop the Pagure
community. I'm still committed to assisting the Free Software
Foundation and other communities with using and contributing to
Pagure. I hope others within the community will consider helping too.
Pagure provides unique features that do not exist in any other forge,
in large part because it is driven by an ideal that open data and
freedom should be core tenants of software project management. And
hey, I hear whispers of new Pagure instances being set up all the
time! We're doing something right here, and it'd be a shame to
squander it.
Heh, the irony is that I started using and contributing to Pagure
*because* I was burned by GitLab...
[1]: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/...
[2]: https://news.ycombinator.com/item?id=19356307
[3]: https://docs.fedoraproject.org/en-US/project/
-- 
真実はいつも一つ！/ Always, there's only one truth!