To sum up what I've heard so far from the developer side:
- I would like to enable tests for my component (yes, I want)
- I will take care of them (really, I see the benefit in CI)
- I want to easily collaborate on tests with qe (direct commits)
- I don't want to give qe commit access to the rpms dist git
This is quite likely the crux of the problem.
Personally, I'm perfectly happy to give write access to any repo to people who have shown that they know what they are doing. We have pull requests in dist-git pagure now, and I think this is a better approach:
- ask QE contributors to submit PRs
- if enough cooperation happens and trust is established, give
privileges to write to the repo directly, possibly with an agreement that this specific person should only touch tests, and not the packaging.
I think it's perfectly fine to never get to point 2.: many many upstream projects require a review from a second person, or sometimes even two reviews before a PR is merged, which means that one _never_ merges their own PR, and another contributor is always involved. We usually don't do this in dist-git, but I'm quite sure that requiring reviews for dist-git changes would raise quality and catch many silly mistakes. Either way, it's nowadays possible to cooperate using a single repo without fully trusting the other person, frictionlessly.
Good point. However, if there is a qe/devel team which prefers to collaborate on tests in a separate repo because this is the most efficient way they found so far, why should we stop them and try to enforce a less efficient workflow? (Which they can workaround by a different git repo.)
- I want to efficiently maintain tests long-term
- I want to have just a single place for my tests (no duplication)
- I don't want to backport new tests to old branches (enable once)
- I want to easily enable testing for all supported branches
Those four items depend strongly on the package. My thinking is biased by some specific usecases (mainly systemd), but I'm sure many other packages are like that: a lot of tests would be for new functionality, and then the tests _are_ tied to the branch being tested.
While I agree that keeping tests separate avoids a bit of effort required to update multiple branches, this effort isn't very big. If the tests indeed apply cleanly to all branches, then it's just a matter of doing 'git merge' or 'git cherry-pick' once per branch.
- I want to keep rpms dist git clean (just metadata & patches)
Meh.
- I want to run all available relevant tests (not to list them)
- I want to execute set of tests based on a tag (e.g. Tier1)
Those two sound like stuff that should be solved in the tooling, whatever is used to run tests.
- I want an easy way to clone tests (fedpkg clone tests/pkg)
Tests alone make no sense, you need something to test, and cloning one repo is easier than cloning two repos, so there's no advantage to the split here.
But "fedpkg clone tests" is easier than cloning from a "random" git repository where I was forced to save my tests because I was not allowed to save them in Fedora tests namespace.
I believe the tests namespace resolves them all.
None of those arguments convince me that separate repos for tests are a good default. Sure, there are specific packages where this will make sense, or specific packagers with idiosyncratic workflows, but I'd rather "start small", with the simple configuration, and only move specific packages to the more complicated setup once that proves to be required.
Why default? Test namespace should be an option. Not the default. Storing tests directly in dist git is and will be possible. Anybody who finds this as a better way can do so. But why enforcing this approach to all?
psss...