0) The v4.14.10 stable updates adds a new executable (tools/objtool/sync- check.sh). Somehow this was added non-executable during my local build of v4.14.10 (on fc26, that is). This made the build fail:
[...] + make -s ARCH=x86_64 V=1 -j4 bzImage make[2]: execvp: ./sync-check.sh: Permission denied make[2]: *** [Makefile:49: [...]/BUILD/kernel-4.14.fc26/linux-4.14.10-1.local0.fc26.x86_64/tools/objtool/objtool] Error 127 make[1]: *** [Makefile:62: objtool] Error 2 make: *** [Makefile:1623: tools/objtool] Error 2 make: *** Waiting for unfinished jobs.... error: Bad exit status from /var/tmp/rpm-tmp.fTUkoT (%build)
RPM build errors: Bad exit status from /var/tmp/rpm-tmp.fTUkoT (%build)
Anybody else seeing this?
1) Switching the specfile from patch to "git apply" seems to do the right thing. This is what I tried:
diff --git a/kernel.spec b/kernel.spec index 965345c2a26e..b2a1ffbe843d 100644 --- a/kernel.spec +++ b/kernel.spec @@ -1267,8 +1267,9 @@ fi # released_kernel with possible stable updates %if 0%{?stable_base} # This is special because the kernel spec is hell and nothing is consistent -xzcat %{SOURCE5000} | patch -p1 -F1 -s -git commit -a -m "Stable update" +xzcat %{SOURCE5000} | git apply - +git add -A +git commit -m "Stable update" %endif
# Drop some necessary files from the source dir into the buildroot
2) Would it make sense to further gitify the specfile and move from patch to "git apply" here (and a few other places)? Or should we expect patch to do the right thing? (In the latter case I guess I might have to report a bug against patch.)
Thanks,
Paul Bolle
On 12/30/2017 04:52 AM, Paul Bolle wrote:
- The v4.14.10 stable updates adds a new executable (tools/objtool/sync-
check.sh). Somehow this was added non-executable during my local build of v4.14.10 (on fc26, that is). This made the build fail:
[...]
- make -s ARCH=x86_64 V=1 -j4 bzImage
make[2]: execvp: ./sync-check.sh: Permission denied make[2]: *** [Makefile:49: [...]/BUILD/kernel-4.14.fc26/linux-4.14.10-1.local0.fc26.x86_64/tools/objtool/objtool] Error 127 make[1]: *** [Makefile:62: objtool] Error 2 make: *** [Makefile:1623: tools/objtool] Error 2 make: *** Waiting for unfinished jobs.... error: Bad exit status from /var/tmp/rpm-tmp.fTUkoT (%build)
RPM build errors: Bad exit status from /var/tmp/rpm-tmp.fTUkoT (%build)
Anybody else seeing this?
Yes, this was something that needed a fixup in rawhide. I added the appropriate chmod. 4.14.10 is now building for F27 and F26.
- Switching the specfile from patch to "git apply" seems to do the right
thing. This is what I tried:
diff --git a/kernel.spec b/kernel.spec index 965345c2a26e..b2a1ffbe843d 100644 --- a/kernel.spec +++ b/kernel.spec @@ -1267,8 +1267,9 @@ fi # released_kernel with possible stable updates %if 0%{?stable_base} # This is special because the kernel spec is hell and nothing is consistent -xzcat %{SOURCE5000} | patch -p1 -F1 -s -git commit -a -m "Stable update" +xzcat %{SOURCE5000} | git apply - +git add -A +git commit -m "Stable update" %endif
# Drop some necessary files from the source dir into the buildroot
- Would it make sense to further gitify the specfile and move from patch to
"git apply" here (and a few other places)? Or should we expect patch to do the right thing? (In the latter case I guess I might have to report a bug against patch.)
We've generally been expecting patch to do the right thing and it's worked so far. I'm not opposed to gitifying more parts of the spec file but do you have a particular reason for doing so? It seemed like when Josh made the original change for git he had a few things in mind.
Thanks, Laura
On Sun, 2017-12-31 at 18:13 -0800, Laura Abbott wrote:
On 12/30/2017 04:52 AM, Paul Bolle wrote:
- Would it make sense to further gitify the specfile and move from patch to
"git apply" here (and a few other places)? Or should we expect patch to do the right thing? (In the latter case I guess I might have to report a bug against patch.)
We've generally been expecting patch to do the right thing and it's worked so far.
A few web searches helped me remember something comparable: https://lkml.org/lkml/2015/1/26/692 .
What I did remember without hitting the web was that "git apply" is meant to be a drop in replacement for patch (for the patches "git diff" creates, that is). Since we are only using patches created by git nowadays, aren't we, we might as well use "git apply" for them.
I'm not opposed to gitifying more parts of the spec file but do you have a particular reason for doing so?
Preventing problems like this (and the one from 2015).
Another thing is that I noticed that the git repository we create during rpmbuild doesn't properly track all changes. In this case we do not "git add" new files. That is why I changed git commit -a -m "..."
to git add -A git commit -m "..."
I also noticed that the removal of .gitignore files isn't tracked. And it's a bit annoying to have the removal of those files show up in "git status". "git status" was one of the things I did when pinpointing this issue.
Of course these are just small things, but they do make pinpointing stuff like this a bit more annoying than it should be.
It seemed like when Josh made the original change for git he had a few things in mind.
A bit off topic: I suppose at the ultimate goal is to do rpmbuild from within a proper git clone of the kernel repository. Ie, using a branch with Fedora's patches, a specfile, and whatever else we need. Perhaps further gitifying the current specfile helps to reach that goal. Not sure, though.
Thanks,
Paul Bolle
On 01/02/2018 08:35 AM, Paul Bolle wrote:
On Sun, 2017-12-31 at 18:13 -0800, Laura Abbott wrote:
On 12/30/2017 04:52 AM, Paul Bolle wrote:
- Would it make sense to further gitify the specfile and move from patch to
"git apply" here (and a few other places)? Or should we expect patch to do the right thing? (In the latter case I guess I might have to report a bug against patch.)
We've generally been expecting patch to do the right thing and it's worked so far.
A few web searches helped me remember something comparable: https://lkml.org/lkml/2015/1/26/692 .
What I did remember without hitting the web was that "git apply" is meant to be a drop in replacement for patch (for the patches "git diff" creates, that is). Since we are only using patches created by git nowadays, aren't we, we might as well use "git apply" for them.
I'm not opposed to gitifying more parts of the spec file but do you have a particular reason for doing so?
Preventing problems like this (and the one from 2015).
Another thing is that I noticed that the git repository we create during rpmbuild doesn't properly track all changes. In this case we do not "git add" new files. That is why I changed git commit -a -m "..."
to git add -A git commit -m "..."
I also noticed that the removal of .gitignore files isn't tracked. And it's a bit annoying to have the removal of those files show up in "git status". "git status" was one of the things I did when pinpointing this issue.
Of course these are just small things, but they do make pinpointing stuff like this a bit more annoying than it should be.
Yes, I discovered the git add bit as well. I think it's probably worth converting over to using git apply assuming nobody else can think of objections.
It seemed like when Josh made the original change for git he had a few things in mind.
A bit off topic: I suppose at the ultimate goal is to do rpmbuild from within a proper git clone of the kernel repository. Ie, using a branch with Fedora's patches, a specfile, and whatever else we need. Perhaps further gitifying the current specfile helps to reach that goal. Not sure, though.
If you're referring to the rpmbuild in the upstream kernel tree, I don't think it's particularly feasible for an official distribution. We end up doing a lot of extra things on top of what's necessary just to package the kernel and modules. I do want to experiment with making the upstream rpmbuild more useful though.
Thanks,
Paul Bolle
Thanks, Laura
On Tue, 2018-01-02 at 12:32 -0800, Laura Abbott wrote:
On 01/02/2018 08:35 AM, Paul Bolle wrote:
A bit off topic: I suppose at the ultimate goal is to do rpmbuild from within a proper git clone of the kernel repository. Ie, using a branch with Fedora's patches, a specfile, and whatever else we need. Perhaps further gitifying the current specfile helps to reach that goal. Not sure, though.
If you're referring to the rpmbuild in the upstream kernel tree, I don't think it's particularly feasible for an official distribution. We end up doing a lot of extra things on top of what's necessary just to package the kernel and modules. I do want to experiment with making the upstream rpmbuild more useful though.
That might be very useful, and probably is a lot of work, but that wasn't what I was hinting at. Sorry.
We're juggling tarballs and patches to build Fedora specific kernel rpms while all information that we need is already in the kernel git repo clone that's already on our machines. So what I was hinting at that I would like to do git remote add kernel-pkg fedoraproject.org/whatever
and then (in a kernel-pkg related branch) do rpmbuild --i-want-a-pony --foo --bar kernel.spec
and end up with Fedora specific kernel rpms.
Sounds simple, so it must be a tricky problem to solve.
Thanks,
Paul Bolle
On Tue, Jan 2, 2018 at 4:55 PM, Paul Bolle pebolle@tiscali.nl wrote:
On Tue, 2018-01-02 at 12:32 -0800, Laura Abbott wrote:
On 01/02/2018 08:35 AM, Paul Bolle wrote:
A bit off topic: I suppose at the ultimate goal is to do rpmbuild from within a proper git clone of the kernel repository. Ie, using a branch with Fedora's patches, a specfile, and whatever else we need. Perhaps further gitifying the current specfile helps to reach that goal. Not sure, though.
If you're referring to the rpmbuild in the upstream kernel tree, I don't think it's particularly feasible for an official distribution. We end up doing a lot of extra things on top of what's necessary just to package the kernel and modules. I do want to experiment with making the upstream rpmbuild more useful though.
That might be very useful, and probably is a lot of work, but that wasn't what I was hinting at. Sorry.
We're juggling tarballs and patches to build Fedora specific kernel rpms while all information that we need is already in the kernel git repo clone that's already on our machines. So what I was hinting at that I would like to do git remote add kernel-pkg fedoraproject.org/whatever
and then (in a kernel-pkg related branch) do rpmbuild --i-want-a-pony --foo --bar kernel.spec
and end up with Fedora specific kernel rpms.
Sounds simple, so it must be a tricky problem to solve.
If you're talking about local builds, it isn't tricky at all. Someone would just need to adjust the kernel.spec to do it and/or write steps to have your local git tree in a state that the existing spec could use. You could even use the existing exploded tree because the configs are all there as well.
If you're talking about building Fedora kernels officially from an exploded tree, there are two tricky issues.
1) You still have to at least create a tarball and upload that because koji can't build from exploded source. That means you're uploading a massive tarball every day and it's terrible.
2) The tracking of patches Fedora carries on top of upstream results in forced rebases. You see that with the existing exploded tree we have on kernel.org, but that literally is just an exploded tree of builds. It's not really great for development. For development purposes, the forced rebases makes it really crappy to work with. The alternative is a ton of merges, which would be possible but alas is not great either.
josh
On Tue, 2018-01-02 at 17:27 -0500, Josh Boyer wrote:
If you're talking about local builds, it isn't tricky at all. Someone would just need to adjust the kernel.spec to do it and/or write steps to have your local git tree in a state that the existing spec could use. You could even use the existing exploded tree because the configs are all there as well.
If the end result is something that closely tracks kernel.git that sounds rather nice. (Even better would be a tree that kernel.git itself uses, but now I'm getting carried away a bit.)
I should try to create something that somehow-sort-of does what I want first, I guess. Bother. I'd rather just snap my fingers!
If you're talking about building Fedora kernels officially from an exploded tree, there are two tricky issues.
- You still have to at least create a tarball and upload that because
koji can't build from exploded source. That means you're uploading a massive tarball every day and it's terrible.
But koji isn't carved in stone, is it?
- The tracking of patches Fedora carries on top of upstream results
in forced rebases. You see that with the existing exploded tree we have on kernel.org, but that literally is just an exploded tree of builds. It's not really great for development. For development purposes, the forced rebases makes it really crappy to work with. The alternative is a ton of merges, which would be possible but alas is not great either.
Sorry, I'm only consuming the kernel.git efforts, so it's hard for me to truly understand the effort involved.
Thanks for taking the time to respond to my semi-cooked ideas!
Paul Bolle
On Tue, Jan 2, 2018 at 11:56 PM, Paul Bolle pebolle@tiscali.nl wrote:
On Tue, 2018-01-02 at 17:27 -0500, Josh Boyer wrote:
If you're talking about local builds, it isn't tricky at all. Someone would just need to adjust the kernel.spec to do it and/or write steps to have your local git tree in a state that the existing spec could use. You could even use the existing exploded tree because the configs are all there as well.
If the end result is something that closely tracks kernel.git that sounds rather nice. (Even better would be a tree that kernel.git itself uses, but now I'm getting carried away a bit.)
I should try to create something that somehow-sort-of does what I want first, I guess. Bother. I'd rather just snap my fingers!
If you're talking about building Fedora kernels officially from an exploded tree, there are two tricky issues.
- You still have to at least create a tarball and upload that because
koji can't build from exploded source. That means you're uploading a massive tarball every day and it's terrible.
But koji isn't carved in stone, is it?
No, but in the current status quo it can't pull from random sources (Fedora config, not koji in general), this is an explicit decision for audit and related requirements so what goes into a build can be reproduced and reviewed.
- The tracking of patches Fedora carries on top of upstream results
in forced rebases. You see that with the existing exploded tree we have on kernel.org, but that literally is just an exploded tree of builds. It's not really great for development. For development purposes, the forced rebases makes it really crappy to work with. The alternative is a ton of merges, which would be possible but alas is not great either.
Sorry, I'm only consuming the kernel.git efforts, so it's hard for me to truly understand the effort involved.
Thanks for taking the time to respond to my semi-cooked ideas!
Paul Bolle _______________________________________________ kernel mailing list -- kernel@lists.fedoraproject.org To unsubscribe send an email to kernel-leave@lists.fedoraproject.org
On Wed, 2018-01-03 at 00:00 +0000, Peter Robinson wrote:
On Tue, Jan 2, 2018 at 11:56 PM, Paul Bolle pebolle@tiscali.nl wrote:
But koji isn't carved in stone, is it?
No, but in the current status quo it can't pull from random sources (Fedora config, not koji in general), this is an explicit decision for audit and related requirements so what goes into a build can be reproduced and reviewed.
Before I start to ask all kinds of silly questions or make silly suggestions: this is documented somewhere, isn't it?
Paul Bolle
On Sun, Dec 31, 2017 at 9:13 PM, Laura Abbott labbott@redhat.com wrote:
On 12/30/2017 04:52 AM, Paul Bolle wrote:
- The v4.14.10 stable updates adds a new executable (tools/objtool/sync-
check.sh). Somehow this was added non-executable during my local build of v4.14.10 (on fc26, that is). This made the build fail:
[...]
- make -s ARCH=x86_64 V=1 -j4 bzImage
make[2]: execvp: ./sync-check.sh: Permission denied make[2]: *** [Makefile:49: [...]/BUILD/kernel-4.14.fc26/linux-4.14.10-1.local0.fc26.x86_64/tools/objtool/objtool] Error 127 make[1]: *** [Makefile:62: objtool] Error 2 make: *** [Makefile:1623: tools/objtool] Error 2 make: *** Waiting for unfinished jobs.... error: Bad exit status from /var/tmp/rpm-tmp.fTUkoT (%build)
RPM build errors: Bad exit status from /var/tmp/rpm-tmp.fTUkoT (%build)
Anybody else seeing this?
Yes, this was something that needed a fixup in rawhide. I added the appropriate chmod. 4.14.10 is now building for F27 and F26.
- Switching the specfile from patch to "git apply" seems to do the right
thing. This is what I tried:
diff --git a/kernel.spec b/kernel.spec index 965345c2a26e..b2a1ffbe843d 100644 --- a/kernel.spec +++ b/kernel.spec @@ -1267,8 +1267,9 @@ fi # released_kernel with possible stable updates %if 0%{?stable_base} # This is special because the kernel spec is hell and nothing is consistent -xzcat %{SOURCE5000} | patch -p1 -F1 -s -git commit -a -m "Stable update" +xzcat %{SOURCE5000} | git apply - +git add -A +git commit -m "Stable update" %endif # Drop some necessary files from the source dir into the buildroot
- Would it make sense to further gitify the specfile and move from patch
to "git apply" here (and a few other places)? Or should we expect patch to do the right thing? (In the latter case I guess I might have to report a bug against patch.)
We've generally been expecting patch to do the right thing and it's worked so far. I'm not opposed to gitifying more parts of the spec file but do you have a particular reason for doing so? It seemed like when Josh made the original change for git he had a few things in mind.
I have to put on my time-machine cap to even try and remember. Here's the best I can come up with.
If you look at where the cases where patch is being used, you'll notice that some of them are before 'git init' is even run. So as it stands currently, you can't use git apply because the git repo doesn't actually exist at those points. Why that is boils down to 2 things:
1) Laziness and/or change minimalization when we introduced this. You might be able to move around the git init so you can use git apply, but I just didn't bother at the time for reasons I don't remember.
2) Upstream sucks at patch generation. If you look at the stable or RC patches, they aren't nice mboxes of discreet commits. They're giant, code-only diffs that don't preserve individual changes at all. Looking at that you wind up just applying the changes wholesale and then doing a commit with "baseline" as the commit message. Given that the commit logs of the git tree during the build itself are useless anyway (because koji doesn't read the commit logs), it doesn't matter but it's still disappointing. It means we wind up having to create the exploded tree with tooling after the fact. Oh well.
So if you want to use git apply instead of patch, I have no objections that I can remember. It'll just require some extra work to make sure the git repo actually exists and that doesn't break other things.
josh
On Tue, 2018-01-02 at 16:28 -0500, Josh Boyer wrote:
So if you want to use git apply instead of patch, I have no objections that I can remember. It'll just require some extra work to make sure the git repo actually exists and that doesn't break other things.
Git apply doesn't need a git repo. It is designed to be a patch replacement.
(There's probably a lot of legacy stuff patch handles that "git apply" doesn't, but no-one cares.)
Thanks,
Paul Bolle
On Tue, Jan 2, 2018 at 4:35 PM, Paul Bolle pebolle@tiscali.nl wrote:
On Tue, 2018-01-02 at 16:28 -0500, Josh Boyer wrote:
So if you want to use git apply instead of patch, I have no objections that I can remember. It'll just require some extra work to make sure the git repo actually exists and that doesn't break other things.
Git apply doesn't need a git repo. It is designed to be a patch replacement.
Interesting. I wasn't aware of that. Seems odd?
(There's probably a lot of legacy stuff patch handles that "git apply" doesn't, but no-one cares.)
True, but I guess I wonder why they bothered designing a patch replacement to begin with.
josh
On Tue, 2018-01-02 at 17:21 -0500, Josh Boyer wrote:
On Tue, Jan 2, 2018 at 4:35 PM, Paul Bolle pebolle@tiscali.nl wrote:
Git apply doesn't need a git repo. It is designed to be a patch replacement.
Interesting. I wasn't aware of that. Seems odd?
Apparently a conscious decision: https://lkml.org/lkml/2013/7/24/520
(There's probably a lot of legacy stuff patch handles that "git apply" doesn't, but no-one cares.)
True, but I guess I wonder why they bothered designing a patch replacement to begin with.
Kill dependencies on other projects?
By getting people to use "git apply" instead of patch they are pushed towards the place where it's obvious to ask why they even bother with patches (once they realize they're applying patches that patch itself can't even handle). And it's jut clone and fetch and pull and whatever thereafter. Dunno.
I'm speculating here as Google was of no use to me here.
Thanks,
Paul Bolle
On Sat, Dec 30, 2017 at 01:52:49PM +0100, Paul Bolle wrote:
- The v4.14.10 stable updates adds a new executable (tools/objtool/sync-
check.sh). Somehow this was added non-executable during my local build of v4.14.10 (on fc26, that is). This made the build fail:
[...]
- make -s ARCH=x86_64 V=1 -j4 bzImage
make[2]: execvp: ./sync-check.sh: Permission denied make[2]: *** [Makefile:49: [...]/BUILD/kernel-4.14.fc26/linux-4.14.10-1.local0.fc26.x86_64/tools/objtool/objtool] Error 127 make[1]: *** [Makefile:62: objtool] Error 2 make: *** [Makefile:1623: tools/objtool] Error 2 make: *** Waiting for unfinished jobs.... error: Bad exit status from /var/tmp/rpm-tmp.fTUkoT (%build)
RPM build errors: Bad exit status from /var/tmp/rpm-tmp.fTUkoT (%build)
Anybody else seeing this?
- Switching the specfile from patch to "git apply" seems to do the right
thing. This is what I tried:
diff --git a/kernel.spec b/kernel.spec index 965345c2a26e..b2a1ffbe843d 100644 --- a/kernel.spec +++ b/kernel.spec @@ -1267,8 +1267,9 @@ fi # released_kernel with possible stable updates %if 0%{?stable_base} # This is special because the kernel spec is hell and nothing is consistent -xzcat %{SOURCE5000} | patch -p1 -F1 -s -git commit -a -m "Stable update" +xzcat %{SOURCE5000} | git apply -
if you get rid of '-F1' does that cause it to work? Or if you add '-C1' to git-apply, does that make it fail? I am assuming the strict context is the difference here.
Cheers, Don
+git add -A +git commit -m "Stable update" %endif
# Drop some necessary files from the source dir into the buildroot
- Would it make sense to further gitify the specfile and move from patch to
"git apply" here (and a few other places)? Or should we expect patch to do the right thing? (In the latter case I guess I might have to report a bug against patch.)
Thanks,
Paul Bolle _______________________________________________ kernel mailing list -- kernel@lists.fedoraproject.org To unsubscribe send an email to kernel-leave@lists.fedoraproject.org
On Tue, 2018-01-02 at 09:50 -0500, Don Zickus wrote:
On Sat, Dec 30, 2017 at 01:52:49PM +0100, Paul Bolle wrote:
diff --git a/kernel.spec b/kernel.spec index 965345c2a26e..b2a1ffbe843d 100644 --- a/kernel.spec +++ b/kernel.spec @@ -1267,8 +1267,9 @@ fi # released_kernel with possible stable updates %if 0%{?stable_base} # This is special because the kernel spec is hell and nothing is consistent -xzcat %{SOURCE5000} | patch -p1 -F1 -s -git commit -a -m "Stable update" +xzcat %{SOURCE5000} | git apply -
if you get rid of '-F1' does that cause it to work?
No, it still fails.
Or if you add '-C1' to git-apply, does that make it fail?
No, I still get 755 file permissions.
(Nit: I think '-C2' is the equivalent of patch's '-F1', given the default of three lines of context in patches, so I used '-C2'.)
I am assuming the strict context is the difference here.
Thanks,
Paul Bolle
On Tue, Jan 02, 2018 at 05:13:59PM +0100, Paul Bolle wrote:
On Tue, 2018-01-02 at 09:50 -0500, Don Zickus wrote:
On Sat, Dec 30, 2017 at 01:52:49PM +0100, Paul Bolle wrote:
diff --git a/kernel.spec b/kernel.spec index 965345c2a26e..b2a1ffbe843d 100644 --- a/kernel.spec +++ b/kernel.spec @@ -1267,8 +1267,9 @@ fi # released_kernel with possible stable updates %if 0%{?stable_base} # This is special because the kernel spec is hell and nothing is consistent -xzcat %{SOURCE5000} | patch -p1 -F1 -s -git commit -a -m "Stable update" +xzcat %{SOURCE5000} | git apply -
if you get rid of '-F1' does that cause it to work?
No, it still fails.
Or if you add '-C1' to git-apply, does that make it fail?
No, I still get 755 file permissions.
(Nit: I think '-C2' is the equivalent of patch's '-F1', given the default of three lines of context in patches, so I used '-C2'.)
Nevermind. I was half asleep when I responded and didn't notice it was a file permissions problem. Laura fixed it with chmod, which patch doesn't do. Whereas git-apply can.
Cheers, Don
kernel@lists.fedoraproject.org
-
Don Zickus -
Josh Boyer -
Laura Abbott -
Paul Bolle -
Peter Robinson