Hi,
With the new Go 1.10 I get
+ echo /usr/share/gocode/src/github.com/hashicorp/go-discover/cmd/discover/main.go + install -m 0755 -vd /builddir/build/BUILDROOT/golang-github-hashicorp-discover-0-0.5.0.20180108git7642001.fc28.llt.x86_64/usr/bin install: creating directory '/builddir/build/BUILDROOT/golang-github-hashicorp-discover-0-0.5.0.20180108git7642001.fc28.llt.x86_64/usr/bin' + install -m 0755 -vp _bin/discover /builddir/build/BUILDROOT/golang-github-hashicorp-discover-0-0.5.0.20180108git7642001.fc28.llt.x86_64/usr/bin/ '_bin/discover' -> '/builddir/build/BUILDROOT/golang-github-hashicorp-discover-0-0.5.0.20180108git7642001.fc28.llt.x86_64/usr/bin/discover' + /usr/lib/rpm/find-debuginfo.sh -j4 --strict-build-id -m -i --build-id-seed 0-0.5.0.20180108git7642001.fc28.llt --unique-debug-suffix -0-0.5.0.20180108git7642001.fc28.llt.x86_64 --unique-debug-src-base golang-github-hashicorp-discover-0-0.5.0.20180108git7642001.fc28.llt.x86_64 --run-dwz --dwz-low-mem-die-limit 10000000 --dwz-max-die-limit 110000000 -S debugsourcefiles.list /builddir/build/BUILD/go-discover-7642001b443a3723e2aba277054f16d1df172d97 extracting debug info from /builddir/build/BUILDROOT/golang-github-hashicorp-discover-0-0.5.0.20180108git7642001.fc28.llt.x86_64/usr/bin/discover dwz: dwz.c:8790: write_die: Assertion `refd != NULL' failed. /usr/lib/rpm/find-debuginfo.sh: line 518: 1870 Aborted (core dumped) dwz $dwz_opts ${dwz_files[@]} /usr/lib/rpm/sepdebugcrcfix: Updated 0 CRC32s, 1 CRC32s did match.
in golang-github-hashicorp-discover
Any idea if it's due to Go 1.10 or another fedora-devel change?
Regards,
----- Original Message -----
From: "nicolas mailhot" nicolas.mailhot@laposte.net To: golang@lists.fedoraproject.org, "Jakub Cajka" jcajka@redhat.com, "Development discussions related to Fedora" devel@lists.fedoraproject.org Sent: Friday, January 12, 2018 4:32:22 PM Subject: Re: F28 System Wide Change: Golang 1.10a
Hi,
With the new Go 1.10 I get
- echo
/usr/share/gocode/src/github.com/hashicorp/go-discover/cmd/discover/main.go
- install -m 0755 -vd
/builddir/build/BUILDROOT/golang-github-hashicorp-discover-0-0.5.0.20180108git7642001.fc28.llt.x86_64/usr/bin install: creating directory '/builddir/build/BUILDROOT/golang-github-hashicorp-discover-0-0.5.0.20180108git7642001.fc28.llt.x86_64/usr/bin'
- install -m 0755 -vp _bin/discover
/builddir/build/BUILDROOT/golang-github-hashicorp-discover-0-0.5.0.20180108git7642001.fc28.llt.x86_64/usr/bin/ '_bin/discover' -> '/builddir/build/BUILDROOT/golang-github-hashicorp-discover-0-0.5.0.20180108git7642001.fc28.llt.x86_64/usr/bin/discover'
- /usr/lib/rpm/find-debuginfo.sh -j4 --strict-build-id -m -i --build-id-seed
0-0.5.0.20180108git7642001.fc28.llt --unique-debug-suffix -0-0.5.0.20180108git7642001.fc28.llt.x86_64 --unique-debug-src-base golang-github-hashicorp-discover-0-0.5.0.20180108git7642001.fc28.llt.x86_64 --run-dwz --dwz-low-mem-die-limit 10000000 --dwz-max-die-limit 110000000 -S debugsourcefiles.list /builddir/build/BUILD/go-discover-7642001b443a3723e2aba277054f16d1df172d97 extracting debug info from /builddir/build/BUILDROOT/golang-github-hashicorp-discover-0-0.5.0.20180108git7642001.fc28.llt.x86_64/usr/bin/discover dwz: dwz.c:8790: write_die: Assertion `refd != NULL' failed. /usr/lib/rpm/find-debuginfo.sh: line 518: 1870 Aborted (core dumped) dwz $dwz_opts ${dwz_files[@]} /usr/lib/rpm/sepdebugcrcfix: Updated 0 CRC32s, 1 CRC32s did match.
in golang-github-hashicorp-discover
Any idea if it's due to Go 1.10 or another fedora-devel change?
Regards,
-- Nicolas Mailhot
I would suspect debug info extraction based on the log, turning off debug package should workaround the issue. Other question if debug info extraction fails due to bug in it or if it is broken by broken debug info in golang binaries. I haven't stumbled on this while rebuilding distro with 1.10 beta1(does the build pass in f27?), this package is not on the list of failed builds(still processing them).
JC
Hi Jakub
I'm not sure if the package exists in devel, of if we're unbundling it from some other package, or if we're executing unit tests previously ignored
The core dump does not stop the package build
Will investigate some more…
Regards,
Hi,
So, to be more scientific, I did a complete rebuild from scratch of my Go spec stash in rawhide, both with Go 1.10 and forcing the previous Go 1.9.2 (about 300 packages, some of which are not supposed to build yet because I still need to finish the unbundling and bootstraping of docker in addition to consul, etcd, cobra and runc)
I have 58 more packages failing with Go 1.10 (though some are probably fallouts that depend on a packages Go 1.10 does not like). Starting with Google’s own golang.og/x/debug!
Maybe Go 1.10 is not ready for use yet?
Build logs of both runs by PM, they take around 100 MiB each uncompressed
Regards,
----- Original Message -----
From: "nicolas mailhot" nicolas.mailhot@laposte.net To: "Jakub Cajka" jcajka@redhat.com Cc: golang@lists.fedoraproject.org, "Development discussions related to Fedora" devel@lists.fedoraproject.org Sent: Monday, January 15, 2018 2:22:26 PM Subject: Re: F28 System Wide Change: Golang 1.10a
Hi,
So, to be more scientific, I did a complete rebuild from scratch of my Go spec stash in rawhide, both with Go 1.10 and forcing the previous Go 1.9.2 (about 300 packages, some of which are not supposed to build yet because I still need to finish the unbundling and bootstraping of docker in addition to consul, etcd, cobra and runc)
I have 58 more packages failing with Go 1.10 (though some are probably fallouts that depend on a packages Go 1.10 does not like). Starting with Google’s own golang.og/x/debug!
Maybe Go 1.10 is not ready for use yet?
Have you investigated the build failures to have such strong claim?
I'm finishing going through distro wide rebuild(all ~500 packages) and roughly 74 packages are failing to build. Nearly all build failures are caused by errors(inefficiencies) in the packages code or stale/inactive packagers, with no direct relation to the go compiler itself. I see less than 1% of packages that look like candidates for golang bugs.
I will post results when I finish(in ~1-2 days).
JC
Build logs of both runs by PM, they take around 100 MiB each uncompressed
Regards,
-- Nicolas Mailhot _______________________________________________ golang mailing list -- golang@lists.fedoraproject.org To unsubscribe send an email to golang-leave@lists.fedoraproject.org
De: "Jakub Cajka"
Have you investigated the build failures to have such strong claim?
I, obviously hadn't have the time to investigate each failure
However, given the builds ran in parallel on the same computer, pointing to the same repositories (with the exception of Go 1.9 forcing in one case), and a large part of the spec stash completely replaces the Fedora Go package baseline with up to date code (often the upstream last commit or release) that actually runs unit tests I'm pretty sure you can not blame the failures on outdated packages.
On the contrary I suspect some of the packages that pass in your tests are obsolete code with disabled unit tests so you're not seeing all the problems.
Regards,
----- Original Message -----
From: "nicolas mailhot" nicolas.mailhot@laposte.net To: golang@lists.fedoraproject.org Sent: Monday, January 15, 2018 2:48:36 PM Subject: Re: F28 System Wide Change: Golang 1.10a
De: "Jakub Cajka"
Have you investigated the build failures to have such strong claim?
I, obviously hadn't have the time to investigate each failure
However, given the builds ran in parallel on the same computer, pointing to the same repositories (with the exception of Go 1.9 forcing in one case), and a large part of the spec stash completely replaces the Fedora Go package baseline with up to date code (often the upstream last commit or release) that actually runs unit tests I'm pretty sure you can not blame the failures on outdated packages.
I'm not generally blaming it on outdated packages(although there are some), I'm mostly blaming it on code that is not following best coding practices(https://tip.golang.org/doc/go1.10#test) or assume some not guaranteed/universal assumptions.
On the contrary I suspect some of the packages that pass in your tests are obsolete code with disabled unit tests so you're not seeing all the problems.
Can you back it up it? Or is it just your assumption? Actually most of the failures approx 70% is occurring in tests(see up mentioned link).
JC
Regards,
Nicolas Mailhot _______________________________________________ golang mailing list -- golang@lists.fedoraproject.org To unsubscribe send an email to golang-leave@lists.fedoraproject.org
De: "Jakub Cajka"
From: "nicolas mailhot"
I'm not generally blaming it on outdated packages(although there are some), I'm mostly blaming it > on code that is not following best coding practices(https://tip.golang.org/doc/go1.10#test) or assume some not guaranteed/universal assumptions.
Well, given that Google’s own golang.org/x/debug fails, I suspect the beast coding practices Go 1.10 relies on are not that well prevalent.
On the contrary I suspect some of the packages that pass in your tests are obsolete code with disabled unit tests so you're not seeing all the problems.
Can you back it up it? Or is it just your assumption? Actually most of the failures approx 70% is > occurring in tests(see up mentioned link).
Thus, counting the tests which had not been written yet in the obsolete versions currently packaged in rawhide, plus all the tests which had never been activated there, it is not surprising current code with unit tests on sees more failures
And getting rawhide to the point one can actually build current Go apps on it requires replacing all this legacy code with current versions
Regards
----- Original Message -----
From: "nicolas mailhot" nicolas.mailhot@laposte.net To: golang@lists.fedoraproject.org Sent: Monday, January 15, 2018 3:47:00 PM Subject: Re: F28 System Wide Change: Golang 1.10a
De: "Jakub Cajka"
From: "nicolas mailhot"
I'm not generally blaming it on outdated packages(although there are some), I'm mostly blaming it > on code that is not following best coding practices(https://tip.golang.org/doc/go1.10#test) or assume some not guaranteed/universal assumptions.
Well, given that Google’s own golang.org/x/debug fails, I suspect the beast coding practices Go 1.10 relies on are not that well prevalent.
If you don't share how, why it fails we can be just speculating(maybe it is bug in your packaging, build setup). I wouldn't assume google in name == it always works ;). JC
On the contrary I suspect some of the packages that pass in your tests are obsolete code with disabled unit tests so you're not seeing all the problems.
Can you back it up it? Or is it just your assumption? Actually most of the failures approx 70% is > occurring in tests(see up mentioned link).
Thus, counting the tests which had not been written yet in the obsolete versions currently packaged in rawhide, plus all the tests which had never been activated there, it is not surprising current code with unit tests on sees more failures
And getting rawhide to the point one can actually build current Go apps on it requires replacing all this legacy code with current versions
Regards
-- Nicolas Mailhot _______________________________________________ golang mailing list -- golang@lists.fedoraproject.org To unsubscribe send an email to golang-leave@lists.fedoraproject.org
----- Mail original ----- De: "Jakub Cajka"
If you don't share how, why it fails we can be just speculating(maybe it is bug in your > packaging, build setup). I wouldn't assume google in name == it always works ;).
I sent you the logs by PM ("only" 5 MiB compressed with xz -9). Did you not receive them ?
Regards,
----- Original Message -----
From: "nicolas mailhot" nicolas.mailhot@laposte.net To: golang@lists.fedoraproject.org Sent: Monday, January 15, 2018 6:16:07 PM Subject: Re: F28 System Wide Change: Golang 1.10a
----- Mail original ----- De: "Jakub Cajka"
If you don't share how, why it fails we can be just speculating(maybe it is bug in your > packaging, build setup). I wouldn't assume google in name == it always works ;).
I sent you the logs by PM ("only" 5 MiB compressed with xz -9). Did you not receive them ?
Yes I did, and I replied to you.
Thinking about it now as both logs are ~100M text blobs, could you give me an list of packages that have failed for you(ideally, but no necessarily with fragments of log where the failure is visible) and srpm link so I can reproduce/investigate the issue?(Although I'm not promising that I will get to it this or next week as I'm quiet busy at the moment.)
JC
Regards,
-- Nicolas Mailhot _______________________________________________ golang mailing list -- golang@lists.fedoraproject.org To unsubscribe send an email to golang-leave@lists.fedoraproject.org
----- Mail original ----- De: "Jakub Cajka" jcajka@redhat.com À: golang@lists.fedoraproject.org Envoyé: Mardi 16 Janvier 2018 10:16:19 Objet: Re: F28 System Wide Change: Golang 1.10a
De: "Jakub Cajka"
From: "nicolas mailhot"
De: "Jakub Cajka"
If you don't share how, why it fails we can be just speculating(maybe it is bug in your > packaging, build setup). I wouldn't assume google in name == it always works ;).
I sent you the logs by PM ("only" 5 MiB compressed with xz -9). Did you not receive them ?
Yes I did, and I replied to you.
I see your message now, I had missed it, sorry :(
Thinking about it now as both logs are ~100M text blobs, could you give me an list of packages that have failed for you(ideally, but no necessarily with fragments of log where the failure is visible)
1. You have a summary of the successes and failures at the end of the log files 2. each failed build run of a package ends with 'Done processing $spec (failure)'
So it's pretty easy to check what package failed and then locate the part of the log dealing with the last build attempt of this package.
Then check if it's because of a missing dep or a go 1.10 specific problem (of course the missing dep may be because of the dep failed due to go 1.10). If it's not obvious the go 1.9 logs provide the logs of the same builds with go 1.9
and srpm link so I can reproduce/investigate the issue?
I'll try to revive my old apache instance to give you access to the files. Ideally I should redo a full copr run but that takes days to run even only on devel x86_64 (and I haven't checked how to force go 1.9 in copr to provide a comparison point). Local full build 'only' takes 8 to 10 hours. On go 1.10 it may be faster so many things fail :(
I'd love to work on go 1.10 but right now my priority is to finish the bootstraping of the baseline on go 1.9 so there's no package in the set waiting on a dep or a dep update to build in full mode. The big remaining bits are docker and maybe kubernetes (some deps want up to date kubernetes libs, not sure yet if it requires updating kubernetes itself to latest stable release). Though it is getting easier so many bits have been updated there are fewer weird failures due to an obsolete part when creating a new package.
Regards,
----- Original Message -----
From: "nicolas mailhot" nicolas.mailhot@laposte.net To: golang@lists.fedoraproject.org Sent: Tuesday, January 16, 2018 11:16:44 AM Subject: Re: F28 System Wide Change: Golang 1.10a
----- Mail original ----- De: "Jakub Cajka" jcajka@redhat.com À: golang@lists.fedoraproject.org Envoyé: Mardi 16 Janvier 2018 10:16:19 Objet: Re: F28 System Wide Change: Golang 1.10a
De: "Jakub Cajka"
From: "nicolas mailhot"
De: "Jakub Cajka"
If you don't share how, why it fails we can be just speculating(maybe it is bug in your > packaging, build setup). I wouldn't assume google in name == it always works ;).
I sent you the logs by PM ("only" 5 MiB compressed with xz -9). Did you not receive them ?
Yes I did, and I replied to you.
I see your message now, I had missed it, sorry :(
Thinking about it now as both logs are ~100M text blobs, could you give me an list of packages that have failed for you(ideally, but no necessarily with fragments of log where the failure is visible)
- You have a summary of the successes and failures at the end of the log
files 2. each failed build run of a package ends with 'Done processing $spec (failure)'
Ah... missed that, I have been just quickly scanning the log's first few 100 lines. What tool are you using for the build? mockchain?
So it's pretty easy to check what package failed and then locate the part of the log dealing with the last build attempt of this package.
Easy but extremely time consuming. I have nicely structured output from mockchain(so no need to writer parser for log file(s)), still it is extremely time consuming, especially when verifying the issues.
Then check if it's because of a missing dep or a go 1.10 specific problem (of course the missing dep may be because of the dep failed due to go 1.10). If it's not obvious the go 1.9 logs provide the logs of the same builds with go 1.9
and srpm link so I can reproduce/investigate the issue?
I'll try to revive my old apache instance to give you access to the files. Ideally I should redo a full copr run but that takes days to run even only on devel x86_64 (and I haven't checked how to force go 1.9 in copr to provide a comparison point). Local full build 'only' takes 8 to 10 hours. On go 1.10 it may be faster so many things fail :(
I'd love to work on go 1.10 but right now my priority is to finish the bootstraping of the baseline on go 1.9 so there's no package in the set waiting on a dep or a dep update to build in full mode. The big remaining bits are docker and maybe kubernetes (some deps want up to date kubernetes libs, not sure yet if it requires updating kubernetes itself to latest stable release). Though it is getting easier so many bits have been updated there are fewer weird failures due to an obsolete part when creating a new package.
Regards,
-- Nicolas Mailhot _______________________________________________ golang mailing list -- golang@lists.fedoraproject.org To unsubscribe send an email to golang-leave@lists.fedoraproject.org
One more thing popped on my mind, I can't really investigate the issues if you have out of the Fedora dependencies(maybe you could fork the packages in pagure??). If you will give me failures that you want to be investigated(with log and srpm), I will be happy to take a look(that do look like golang bugs, no broken deps, code issues, etc.).
JC
De: "Jakub Cajka"
- You have a summary of the successes and failures at the end of the log
files 2. each failed build run of a package ends with 'Done processing $spec (failure)'
Ah... missed that, I have been just quickly scanning the log's first few 100 lines. What tool are you using for the build? mockchain?
It's a quick and dirty script that is a bit smarter than mockchain (knows about bootstraping and release bumping), but with pretty raw logging facilities. I can attach it if you want.
I will look at mockchain but given its man page it is far from evident it is possible to wrap around it in a bootstraping context (and Go packages absolutely need this given all the cycles between them)
So it's pretty easy to check what package failed and then locate the part of the log dealing with the last build attempt of this package.
Easy but extremely time consuming. I have nicely structured output from mockchain(so no need to writer parser for log file(s)), still it is extremely time consuming, especially when verifying the issues.
I understand, at this point my Go toolchain makes creating and updating Go packages trivial. The time consuming phase is build log analysis (especially to sort what unit tests are relevant and what unit tests can not work in a mock env).
Regards,
golang@lists.fedoraproject.org
-
Jakub Cajka -
nicolas.mailhot＠laposte.net