Hey all,
It seems that upstream glibc disabled support for generating DT_HASH tables for its libraries and binaries, which breaks Linux games that use Epic Games' Easy Anti-Cheat (EAC).
There's a pretty decent write-up about this on LWN: https://lwn.net/Articles/904892/
Can we turn this back on for Fedora glibc until we can get Epic to make fixes for this and roll them out? I'd really like my Linux games to not break on upgrading to Fedora 37...
I'm inclined to vote -1, if everyone keeps DT_HASH around I doubt Epic will fix the issue.
Neal Gompa wrote:
There's a pretty decent write-up about this on LWN: https://lwn.net/Articles/904892/
Here's a link that actually works: https://lwn.net/SubscriberLink/904892/dba951441b61cbdc/ (Putting the title and "SubscriberLink", both between quotes, into a search engine does wonders.)
Kevin Kofler
Kevin Kofler via devel wrote:
Neal Gompa wrote:
There's a pretty decent write-up about this on LWN: https://lwn.net/Articles/904892/
Here's a link that actually works: https://lwn.net/SubscriberLink/904892/dba951441b61cbdc/ (Putting the title and "SubscriberLink", both between quotes, into a search engine does wonders.)
PS: I find it incredibly rude to share a paywalled link on a public mailing list, all the more in cases like this where there is a legal way to share the content without excluding large parts of the community.
Kevin Kofler
On Mon, Aug 22, 2022 at 4:21 AM Kevin Kofler via devel devel@lists.fedoraproject.org wrote:
PS: I find it incredibly rude to share a paywalled link on a public mailing list,
I 100% agree with this (although, to be slightly fair, this does regularly happen on many of the lists I read, as people just presume (falsely) that others subscribe to the same pay-walled sites the submitter's do, and until it is pointed out the submitter had not thought it through).
all the more in cases like this where there is a legal way to share the content without excluding large parts of the community.
I appreciate that others are willing to share to the community, but I also do understand that should subscribers continuously share to a large enough community the subscription model will eventually fail apart (and will result in another "and this is why we can't have nice things" collapse).
On 20/08/2022 21:42, Neal Gompa wrote:
It seems that upstream glibc disabled support for generating DT_HASH tables for its libraries and binaries, which breaks Linux games that use Epic Games' Easy Anti-Cheat (EAC).
DT_HASH was deprecated for 15+ years. We shouldn't take care of proprietary DRMs.
Can we turn this back on for Fedora glibc until we can get Epic to make fixes for this and roll them out?
-1 from me.
It's Epic's fault. They must update their anti-cheat to use the modern API.
On 8/21/22 10:59, Vitaly Zaitsev via devel wrote:
On 20/08/2022 21:42, Neal Gompa wrote:
It seems that upstream glibc disabled support for generating DT_HASH tables for its libraries and binaries, which breaks Linux games that use Epic Games' Easy Anti-Cheat (EAC).
DT_HASH was deprecated for 15+ years. We shouldn't take care of proprietary DRMs.
Can you cite a source for this?
Things like the gabi still list DT_HASH is mandatory[1] (though if it truly is is debatable).
Its also worth mentioning that DT_GNU_HASH is not a drop in replacement for DT_HASH and had no standardization or specification to speak of so its hard to follow.
Can we turn this back on for Fedora glibc until we can get Epic to make fixes for this and roll them out?
-1 from me.
It's Epic's fault. They must update their anti-cheat to use the modern API.
More reports have come out claiming this also affects the game Shovel Knight[2] and the open source library libstrangle[3], there is the non 0 chance that there are more programs out there in the wild that this will break.
It feels irresponsible of the glibc maintainers to suddenly respect the toolchains desired hash type when they haven't for years and then do it with little to no announcement resulting in broken software
[1] https://refspecs.linuxfoundation.org/elf/gabi4+/ch5.dynamic.html
[2] https://github.com/ValveSoftware/Proton/issues/6051#issuecomment-1212748397
On Sun, Aug 21, 2022 at 12:05:11PM +0200, Jan Drögehoff wrote:
It's Epic's fault. They must update their anti-cheat to use the modern API.
More reports have come out claiming this also affects the game Shovel Knight[2] and the open source library libstrangle[3], there is the non 0 chance that there are more programs out there in the wild that this will break.
It feels irresponsible of the glibc maintainers to suddenly respect the toolchains desired hash type when they haven't for years and then do it with little to no announcement resulting in broken software
To be precise, everything in Fedora except glibc is only built with DT_GNU_HASH and no DT_HASH since July 2006, glibc has been an exception that has been built with both because of statically linked programs from 16+ years ago that wouldn't support it. If all they want is be able to interpose dlsym, they could just use dlvsym to look up the original sym, instead of diving into the hash tables.
Jakub
On 8/21/22 12:44, Jakub Jelinek wrote:
On Sun, Aug 21, 2022 at 12:05:11PM +0200, Jan Drögehoff wrote:
It's Epic's fault. They must update their anti-cheat to use the modern API.
More reports have come out claiming this also affects the game Shovel Knight[2] and the open source library libstrangle[3], there is the non 0 chance that there are more programs out there in the wild that this will break.
It feels irresponsible of the glibc maintainers to suddenly respect the toolchains desired hash type when they haven't for years and then do it with little to no announcement resulting in broken software
To be precise, everything in Fedora except glibc is only built with DT_GNU_HASH and no DT_HASH since July 2006, glibc has been an exception that has been built with both because of statically linked programs from 16+ years ago that wouldn't support it.
I think its worth putting an emphasis on the fact that this was glibc intentionally ignoring the toolchain hash type and simply going with both and not something Fedora explicitly decided to do.
If all they want is be able to interpose dlsym, they could just use dlvsym to look up the original sym, instead of diving into the hash tables.
I do not think the change on glibcs part is bad, the hack was terrible to begin with but removing it broke the ABI and the lack of any announcement of it beforehand is now causing problems that distro maintainers and software developers have to deal with
On Sun, Aug 21, 2022 at 9:19 AM Jan Drögehoff sentrycraft123@gmail.com wrote:
On 8/21/22 12:44, Jakub Jelinek wrote:
On Sun, Aug 21, 2022 at 12:05:11PM +0200, Jan Drögehoff wrote:
It's Epic's fault. They must update their anti-cheat to use the modern API.
More reports have come out claiming this also affects the game Shovel Knight[2] and the open source library libstrangle[3], there is the non 0 chance that there are more programs out there in the wild that this will break.
It feels irresponsible of the glibc maintainers to suddenly respect the toolchains desired hash type when they haven't for years and then do it with little to no announcement resulting in broken software
To be precise, everything in Fedora except glibc is only built with DT_GNU_HASH and no DT_HASH since July 2006, glibc has been an exception that has been built with both because of statically linked programs from 16+ years ago that wouldn't support it.
I think its worth putting an emphasis on the fact that this was glibc intentionally ignoring the toolchain hash type and simply going with both and not something Fedora explicitly decided to do.
If all they want is be able to interpose dlsym, they could just use dlvsym to look up the original sym, instead of diving into the hash tables.
I do not think the change on glibcs part is bad, the hack was terrible to begin with but removing it broke the ABI and the lack of any announcement of it beforehand is now causing problems that distro maintainers and software developers have to deal with
Right. To make matters worse, *nobody* outside of the glibc developers knew DT_HASH was deprecated for the GNU ELF ABI. And DT_GNU_HASH isn't specified anywhere that I found. Additionally, given that DT_HASH is currently mandatory in the SysV ABI (which is what Linux uses), it's clear there's a documentation problem.
I don't disagree that glibc should respect what distros set, I'm just asking us to build glibc with a DT_HASH table added until a formal deprecation/retirement cycle is done with all stakeholders aware of the change and everyone aware of the consequences of such a breakage.
it's clear there's a documentation problem [with DT_GNU_HASH]
Partly due to lack of documentation, already I have seen "abuses" of the DT_GNU_HASH format. In particular, some versions of Rust and/or musl run-times use (0 == nbucket) to mean something like "there is no hash table information" but do not specify all the implications for the run-time symbol table. Other versions use (1 == nbucket && 0 == buckets[0] && 1 == n_bitmask && 0 == bitmask[0]) for a similar purpose. Similarly, Rust and Android can trim "unused" zero entries from the end of &hash_array[nbucket], even though buckets[] and hash_array[] should be parallel.
On 8/21/22 12:51, John Reiser wrote:
it's clear there's a documentation problem [with DT_GNU_HASH]
Partly due to lack of documentation, already I have seen "abuses" of the DT_GNU_HASH format. In particular, some versions of Rust and/or musl run-times use (0 == nbucket) to mean something like "there is no hash table information" but do not specify all the implications for the run-time symbol table. Other versions use (1 == nbucket && 0 == buckets[0] && 1 == n_bitmask && 0 == bitmask[0]) for a similar purpose. Similarly, Rust and Android can trim "unused" zero entries from the end of &hash_array[nbucket], even though buckets[] and hash_array[] should be parallel.
Please report a bug at https://github.com/rust-lang/rust/issues.
On 8/21/22 10:14, Demi Marie Obenour wrote:
On 8/21/22 12:51, John Reiser wrote:
it's clear there's a documentation problem [with DT_GNU_HASH]
Partly due to lack of documentation, already I have seen "abuses" of the DT_GNU_HASH format. In particular, some versions of Rust and/or musl run-times use (0 == nbucket) to mean something like "there is no hash table information" but do not specify all the implications for the run-time symbol table. Other versions use (1 == nbucket && 0 == buckets[0] && 1 == n_bitmask && 0 == bitmask[0]) for a similar purpose. Similarly, Rust and Android can trim "unused" zero entries from the end of &hash_array[nbucket], even though buckets[] and hash_array[] should be parallel.
Please report a bug at https://github.com/rust-lang/rust/issues.
https://github.com/rust-lang/rust/issues/100859 "abuse of DT_GNU_HASH descriptor"
On Sun, Aug 21, 2022 at 09:51:42AM -0700, John Reiser wrote:
it's clear there's a documentation problem [with DT_GNU_HASH]
Partly due to lack of documentation, already I have seen "abuses"
So what is https://akkadia.org/drepper/dsohowto.pdf https://sourceware.org/legacy-ml/binutils/2006-10/msg00377.html https://flapenguin.me/elf-dt-gnu-hash then?
Jakub
On Mon, Aug 22, 2022 at 10:01 AM Jakub Jelinek jakub@redhat.com wrote:
On Sun, Aug 21, 2022 at 09:51:42AM -0700, John Reiser wrote:
it's clear there's a documentation problem [with DT_GNU_HASH]
Partly due to lack of documentation, already I have seen "abuses"
So what is https://akkadia.org/drepper/dsohowto.pdf https://sourceware.org/legacy-ml/binutils/2006-10/msg00377.html https://flapenguin.me/elf-dt-gnu-hash then?
I'm sorry, which point were you trying to prove with these links?
-> https://akkadia.org/drepper/dsohowto.pdf
This is a ~50 page document, of which one page refers to DT_GNU_HASH, and only contains a short description of how to use it.
- https://sourceware.org/legacy-ml/binutils/2006-10/msg00377.html
This is a mailing list post by you, and you wrote:
"""but for the final specification I'm afraid you need to read binutils and/or libc sources."""
- https://flapenguin.me/elf-dt-gnu-hash
This looks like a reverse-engineering effort of the binary DT_GNU_HASH format based on the source code - it starts with:
"""The problem with it is that DT_GNU_HASH is not documented anywhere other than in GNU binutils and glibc source code."""
Fabio
Hi all,
On Sunday, 21 August 2022 17:29:56 CEST Neal Gompa wrote:
I don't disagree that glibc should respect what distros set, I'm just asking us to build glibc with a DT_HASH table added until a formal deprecation/retirement cycle is done with all stakeholders aware of the change and everyone aware of the consequences of such a breakage.
+1. Deprecations of this sort should follow a proper process with sufficient formal advance notice.
Best, Christoph
On Mon, 2022-08-22 at 08:43 +0200, Christoph Erhardt wrote:
Hi all,
On Sunday, 21 August 2022 17:29:56 CEST Neal Gompa wrote:
I don't disagree that glibc should respect what distros set, I'm just asking us to build glibc with a DT_HASH table added until a formal deprecation/retirement cycle is done with all stakeholders aware of the change and everyone aware of the consequences of such a breakage.
+1. Deprecations of this sort should follow a proper process with sufficient formal advance notice.
+1. Such breakage will just harm the reputation Linux (IMO).
Best, Christoph _______________________________________________ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
On Mon, Aug 22, 2022 at 10:31:03AM +0100, Sérgio Basto wrote:
On Mon, 2022-08-22 at 08:43 +0200, Christoph Erhardt wrote:
Hi all,
On Sunday, 21 August 2022 17:29:56 CEST Neal Gompa wrote:
I don't disagree that glibc should respect what distros set, I'm just asking us to build glibc with a DT_HASH table added until a formal deprecation/retirement cycle is done with all stakeholders aware of the change and everyone aware of the consequences of such a breakage.
+1. Deprecations of this sort should follow a proper process with sufficient formal advance notice.
+1. Such breakage will just harm the reputation Linux (IMO).
A bit late to the game (I saw the email from Florian that an update is being worked on), but +1 from me too.
Zbyszek
On Tue, Aug 23, 2022 at 4:13 AM Florian Weimer fweimer@redhat.com wrote:
We are working on this and hope to have an update soon.
Well, we just released the Beta and people have noticed that this is still broken. Do we have an ETA on a fix? Because this is going to be a major black eye for *us* if it stays broken through to GA.
On 14/09/2022 05:27, Neal Gompa wrote:
Well, we just released the Beta and people have noticed that this is still broken. Do we have an ETA on a fix? Because this is going to be a major black eye for*us* if it stays broken through to GA.
Epic Games had more than a month to fix the problem but they did nothing.
* Vitaly Zaitsev via devel:
On 14/09/2022 05:27, Neal Gompa wrote:
Well, we just released the Beta and people have noticed that this is still broken. Do we have an ETA on a fix? Because this is going to be a major black eye for*us* if it stays broken through to GA.
Epic Games had more than a month to fix the problem but they did nothing.
Why do you say that? These multi-ISV coordination issues can be difficult to resolve.
Thanks, Florian
devel@lists.stg.fedoraproject.org
-
Christoph Erhardt -
Demi Marie Obenour -
Fabio Valentini -
Florian Weimer -
Gary Buhrmaster -
Jakub Jelinek -
Jan Drögehoff -
John Reiser -
Kevin Kofler -
Leigh Scott -
Neal Gompa -
Sérgio Basto -
Vitaly Zaitsev -
Zbigniew Jędrzejewski-Szmek