devel December 2021

devel@lists.stg.fedoraproject.org

191 participants
293 discussions

Mock v2.16 release, mock-core-configs v36.4
by Pavel Raiskup 11 Feb '22

11 Feb '22

Hello! I'm glad I can announce that we have a new release of Mock. See the full release notes [1]. The major change that happened is the removal of 'epel-8' config files, as a follow-up for [2] discussion (and of course on *devel lists, big thanks to everyone for the discussion). Note that this is is the last v2 release being shipped to all supported Fedora/EPEL versions. From now on, we'll move to v3 with development (in 'main' branch) and EPEL 7 stays on v2 (in 'mock-2' branch, bugfix only). [1] https://rpm-software-management.github.io/mock/Release-Notes-2.16 [2] https://pagure.io/epel/issue/133 [Fedora 35]: https://bodhi.fedoraproject.org/updates/FEDORA-2021-a7d4aaa6fe [Fedora 34]: https://bodhi.fedoraproject.org/updates/FEDORA-2021-0947974f0a [EPEL 8]: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2021-2d0f959e00 [EPEL 7]: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2021-82ccb8f2b7 Happy building! Pavel

9 27

libcurl-minimal
by Zbigniew Jędrzejewski-Szmek 10 Feb '22

10 Feb '22

Hi Kamil and everyone, what is the plan with introduction of libcurl-minimal in Fedora? IIUC, libcurl and libcurl-minimal both have the same Provides, so libcurl-minimal can be used to satisfy automatically generated dependencies: $ dnf repoquery --provides libcurl-minimal libcurl = 7.78.0-3.fc35 libcurl(x86-32) = 7.78.0-3.fc35 libcurl(x86-64) = 7.78.0-3.fc35 libcurl-minimal = 7.78.0-3.fc35 libcurl-minimal(x86-32) = 7.78.0-3.fc35 libcurl-minimal(x86-64) = 7.78.0-3.fc35 libcurl.so.4 libcurl.so.4()(64bit) $ dnf repoquery --provides libcurl libcurl = 7.78.0-3.fc35 libcurl(x86-32) = 7.78.0-3.fc35 libcurl(x86-64) = 7.78.0-3.fc35 libcurl-full = 7.78.0-3.fc35 libcurl-full(x86-32) = 7.78.0-3.fc35 libcurl-full(x86-64) = 7.78.0-3.fc35 libcurl.so.4 libcurl.so.4()(64bit) AFAICS, no other package makes use of libcurl-{full,minimal}. In systemd we only care about a narrow subset of protocols, so libcurl-minimal is perfect. I considered adding Suggests:libcurl-minimal%{_isa} in systemd. IIUC, that'd bias dnf towards the installation of libcurl-minimal. But the problem is that if some other package expects libcurl in the full version, it'll be disappointed. Hence my question: how to proceed with pulling in libcurl-minimal where it'd be useful? Should I just add Suggests:libcurl-minimal%{_isa} in systemd and let the maintainers of other packages add Recommends:libcurl-minimal%{_isa} or Requires:libcurl-minimal%{_isa} if they need it? What packages would that be? Another option would be do not do any of this at package level, but instead pull in libcurl-minimal through comps or kickstart or equivalent when doing installations. (Sorry if this is all documented somewhere… I looked around, but didn't see anything relevant.) Zbyszek

11 23

F37 Change: RetireARMv7 (System-Wide Change proposal)
by Ben Cotton 07 Feb '22

07 Feb '22

https://fedoraproject.org/wiki/Changes/RetireARMv7 == Owner == * Name: [[User:pbrobinson| Peter Robinson]] * Email: <pbrobinson(a)fedoraproject.org> == Detailed Description == The ARMv7 arm architecture was the second variant of the arm architecture that Fedora has supported, the first was ARMv5, the third is aarch64. The proposal is to retire ARMv7 as part of the Fedora 37 release. This will allow ARMv7/armhfp to be supported until the Fedora 36 end of life in around June 2023. Overall arm32 is generally waning with generally few new ARMv7 devices added to Fedora in recent releases. To add to that a number of newer Fedora features designed to improve speed and security of the Fedora release are causing 32 bit architectures in general primarily due to the process memory limit when linking large applications. The ARMv7/armhfp is the last fully supported 32 bit architecture, we still currently build i686 packages, but it's not shipped as artefacts. == Benefit to Fedora == The primary benefit is to maintainers of the ARM architecture, the various toolchain teams and package maintainers in general. == Scope == * Proposal owners: Work with rel-eng to disable the architecture in koji, remove all the various pungi pieces and clean up any other release detritus. * Other developers: No action required. * Release engineering: [https://pagure.io/releng/issue/10387 Releng issue #10387] Disable a bunch of stuff, it's really just one koji admin command and a PR for pungi config changes * Policies and guidelines: No initial updates to policies and guidelines as ARMv7 won't completely disappear until F-36 EOL. * Trademark approval: N/A (not needed for this Change) == Upgrade/compatibility impact == Any current users of Fedora on ARMv7 devices won't be able to upgrade to Fedora 37, they will have to stay on Fedora 36 until it's EOL. == How To Test == There's not really anything to test as it's removing the support for an architecture. == User Experience == Any current users of Fedora on ARMv7 devices won't be able to upgrade to Fedora 37, they will have to stay on Fedora 36 until it's EOL. == Dependencies == N/A. == Contingency Plan == Continue on as before with the added advantage of the people that protested the removal of the architecture will be actively contributing to the maintenance of the architecture * Contingency mechanism: Leave enabled. We basically won't get to this if FESCo doesn't approve the change. * Contingency deadline: Mass rebuild. * Blocks release? Yes * Blocks product? Yes == Documentation == N/A == Release Notes == Fedora Linux 37 with the ARMv7 architecture is retired into the sunset. There will definitely be celebrations, there will likely be some that shed some tears! Overall for the maintainers it will likely be seen as a net win, for the few, generally shrinking, users it's probably a net loss but they can probably just go and buy a Raspberry Pi Zero 2W for US$15. ¯\_(ツ)_/¯ -- Ben Cotton He / Him / His Fedora Program Manager Red Hat TZ=America/Indiana/Indianapolis

23 30

F36 Change: Enable exclude_from_weak_autodetect by default in LIBDNF (System-Wide Change proposal)
by Ben Cotton 07 Feb '22

07 Feb '22

https://fedoraproject.org/wiki/Changes/ExcludeFromWeakAutodetect == Summary == exclude_from_weak_autodetect enables autodetection of unmet weak dependencies (Recommends or Supplements) of installed packages and blocks installation of packages satisfying already unmet dependencies. In other words: When you don't have the recommended package installed, it won't be automatically installed with future upgrades of the recommending package. == Owner == * Name: [[User:jmracek| Jaroslav Mracek]] * Email: jmracek(a)redhat.com == Detailed Description == The feature is designed to prevent an install of removed weak dependencies from the system by users and to not install weak dependencies missing after system deployment. It will change the behavior of DNF, microdnf, and PackageKit. The feature will be backported to all Fedoras, but in default, the feature will be off. Additional information: https://bugzilla.redhat.com/show_bug.cgi?id=1699672 The default value for exclude_from_weak_autodetect configuration can be overridden in `/etc/dnf/dnf.conf` == Feedback == The feature was requested by [[User:Churchyard|Miro Hrončok]] and supported by many others: See [https://bugzilla.redhat.com/show_bug.cgi?id=1699672 rhbz#1699672] for more feedback. == Benefit to Fedora == After the installation of a fresh system, the first upgrade will not install a lot of weak dependencies. Some of them were excluded from the kick-start installation set for good reasons (security, image size, minimal functional set, ...), but after the first update, all weak dependencies are installed, therefore some features of deployment simply disappear. == Scope == * Proposal owners: ** The feature is ready in Pull Request - https://github.com/rpm-software-management/libdnf/pull/1279 ** PRs only wait for a release of libsolv ** The Feature will be enabled in upstream as default, therefore from Fedora 36, we start to release libdnf without a revert patch of default in comparison to upstream. * Other developers: The change requires a new release of libsolv. * Release engineering: * Policies and guidelines: A packaging guideline should be added that discourages or forbids weak dependencies on fully versioned (sub)packages (see [https://bugzilla.redhat.com/show_bug.cgi?id=1699672#c44 the details]). * Trademark approval: N/A (not needed for this Change) * Alignment with Objectives: == Upgrade/compatibility impact == No manual changes will be required. After the libdnf update, this feature will be on by default. == How To Test == 1. Install package without satisfied weak dependencies 2. Upgrade the upgrade. With exclude_from_weak_autodetect=true, it will not install weak dependencies of already installed packages. With exclude_from_weak_autodetect=false, weak dependencies will be installed during upgrades. == User Experience == The change in default will help to keep some values for particular deployments (a minimal system will be still minimal without disabling weak dependencies). Users will be able to remove particular weak dependencies and they will be not installed on the first upgrade. In case when the feature will not work according to the user expectation it can be switched off in the dnf configuration file. == Dependencies == libsolv - Required code changes are already in the libsolv upstream. We only wait for the next libsolv release. == Contingency Plan == There are no external dependencies, therefore we can easily postpone the feature and the change of default behavior. * Contingency mechanism: (What to do? Who will do it?) * Contingency deadline: beta freeze * Blocks release? No == Documentation == The feature will be documented in dnf man pages. -- Ben Cotton He / Him / His Fedora Program Manager Red Hat TZ=America/Indiana/Indianapolis

13 35

what is wrong with this conditional scriplet on rpm.spec
by Sérgio Basto 02 Feb '22

02 Feb '22

Hi, I just notice build in mock or koji this scriptlet [1] on a build for Fedora gives me "is rhel 8 or 9" , what I'm missing ? Thank you [1] %if ! 0%{?rhel} >= 8 echo "is not rhel >= 8" %else echo "is rhel 8 or 9" %endif -- Sérgio M. B.

2 3

Announcing LLVM Snapshot Packages for Fedora Linux
by Konrad Kleine 01 Feb '22

01 Feb '22

Dear Fedora packagers, developers and users, we have some good news for you: We are beginning to build nightly snapshot packages of LLVM for the latest versions of Fedora Linux (currently 34, 35 and rawhide) for a growing list of architectures. You can grab them here: https://copr.fedorainfracloud.org/coprs/g/fedora-llvm-team/llvm-snapshots/ Feel free to enable the copr repository with $ dnf copr enable @fedora-llvm-team/llvm-snapshots and then install the i.e. latest clang with $ dnf install clang Beware, that a snapshot release of LLVM is probably more unstable than a regular release! If you run into a problem, I would kindly ask you to wait and try it again with the next snapshot. We hope you enjoy this peek into the next version of LLVM that you can now try without too much hassle and without compiling it every day on your own. Regards, Konrad Kleine Senior Software Engineer, Platform Tools Red Hat <https://www.redhat.com> kkleine(a)redhat.com M: +49(0)151/21000244 D87A 77F4 2A58 C72D 12A7 203B C0A0 2C32 BCB7 3099 <https://www.redhat.com>

7 17

F36 Change: Ruby 3.1 (System-Wide Change proposal)
by Ben Cotton 28 Jan '22

28 Jan '22

https://fedoraproject.org/wiki/Changes/Ruby_3.1 == Summary == Ruby 3.1 is the latest stable version of Ruby. Many new features and improvements are included for the increasingly diverse and expanding demands for Ruby. With this major update from Ruby 3.0 in Fedora 35 to Ruby 3.1 in Fedora 36, Fedora becomes the superior Ruby development platform. == Owner == * Name: [[User:vondruch| Vít Ondruch]] * Email: vondruch(a)redhat.com == Detailed Description == Ruby 3.1 is upstream's new major release of Ruby. Many new features and improvements are included. === YJIT: New experimental in-process JIT compiler === Ruby 3.1 merges YJIT, a new in-process JIT compiler developed by Shopify. Since Ruby 2.6 introduced MJIT in 2018, its performance greatly improved, and finally we achieved Ruby3x3 last year. But even though Optcarrot has shown impressive speedups, the JIT hasn’t benefited real world business applications. Recently Shopify contributed many Ruby improvements to speed up their Rails application. YJIT is an important contribution, and aims to improve the performance of Rails applications. Though MJIT is a method-based JIT compiler and uses an external C compiler, YJIT uses Basic Block Versioning and includes JIT compiler inside it. With Lazy Basic Block Versioning (LBBV) it first compiles the beginning of a method, and incrementally compiles the rest when the type of arguments and variables are dynamically determined. See YJIT: a basic block versioning JIT compiler for CRuby for a detailed introduction. With this technology, YJIT achieves both fast warmup time and performance improvements on most real-world software, up to 22% on railsbench, 39% on liquid-render. YJIT is still an experimental feature, and as such, it is disabled by default. If you want to use this, specify the --yjit command-line option to enable YJIT. It is also limited to macOS & Linux on x86-64 platforms for now. https://bugs.ruby-lang.org/issues/18229 https://shopify.engineering/yjit-just-in-time-compiler-cruby https://www.youtube.com/watch?v=PBVLf3yfMs8 === debug gem: A new debugger === A new debugger debug.gem is bundled. debug.gem is fast debugger implementation and it provides many features like remote debugging, colorful REPL, IDE (VSCode) integration and more. It replaces lib/debug.rb standard library. === error_highlight: Fine-grained error location in backtrace === A built-in gem, error_highlight, has been introduced. It includes fine-grained error location in backtrace: $ ruby test.rb test.rb:1:in `<main>': undefined method `time' for 1:Integer (NoMethodError) 1.time {} ^^^^^ Did you mean? times This gem is enabled by default. You can disable it by using a command-line option --disable-error_highlight. See the repository in detail. === Irb improvement === === Other Notable New Features === * Language ** Values in Hash literals and keyword arguments can be omitted. ** Pin operator in pattern matching now takes an expression. * RBS ** `rbs collection` has been introduced to manage gems’ RBSs. ** Many signatures for built-in and standard libraries have been added/updated. ** It includes many bug fixes and performance improvements too. * TypeProf ** Experimental IDE support has been implemented. ** Many bug fixes and performance improvements. === Performance improvements === * MJIT ** For workloads like Rails, the default --jit-max-cache is changed from 100 to 10000. The JIT compiler no longer skips compilation of methods longer than 1000 instructions. ** To support Zeitwerk of Rails, JIT-ed code is no longer cancelled when a TracePoint for class events is enabled. === Other notable changes since 3.0 === * One-line pattern matching, e.g., ary => [x, y, z], is no longer experimental. * Multiple assignment evaluation order has been changed slightly. ** foo[0], bar[0] = baz, qux was evaluated in order baz, qux, foo, and then bar in Ruby 3.0. In Ruby 3.1, it is evaluated in order foo, bar, baz, and then qux. * Variable Width Allocation: Strings (experimental) * Standard libraries updates == Benefit to Fedora == With a latest release, Ruby language is supporting the newest language features, which enables even faster and easier development of Ruby applications. == Scope == * Proposal owners: ** Finish packaging of Ruby 3.1. Current changes available in PR https://src.fedoraproject.org/rpms/ruby/pull-request/106 ** Rebuilding of Ruby packages providing native extensions (i.e. packages which depends on libruby). * Other developers: ** Rebuild of packages with binary extensions (i.e. packages which depends on libruby) will be handled automatically, but some packages might need fixes/updates to support Ruby 3.1 properly. * Release engineering: [https://pagure.io/releng/issue/10478 #10478] ** The packages are going to be rebuild in side-tag, but that does not need releng involvement nowadays. * Policies and guidelines: N/A (not needed for this Change) * Trademark approval: N/A (not needed for this Change) * Alignment with Objectives: == Upgrade/compatibility impact == * User specific Ruby binary extensions need to be rebuild. * Ruby packages/application dependencies might need to be adjusted if net-* and other newly bundled gems are used. == How To Test == * No special hardware is needed. * To test, install Ruby 3.1. The test builds are pusblished in PR or on Ruby-SIG ML * Try to locally rebuild your packages using Ruby 3.1. * Use the packages with your applications previously written in Ruby. * If something doesn't work as it should, let us know. == User Experience == The Ruby programs/scripts should behave as they were used to. == Dependencies == <pre> $ dnf repoquery --disablerepo=* --enablerepo=rawhide --enablerepo=rawhide-source --arch=src --whatrequires 'ruby-devel' | sort | uniq | wc -l 130 </pre> == Contingency Plan == * Contingency mechanism: We would like to get a special buildroot tag to be able to rebuild necessary the packages with Ruby 3.1. If anything goes wrong, the tag could be easily dropped and previous version of Ruby 3.0 and its dependencies stays intact. The tag would be merged into F36 after everything is rebuild. * Contingency deadline: Mass Rebuild * Blocks release? No == Documentation == * [http://www.ruby-doc.org/ Help and documentation for the Ruby programming language] * [https://github.com/ruby/ruby/blob/v3_1_0_preview1/NEWS.md Ruby 3.1.0 NEWS] * [https://www.ruby-lang.org/en/news/2021/11/09/ruby-3-1-0-preview1-released/ Ruby 3.1 release announcement] == Release Notes == * The Ruby 3.1 bumps soname, therefore Ruby packages, which use binary extensions, should be rebuilt. Nevertheless, since upstream paid great attention to source compatibility, no changes to your code are needed. https://github.com/ruby/ruby/blob/v3_1_0_preview1/NEWS.md -- Ben Cotton He / Him / His Fedora Program Manager Red Hat TZ=America/Indiana/Indianapolis

5 4

F36 Change: Enable fs-verity in RPM (System-Wide Change proposal)
by Ben Cotton 26 Jan '22

26 Jan '22

https://fedoraproject.org/wiki/Changes/FsVerityRPM == Summary == Enable the use of fsverity for installed RPM files validation. == Owners == * Name: [[User:Dcavalca|Davide Cavalca]], [[User:Borisb|Boris Burkov]], [[User:Filbranden|Filipe Brandenburger]], [[User:Salimma|Michel Alexandre Salim]], [[User:Malmond|Matthew Almond]] * Email: dcavalca(a)fb.com, borisb(a)fb.com, filbranden(a)fb.com, michel(a)fb.com, malmond(a)fb.com == Detailed description == fs-verity is a [https://www.kernel.org/doc/html/latest/filesystems/fsverity.html Linux kernel feature] that does transparent on-demand integrity/authenticity verification of the contents of read-only files, using a hidden Merkle tree (hash tree) associated with the file. The mechanism is similar to dm-verity, but implemented at the file level rather than at the block device level. When fsverity is enabled for a file, the kernel reads every block and generates a hash tree for it, which is stored within the filesystem. On subsequent reads, the kernel computes the block hash and compares it with the one stored in the tree, protecting against alterations and corruption. Because this happens at the filesystem data block read layer, it encompasses all file operations (<code>open</code>, <code>mmap</code>,<code>exec</code>, etc.). In the context of rpm, there are two parts to this: * at build time, we compute the Merkle tree for the files within a package, then sign it and ship it as part of the rpm metadata; * at run time, if the fsverity rpm plugin is enabled, rpm will install the fsverity signature key and enable fsverity on files that are installed. This proposal is primarily concerned with the first part, which will make it possible for users to leverage fs-verity for RPM if they so desire. Specifically, installing and enabling the fs-verity rpm plugin by default is explicitly considered out of scope here. === Caveats === ==== Merkle tree cost ==== The Merkle tree used by fsverity needs to be generated (once at build time, once when the package is installed) and stored on disk. The generation process involves reading all blocks and computing the hash, which has a non-trivial cost; however, it does not appear to meaningfully slow down package installs during empirical testing. Once generated, the Merkle tree will use up some disk space for its storage (about 1/127th of the original file size). Note that the Merkle tree is ''not'' shipped with the RPM itself (only its signature is) and is only generated and stored at install time if the fsverity rpm plugin is enabled. Hence, there is no cost (neither in generation time nor in disk space usage) if the plugin is disabled. ==== Signature overhead cost ==== To leverage fsverity every rpm needs to include the hash signature as part of its metadata, which will increase its size. The signature size is roughly proportional to the number of files in the package. From empirical testing, in the vast majority of cases we expect to see minimal to no size increase thanks to RPM header packing. === Relationship with IMA === [https://sourceforge.net/p/linux-ima/wiki/Home/ IMA] is another technology meant to provide detection of file alterations. IMA and fsverity operate very differently, and are somewhat complementary. fs-verity works by using a Merkle tree to generate a checksum for every data block in the system, and reads will fail if a single data block read fails it’s checksum. The signature of the the file is validated against a public key loaded into the kernel keyring. Because fsverity operates on block reads, its runtime cost is small (as it only needs to verify the block that is being accessed), and it can protect from alterations at any point in time. IMA works by measuring a file as a whole and comparing its signature whenever it’s read of executed. It has a higher runtime cost than fsverity (as it needs to verify the whole file at once) and it cannot detect subsequent alternations. IMA provides a much more rich and complex policy system, allowing one to define system-wide policies around trusted files that tie into LSMs such as SELinux. IMA and fsverity could potentially be integrated (meaning, an fsverity backend for IMA could be implemented to leverage its policy controls), but this is not currently planned or being worked on. === Relationship with native checksums === By default, btrfs already checksums each file extent, which could potentially be leveraged to implement a HMAC solution. This currently exists as a [https://lore.kernel.org/linux-btrfs/SN4PR0401MB3598DDEF9BF9BACA71A1041D9BA7… patch series] but it hasn’t been merged yet. Similarly to IMA, we see this approach as complementary to fs-verity. The [https://developers.facebook.com/blog/post/2021/10/19/fs-verity-support-in-b… blog post] goes into more details of the tradeoffs involved. == Feedback == Pending the devel discussion == Benefit to Fedora == The main benefit is the ability to do block-level verification of RPM-installed files. In turn, this can be used to implement usecase-specific validation and verification policies depending on the environment requirements. == Scope == * Proposal owners ** btrfs kernel enablement work ([https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?… landed in 5.15]); see this [https://developers.facebook.com/blog/post/2021/10/19/fs-verity-support-in-b… blog post] for more details ** koji integration: koji will need to add the fs-verity metadata to packages when signing them * Other developers: ** deploy the koji integration changes to production * Release engineering: tbd * Policies and guidelines: N/A * Trademark approval: N/A == Upgrade/compatibility impact == None == How to test == Install the fs-verity RPM plugin to validate package contents: <pre>$ sudo dnf install rpm-plugin-fsverity</pre> Note that this will only be useful if the packages being installed contain the appropriate fs-verity metadata (which, for Fedora upstream packages, requires Koji integration that is part of this Change). However, you should still be able to test this if you locally sign a package with <code>rpmsign --addverity</code>. == User experience == This Change is fully transparent and there is no user impact by default. If the user chooses to enable the fs-verity RPM plugin, they can then leverage the additional verification features provided by fs-verity. == Dependencies == * fs-verity support is available in RPM as of 4.17, which is available as of Fedora 35 and is already enabled in rpm-4.17.0-0.rc1.0.fc36 * CONFIG_FS_VERITY in the kernel config; this is already enabled * fs-verity requires filesystem support; currently support for ext4 and f2fs is already available; support for btrfs landed in 5.15 * there is no filesystem dependency on the builders, only at runtime (and only if the rpm fsverity plugin is installed and one wishes to use it) == Contingency plan == Revert the changes to koji. == Documentation == * https://www.kernel.org/doc/html/latest/filesystems/fsverity.html * https://developers.facebook.com/blog/post/2021/10/19/fs-verity-support-in-b… * The proposal owners plan to document the fsverity plugin and integration in RPM (https://github.com/rpm-software-management/rpm/issues/1849) == Release Notes == The RPM package manager now supports validation of file contents using fs-verity. -- Ben Cotton He / Him / His Fedora Program Manager Red Hat TZ=America/Indiana/Indianapolis

31 71

F36 Change: Package information on ELF objects (System-Wide Change proposal)
by Ben Cotton 26 Jan '22

26 Jan '22

https://fedoraproject.org/wiki/Changes/Package_information_on_ELF_objects == Summary == All binaries (executables and shared libraries) are annotated with an ELF note that identifies the rpm for which this file was built. This allows binaries to be identified when they are distributed without any of the rpm metadata. `systemd-coredump` uses this to log package versions when reporting crashes. == Owner == * Name: [[User:Zbyszek|Zbigniew Jędrzejewski-Szmek]] * Email: zbyszek(a)in.waw.pl * Name: Lennart Poettering * Email: mzsrqben(a)0pointer.net == Detailed Description == People mix binaries (programs and libraries) from different distributions (for example using Fedora containers on Debian or vice versa), and distribute binaries without packaging metadata (for example by stripping everything except the binary from a container image, also removing `/usr/lib/.build-id/*`), compile their own rpm packages (for internal distribution and installation), and compile and distribute their own binaries. Sometimes we need to introspect a binary and figure out its provenance, for example when a program crashes and we are looking at a core dump, but also when we have a binary without the packaging metadata. When the need to introspect a binary arises, we have some very good mechanisms to show the provenance: when a file is installed through the package manager we can directly list the providing package, but even without this we can use build-ids embedded in the binary to uniquely identify the originating build. But those mechanisms work best when we're in the realm of a single distribution. In particular, build-ids can be easily tied to a source rpm, but only when we have the source rpm is part of the distribution and the build-id was registered in the appropriate database which maps build-ids to real package names. When we move outside of the realm of a single distribution, it can be hard to figure out where a given binary originates from. If we know that a binary is from a given distribution, we may be able to use some distro-specific mechanism to figure out this information. But those mechanisms will be different for different distributions and will often require network access. With this change we aim to provide a mechanism that is is very simple, provides a "human-readable" origin information without further processing, is portable across distros, and works without network access. The directly motivating use case is display of core dumps. Right now we have build-ids, but those are just opaque hexadecimal numbers that are not meaningful to users. We would like to immediately list versions of packages involved in the crash (including both the program and any libraries it links to). It is not enough to query the rpm database to do the equivalent of `rpm -qf …`: very often programs crash after some packages have been upgraded and the binaries loaded into memory are not the binaries that are currently present on disk, or when through some mishap, the binaries on disk do not match the installed rpms. A mechanism that works without rpm database lookup or network access allows this information to be showed immediately in `coredumpctl` listings and journal entries about the crash. This includes crashes that happen in the initrd and sandboxed containers. A second motivating use case is when users distribute their own binaries and would like to collect crash information. Build-ids are a solution that is technically possible, but easy to get wrong in practice: users would need to immediately record the build-id after the build and store the mapping to program names, versions, and build number in some database. It's much easier to be able to record something during the build in the build product itself. A third motivating use case is the general mixing of Fedora binaries with programs and libraries from different distributions, both with our binaries being used as the base for foreign binaries, and the other way around. Whilst most distributions provide some mechanism to figure out the source build information, those mechanisms vary by distribution and may not be easy to access from a "foreign" system. Such mixing is expected with containers, flatpaks, snaps, Python binary wheels, anaconda packages, and quite often when somebody compiles a binary and puts it up on the web for other people to download. We propose a new mechanism which is designed to be very simple but extensible: a small JSON document is embedded in an section in the ELF binary. This document can be easily read by a human if necessary, but it is also well-defined and can be processed programatically. For example, `systemd-coredump` will immediately make use of this to display package ''nevra'' information for crashes. The format is also easy to generate, so it can be added to any build system, either using the helpers that we provide or even reimplemented from scratch. For the case where we mix binaries from different distros (the third motivating use case above), this approach is the most useful when this system is used by all distros and even non-distro builds. The more widely it is used, the more useful it becomes. The specification was developed in collaboration with Debian developers, and we hope that Fedora and Debian will lead the way for this to become as widely used as build-ids. But even if the information is only available from some distros, it is still useful, except that fallback mechanisms need to be implemented. === Existing system: `.note.gnu.build-id` === We already have build-ids: every ELF object has a `.note.gnu.build-id` note, and given a core file, we can read the build-id and look it up in the rpm database (`dnf repoquery --whatprovides debuginfo(build-id) = …`) to map it to a package name. Build-ids are unique and compact and very generic and work as expected in general. But they have some downsides: * build-ids are not very informative for users. Before the build-id is converted back to the appropriate package, it's completely opaque. * build-ids require a working rpm database or an internet connection to map to the package name. Three important cases: * minimal containers: the rpm database is not installed in the containers. The information about build-ids needs to be stored externally, so package name information is not available immediately, but only after offline processing. The new note doesn't depend on the rpm db in any way. * handling of a core from a container, where the container and host have different distros * self-built and external packages: unless a lot of care is taken to keep access to the debuginfo packages, this information may be lost. The new note is available even if the repository metadata gets lost. Users can easily provide equivalent information in a format that makes sense in their own environment. It should work even when rpms and debs and other formats are mixed, e.g. during container image creation. === New system: `.note.package` === The new note is created and propagated similarly to `.note.gnu.build-id`. The difference is that we inject the information about package ''nevra'' from the build system. The implementation is very simple: `%{build_ldflags}` are extended with a command to insert a custom note as a separate section in an ELF object. See [https://github.com/systemd/package-notes/blob/main/hello.spec hello.spec] for an example. This is done in the default macros, so all packages that use the prescribed link flags will be affected. The note is a compact json string. This allows the format to be trivially extensible (new fields can be added at will), easy to process (json is extremely popular and parsers are widely available). Using a single field rather than a set of separated notes is more space-efficient. With multiple fields the padding and alignment requirements cause unnecessary overhead. The system was designed with cross-distro collaboration and is flexible enough to identify binaries from different packaging formats and build systems (rpms, debs, custom binaries). See https://systemd.io/COREDUMP_PACKAGE_METADATA/ for detailed description of the format. One of the advantages of using an ELF note, as opposed to say a series of extended attributes on the binary itself, is that the ELF note gets automatically captured and copied into a core file by the kernel. Extended attributes would have to be copied manually, which might not even be possible because the binary on disk may have been removed by the time the crash is analyzed. The overhead is about 200 bytes for each ELF object. We have about overall 33200 files in `/usr/s?bin/` and about 36600 `.so` files (F35, single architecture, results from `dnf repoquery -l 2>/dev/null | rg '^/usr/s?bin/' | sort -u | wc -l`, `dnf repoquery -l 2>/dev/null | rg '^/usr/lib64/.*\.so$' |sort -u|wc -l`). If we do this for the whole distro, we get 69800 × 200 = 13 MB. For a typical installation, we can expect about 300–400 kB. Thus the overhead of additionally used space is neglible (also see the Feedback section for more discussion). Precise measurements TBD once this is turned on and we have real measurements for a larger number of builds. === Examples === <pre> $ objdump -s -j .note.package build/libhello.so build/libhello.so: file format elf64-x86-64 Contents of section .note.package: 02ec 04000000 63000000 7e1afeca 46444f00 ....c...~...FDO. 02fc 7b227479 7065223a 2272706d 222c226e {"type":"rpm","n 030c 616d6522 3a226865 6c6c6f22 2c227665 ame":"hello","ve 031c 7273696f 6e223a22 302d312e 66633335 rsion":"0-1.fc35 032c 2e783836 5f363422 2c226f73 43706522 .x86_64","osCpe" 033c 3a226370 653a2f6f 3a666564 6f726170 :"cpe:/o:fedorap 034c 726f6a65 63743a66 65646f72 613a3333 roject:fedora:33 035c 227d0000 "}.. </pre> <pre> $ readelf --notes build/hello | grep "description data" | sed -e "s/\s*description data: //g" -e "s/ //g" | xxd -p -r | jq readelf: build/hello: Warning: Gap in build notes detected from 0x1091 to 0x10de readelf: build/hello: Warning: Gap in build notes detected from 0x1091 to 0x10af readelf: build/hello: Warning: Gap in build notes detected from 0x1091 to 0x119f { "type": "rpm", "name": "hello", "version": "0-1.fc35.x86_64", "osCpe": "cpe:/o:fedoraproject:fedora:33" } </pre> <pre> $ coredumpctl info PID: 44522 (fsverity) ... Package: fsverity-utils/1.3-1 build-id: ac89bf7175b04d7eec7f6544a923f45be111f0be Message: Process 44522 (fsverity) of user 1000 dumped core. Found module /home/bluca/git/fsverity-utils/libfsverity.so.0 with build-id: fa40fdfb79aea84167c98ca8a89add9ac4f51069 Metadata for module /home/bluca/git/fsverity-utils/libfsverity.so.0 owned by FDO found: { "packageType" : "deb", "package" : "fsverity-utils", "packageVersion" : "1.3-1" } Found module linux-vdso.so.1 with build-id: aba08e06103f725e26f1d7c178fb6b76a564a35d Found module libpthread.so.0 with build-id: e91114987a0147bd050addbd591eb8994b29f4b3 Found module libdl.so.2 with build-id: d3583c742dd47aaa860c5ae0c0c5bdbcd2d54f61 Found module ld-linux-x86-64.so.2 with build-id: f25dfd7b95be4ba386fd71080accae8c0732b711 Found module libcrypto.so.1.1 with build-id: 749142d5ee728a76e7cdc61fd79d2311a77405a2 Found module libc.so.6 with build-id: 18b9a9a8c523e5cfe5b5d946d605d09242f09798 Found module fsverity with build-id: ac89bf7175b04d7eec7f6544a923f45be111f0be Metadata for module fsverity owned by FDO found: { "packageType" : "deb", "package" : "fsverity-utils", "packageVersion" : "1.3-1" } Stack trace of thread 44522: #0 0x00007fe7c8af26f4 __GI___nanosleep (libc.so.6 + 0xc66f4) #1 0x00007fe7c8af262a __sleep (libc.so.6 + 0xc662a) #2 0x00005608481407dd main (fsverity + 0x27dd) #3 0x00007fe7c8a5009b __libc_start_main (libc.so.6 + 0x2409b) #4 0x000056084814094a _start (fsverity + 0x294a) </pre> == Feedback == See [https://github.com/systemd/systemd/issues/18433 systemd issue #18433] for upstream discussion and implementation proposals. === Concerns about additional changes to files === <pre> 17:32:30 <Eighth_Doctor> I think zbyszek underestimates how much of a problem it is to stamp every ELF binary with ''nevra'' data 17:32:44 <mhroncok> zbyszek: so, assuming python has ~100 ELF .so files and I change one text file 17:33:22 <mhroncok> (ignore for the time being that the .so files often changed because of toolchain updates and assume they are stable) </pre> I tested this with python3.10. So far there are 13 builds of that package in F35: `python3.10-3.10.0-1.fc35`, `python3.10-3.10.0~a6-1.fc35`, `python3.10-3.10.0~a6-2.fc35`, `python3.10-3.10.0~a7-1.fc35`, `python3.10-3.10.0~b1-1.fc35`, `python3.10-3.10.0~b2-2.fc35`, `python3.10-3.10.0~b2-3.fc35`, `python3.10-3.10.0~b3-1.fc35`, `python3.10-3.10.0~b4-1.fc35`, `python3.10-3.10.0~b4-2.fc35`, `python3.10-3.10.0~b4-3.fc35`, `python3.10-3.10.0~rc1-1.fc35`, `python3.10-3.10.0~rc2-1.fc35`. I extracted the builds (for `.x86_64`) and made a list of all `.so` files (1368 files), and calculated sha256 hashes for them. No two files repeat, there are 1368 distinct hashes. So the files are '''already''' different between builds and the additional proposed metadata does will not make a significant difference. Note that this range of Python versions encompasses periods when the package is under development and undergoes significant changes (alpha versions), and when it's only undergoing small changes (rc versions). The fact that we get different files in each build is not surprising, because files embed build-ids which differ between builds. But even if we ignore those, binaries generally differ between builds. Even sizes tend to vary between builds: there are 636 distinct `.so` file sizes, i.e. on average any given size only repeats twice (presumably most often for the same file). Running `diffoscope` on `.so` files from different builds shows minor changes in the assembly which I did not analyze futher. If people have specific questions, for example about overhead in some scenario, I'd be happy to answer them. Until now, the issues that were raised were very vague, so it's impossible to answer them. === Why not just use the rpm database? === <pre> 17:34:33 <dcantrell> The main reason for this appears to be that we need the RPM db locally to resolve build-ids to package names. But since containers wipe /var/lib/rpm, we can't do that. So the solution is to put the ''nevra'' in ELF metadata? 17:34:39 <dcantrell> That feels like the wrong approach. </pre> First, there are legitimate reasons to strip packaging metadata from images. For example, for an initrd image from rpms, I get 117 MB of files (without compression), and out of this `/var/lib/rpm` is 5.9 MB, and `/var/lib/dnf` is 4.2 MB. This is an overhead of 9%. This is ''not much'', but still too much to keep in the image unless necessary. Similar ratios will happen for containers of similar size. Reducing image size by one tenth is important. There is no `rpm` or `dnf` in the image, to the package database is not even usable without external tools. As discussed on IRC (https://meetbot.fedoraproject.org/teams/fesco/fesco.2021-05-11-17.01.log.ht…) the containers ''we'' build don't wipe this metadata, but custom Dockerfiles do that. Second, as described in Description section above, not everybody and everything uses rpm. The Fedora motto is "we make an operating system and we make it easy for you to do useful stuff with it" (and yes, this is an actual quote from the official docs), and this stuff involves reusing our binaries in containers and custom installations and whatnot, not just straightforward installations with `dnf`. And in the other direction, people will build their own binaries that are not packaged as rpms. But it is still important to be able to figure out the exact version of a binary, especially after it crashes. === Why do this in Fedora? === <pre> 17:36:49 <mhroncok> I don't understand how non-rpm distros and custom built binaries are affected by our rpm-build environment :/ </pre> The idea is that we inject this into our build system, and Debian injects this into their build system, and so on… As mentioned, this is a cross-distro effort. Also, people can use it in their custom build systems if they build and distribute binaries internally. The scheme would obviously be most useful if used comprehensively, but it's still useful when available partially. We hope that Fedora can lead the way. (This is similar to build-ids: when initially adopted, they were used only by some distros, but were useful even then. Nowadays, with comprehensive adoption, they are even more useful.) https://hpc.guix.info/blog/2021/09/whats-in-a-package/ contains a nice description of a pathological case of packaging hacks and binary redistribution. When trying to unravel something like this, information embedded directly in the binaries would be quite useful. == Benefit to Fedora == A simple and reliable way to gather information about package versions of programs is added. It enhances, instead of replacing, the existing mechanisms. It is particularly useful when reporting crash dumps, but can also be used for image introspection and forensincs, license checks and version scans on containers, etc. If we adopt this in Fedora, Fedora leads the way on implementing the standard. Fedora binaries used in any context can be easily recognized. Fedora binaries provide a better basis to build things. If other distros adopt this, we can introspect and report on those binaries easily within the Fedora context. For example, when somebody is using a container with some programs that originate in the Debian ecosystem, we would be able to identify those programs without tools like `apt` or `dpkg-query`. Core dump analaysis executed in the Fedora host can easily provide useful information about programs from foreign builds. == Implementation in Other Distributions == === Microsoft CBL-Mariner === [https://en.wikipedia.org/wiki/CBL-Mariner CBL-Mariner] is an [https://github.com/microsoft/CBL-Mariner open source] Linux distribution created by Microsoft, targeted at first-party and container workloads on Azure. It is used both as a container runner host and a base container image. Mariner adopted the ELF stamping packaging metadata spec in [https://github.com/microsoft/CBL-Mariner/blob/1.0/SPECS/mariner-rpm-macros/… version 1.0], initially to add OS metadata, and package-level metadata will be added in a following release. === Debian === A package-level proof-of-concept is included in the [https://github.com/systemd/package-notes/blob/main/dh_package_notes package-notes] repository. A [https://salsa.debian.org/bluca/debhelper/-/tree/notes_metadata system-level proof-of-concept] that enables ELF stamping by default in all builds implicitly will be proposed for adoption in the future. == Scope == * Proposal owners: ** create a specification (First version DONE: [https://systemd.io/COREDUMP_PACKAGE_METADATA COREDUMP_PACKAGE_METADATA]. We might need to make some adjustments based on the deployment in Fedora, but no big changes are expected.) ** write a script to generate the package note (First version DONE: [https://github.com/systemd/package-notes/blob/main/generate-package-notes.py generate-package-notes.py]) ** provide a patch for `redhat-rpm-config` to insert appropriate compilation options ** extend systemd's coredumpctl to extract and display this information (DONE: [https://github.com/systemd/systemd/pull/19135 PR #19135], available in systemd-249) ** submit pull request to Packaging Guidelines * Other developers: ** possibly add support in abrt? * Release engineering: There should be no impact. * Policies and guidelines: The new flags should be mentioned in Packaging Guidelines. * Trademark approval: N/A (not needed for this Change) N/A * Alignment with Objectives: It might be relevant for Minimization. Even though it increases the image size a tiny bit, it makes minimized images work a bit better. == Upgrade/compatibility impact == No impact. == How To Test == <pre> $ bash -c 'kill -SEGV $$' $ coredumpctl TIME PID UID GID SIG COREFILE EXE SIZE PACKAGE Mon 2021-03-01 14:37:22 CET 855151 1000 1000 SIGSEGV present /usr/bin/bash 51.7K bash-5.1.0-2.fc34.x86_64 </pre> == User Experience == `coredumpctl` should display information about package versions. `readelf --notes` or similar tools can be used on `.so` files and compiled programs to extract the JSON blurb that describes the originating package. == Dependencies == None. == Contingency Plan == * Contingency mechanism: Remove the new compilation flags. Rebuild any packages that were build with the new flags. * Contingency deadline: Beta freeze. * Blocks release? No. == Documentation == * https://systemd.io/COREDUMP_PACKAGE_METADATA/ * https://github.com/systemd/package-notes See also [[Changes/DebuginfodByDefault]]. -- Ben Cotton He / Him / His Fedora Program Manager Red Hat TZ=America/Indiana/Indianapolis

29 143

Any interest in ROCm packaging?
by Jeremy Newton 25 Jan '22

25 Jan '22

Full disclosure, I am both a Fedora packager and an employee at AMD. To be clear, the following is not at all endorsed by my employer; my interest and use of Fedora is purely a personal hobby and I would like to keep it that way. There has been a recent effort to step up Debian packaging of ROCm, and would like to see if anyone has some interest in expanding the Fedora ROCm packages. I see there's a few packages already, and I'm hoping to help with some internal processes to make ROCm more distro friendly, such as better FHS compliance, clearer licensing, etc. Anyone interested? I would be happy to try to help or review package requests :)

10 15

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

devel December 2021