devel September 2020

devel@lists.fedoraproject.org

239 participants
410 discussions

F34 Change proposal: Debug Info Standardization (from DWZ to -fdebug-types-section) (System-Wide Change proposal)
by Ben Cotton 07 Oct '20

07 Oct '20

https://fedoraproject.org/wiki/Changes/DebugInfoStandardization == Summary == Fedora 18 implemented [[Features/DwarfCompressor]]. As the format did not get widespread and the tool is not much maintained it became burden to make existing debugging tools compatible with Fedora debug info. == Owner == * Name: [[User:jankratochvil| Jan Kratochvil ]] * Email: jan.kratochvil(a)redhat.com == Detailed Description == Debug info files *.debug contained in *-debuginfo.rpm are very big in general (x86_64 Fedora 32 distribution has debug/ directory of 82GB while all its other files are only 75GB). There exist several methods how to make the *-debuginfo.rpms at least a bit smaller. Fedora 18 started using DWZ tool (from [[Features/DwarfCompressor]]) while [https://gcc.gnu.org/pipermail/gcc-patches/2008-August/246281.html Google implemented] the same goal in a different way called -fdebug-types-section. Almost nobody uses existing Fedora DWZ (only Fedora/CentOS/RHEL and SuSE OSes) and so its support is missing in tools like [https://lldb.llvm.org/ LLDB], [[llvm-dwarfdumphttps://llvm.org/docs/CommandGuide/llvm-dwarfdump.html|llvm-d…]] or binutils readelf. -fdebug-types-section is used internally by Google (produced by clang). Debian does not store any debug info archives. Ubuntu uses neither -fdebug-types-section nor DWZ. * DWZ advantage: On the whole Fedora distro it saves 3.3% (5GB of the 157GB distribution size) ** If the 3.3% size increase is a concern I can implement a different optimization ([https://whova.com/embedded/session/llvm_202010/1193947/ talk (2)]) as a GCC post-processing phase which would require no changes in any DWARF consumers. * DWZ disadvantage: DWZ has currently less support across consumers (LLDB, llvm-dwarfdump, binutils readelf) * DWZ disadvantage: DWZ requires 8x times more complicated (LoC count) support in consumers than -fdebug-types-section. * DWZ disadvantage: DWZ cannot update LLVM .debug_names index which can be generated only by clang (it cannot be regenerated later for DWZ-compressed file) * DWZ disadvantage: DWZ DWARF-5 support is a work-in-progress. DWZ has been blocking DWARF-5 for Fedora for 3.5 years and only after I have now proposed to drop DWZ Mark Wielaard has started porting DWZ to DWARF-5. It can be expected next DWARF extensions will remain unsupported again. Even currently there is no plan to support DWARF-5 features used by clang which may need -fdebug-types-section for clang-built binaries or no size optimization of clang-built debug info at all. * DWZ disadvantage: Compilation (linking) requires for C++ up to 2x as big disk space (as DWZ is processing files after linker and DWZ is incompatible with -fdebug-types-section) * DWZ disadvantage: Compilation (linking) is slower This proposed DWARF format was originally submitted already for Fedora 18 as [[Features/DebugTypesSections]]. == Benefit to Fedora == * Better compatibility with existing debugging and tracing tools, primarily [https://lldb.llvm.org/ LLDB]. * Less resource-intensive rebuilds of C++ packages (in disk space, memory requirements and compilation time). == Scope == * Proposal owners: It affects all packages generating *-debuginfo.rpm, that is compiled (not scripted) languages. * Other developers: Report any possible debuginfo incompatibility (unexpected). * Release engineering: [https://pagure.io/releng/issues #Releng issue number] (a check of an impact with Release Engineering is needed) * Policies and guidelines: All the needed changes should be done in [https://src.fedoraproject.org/rpms/redhat-rpm-config redhat-rpm-config]. The [https://src.fedoraproject.org/rpms/dwz dwz package] can be then retired. * Trademark approval: N/A (not needed for this Change) * Alignment with Objectives: The size differences are only for *-debuginfo.rpm which is outside of scope of the listed objectives. == Upgrade/compatibility impact == As *-debuginfo.rpm have to exactly match NVRA of its binary package the compatibility is not relevant. Existing tools supporting DWZ will still support the DWZ file format in packages which have not been rebuilt. == How To Test == The change will update [https://src.fedoraproject.org/rpms/redhat-rpm-config redhat-rpm-config] by [https://people.redhat.com/jkratoch/redhat-rpm-config-fdebug-types-section.p… an -fdebug-types-section patch]. Then one can use rpmbuild to rebuild a package. For mock use -a|--addrepo with modified redhat-rpm-config.rpm (with increased NVRA). For packages already rebuilt in Koji nothing is needed. Test programs like lldb and gdb if they still can print source code, function parameters, variables etc. One should also verify integrated testsuites of tools like clang, lldb, gcc, binutils, gdb, elfutils or rpm are not regressing with the -fdebug-types-section option. One can also compare *.debug files built with/without DWZ and/or -fdebug-types-section using [https://src.fedoraproject.org/rpms/libabigail libabigail] utility dwdiff but that will be rather done by the change owner. == User Experience == No user visible change. This affects what tools can developers use. == Dependencies == none == Contingency Plan == * Contingency mechanism: Revert the change in [https://src.fedoraproject.org/rpms/redhat-rpm-config redhat-rpm-config]. Fedora can continue using DWZ, just some debugging/tracing tools will stay incompatible. * Contingency deadline: beta freeze * Blocks release? No * Blocks product? N/A == Documentation == * [http://www.dwarfstd.org/doc/DWARF5.pdf DWARF-5] E.2 Using Type Units * [https://gcc.gnu.org/onlinedocs/gcc/Debugging-Options.html#index-fdebug-type… GCC -fdebug-types-section] -- Ben Cotton He / Him / His Senior Program Manager, Fedora & CentOS Stream Red Hat TZ=America/Indiana/Indianapolis

17 80

readelf seems broken in F33
by Steve Grubb 06 Oct '20

06 Oct '20

Hello, I was doing some binary analysis of files in F33 and have run across something odd. readelf -s /usr/sbin/auditd | grep GLIBC produces a lot of output like: 182: 0000000000000000 0 FUNC GLOBAL DEFAULT UND [...](a)GLIBC_2.2.5 (3) 184: 0000000000000000 0 FUNC GLOBAL DEFAULT UND [...](a)GLIBC_2.2.5 (3) 185: 0000000000000000 0 FUNC GLOBAL DEFAULT UND [...](a)GLIBC_2.2.5 (3) 186: 0000000000000000 0 FUNC GLOBAL DEFAULT UND [...](a)GLIBC_2.2.5 (3) 187: 0000000000000000 0 FUNC GLOBAL DEFAULT UND [...](a)GLIBC_2.3.2 (2) 191: 0000000000000000 0 FUNC GLOBAL DEFAULT UND alarm(a)GLIBC_2.2.5 (3) 194: 0000000000000000 0 FUNC GLOBAL DEFAULT UND [...](a)GLIBC_2.2.5 (3) 195: 0000000000000000 0 FUNC GLOBAL DEFAULT UND close(a)GLIBC_2.2.5 (3) 197: 0000000000000000 0 FUNC GLOBAL DEFAULT UND [...](a)GLIBC_2.2.5 (3) It's missing a lot of symbols. Is this something with readelf or an odd effect of the LTO changes? -Steve

3 3

Orphaning openbabel
by Dominik 'Rathann' Mierzejewski 06 Oct '20

06 Oct '20

Hi everyone, I've finally admitted to myself that I have no time to take good care of openbabel: https://src.fedoraproject.org/rpms/openbabel It has one FTBFS bug in rawhide (related to cmake macro changes) and two requests to update. One in Fedora to move to the recent 3.x series and one for EPEL to update from 2.3 to 2.4. It has 9 consumers in Fedora: avogadro avogadro2 chemtool ghemical gnome-chemistry-utils IQmol kalzium molsketch xdrawchem Hopefully, one of their maintainers (Bcc'd) or someone from SciTech SIG (Cc'd) can pick this up. I'll orphan it in a few days if nobody responds here with their FAS ID. Regards, -- Fedora https://getfedora.org | RPM Fusion http://rpmfusion.org There should be a science of discontent. People need hard times and oppression to develop psychic muscles. -- from "Collected Sayings of Muad'Dib" by the Princess Irulan

4 8

Fedora 32/33 livedisks do not boot on M$ system(s)
by Marius Schwarz 06 Oct '20

06 Oct '20

Hi, the livecds from F32 and F33 are suffering from a problem not booting on Microsoft device(s) https://bugzilla.redhat.com/show_bug.cgi?id=1879921 https://bugzilla.redhat.com/show_bug.cgi?id=1883593 F31 is booting fine, the newer ones not. Looks like a GrubBootloader issue to me, as not even grub comes up. An instant reset back into the bios happens if a usb boot is tried. Unfortunatly the assignee is not reacting and i would really miss my linux surface after upgrading to F32 :D Best regards, Marius Schwarz

10 37

Orphaning vdr-skinsoppalusikka and the status of my packages (looking for co-maintainers for Finnish spell checking)
by Ville-Pekka Vainio 04 Oct '20

04 Oct '20

Hi all, As some of you may have noticed, I have been away from Fedora packaging for quite a while. I've been hoping to find the time for packaging work, but it hasn't happened. In hindsight, I should have let the project know earlier. I have now orphaned vdr-skinsoppalusikka, because I don't have a working VDR installation anymore. If someone has the ability, my user (vpv) can be dropped from the packages vdr, vdr-epgsearch, vdr-femon and vdr-osdteletext. My main interest in Fedora packaging used to be Finnish spell checking. The spell checking stack, consisting of libreoffice-voikko, libvoikko, voikko-fi and foma (as a dependency) went through quite major changes in 2017-2019. The vocabulary package voikko-fi requires foma to build. I managed to get the foma package reviewed and into Fedora in 2018. It was automatically orphaned and then retired due to FTBFS (https://src.fedoraproject.org/rpms/foma). The code has now been fixed in Github (https://github.com/mhulden/foma) and it should build, but I haven't tried it yet. If I get it to build, I might ask for it to be added back. It seems like I'll be able to work on packaging for a few days a year, so don't expect any miracles. Once foma is built, voikko-fi can be built. Even though it shares some history with the previous vocabulary package malaga-suomi-voikko, it probably needs a new review. Then libvoikko could be updated to use the new vocabulary. If anyone is interested in working on these, I would be happy to get co-maintainers. I'm willing to stay as a co-maintainer of gpodder and python-mygpoclient. I code Python at $DAYJOB so I might be able to help. Best regards, Ville-Pekka Vainio (vpv)

2 2

paranoid md raid1 -> Btrfs migration tools?
by Daniel Pocock 02 Oct '20

02 Oct '20

Are there any tools in Fedora to help people migrate an md RAID1 array to btrfs, with data verification checks? For example 1) check for differences between source file sectors on each source drive 2) for each file, if the file format has a checksum, check it somehow and alert on failures The new emphasis on Btrfs is interesting but it is not much help if people copy in files that are already spoiled. If people can still recover a good copy from their md array then it is a good idea to do so. I could imagine using kpartx to script a solution to (1) above, skipping over the md headers. Some kind of shim may be needed to fool the kernel to see a different UUID for each source volume so they can be mounted simultaneously without md. Regards, Daniel

4 5

LTO and F33
by Jeff Law 02 Oct '20

02 Oct '20

So we're at a point where the F33 FTBFS issues related to LTO that I'm aware of have been resolved (by opting the package out of LTO). I still expect some LTO issues will pop up as packages fix things like missing dependencies, cmake macros, etc. I continue to be available to investigate potential LTO issues, but package maintainers will need to contact me as I'm not actively looking for new LTO issues. My focus is now turning to the packages with LTO opt-outs. I'll be extracting bug reports for upstream (primarily GCC), trying simple workarounds for old style symbol versioning, identifying backports from upstream GCC that allow us to remove LTO opt-outs and the like. So there should be a trickle of opt-outs removed, but otherwise should largely be invisible to the F33 release process. I'd like to thank everyone involved for being patient while we worked through various issues and I look forward to continuing to make incremental improvements now that the bulk of the LTO work has landed. jeff

5 17

F34 Change: Reduce installation media size by improving the compression ratio of SquashFS filesystem (Self-Contained Change)
by Ben Cotton 02 Oct '20

02 Oct '20

https://fedoraproject.org/wiki/Changes/OptimizeSquashFS == Summary == Improve compression ratio of SquashFS filesystem on the installation media. == Owner == * Name: [[User:bkhomuts|Bohdan Khomutskyi]] * Email: bkhomuts(a)redhat.com == Detailed Description == As of Fedora 31, the LiveOS/squashfs.img file on the installation image, is compressed with default block size of mksquashfs. The default block size is 128k. Additionaly, lorax sets BCJ filter depending on the architecture. Those parameters can be adjusted which will lead to a better compression ratio and/or reduction of the CPU usage at build time. This is simple to achieve. Recently, Lorax has gotten support for adjusting the compression options for mksquashfs via the configuration file. The file should be altered as following: <pre> [compression] bcj = no args = -b 1M -Xdict-size 1M -no-recovery </pre> Where -b 1M and -Xdict-size 1M are block and dictionary sizes respectively; bcj -- branch-call-jump filter. Based on the results above, I'd suggest selecting the following ''optimal configuration'': XZ algorithm, with block size of 1MiB and without BCJ filter (plain xz -b 1M, without -Xbcj x86). On the right, you can see the impact of the compression algorithms on installation time. As can be seen from the picture on the right hand side, selecting 'plain xz -b 1M configuration' has minimal impact on the installation time and CPU usage during the installation. The compression will result in +6.51% or, in real terms, +24.94s additional installation time, compared to the savings of 142 MiB on the installation media, refer to the documentation section to download the ISO images. This increase in installation time will be compensated by the change in the installer: https://github.com/rhinstaller/anaconda/pull/2292 I noticed, that even with maximum compression, CPU is not fully utilized during installation. == Benefit to Fedora == * Reduction of the installation media size and the cost of storing and distributing Fedora. * Reduction of the CPU usage at build time. Depending on which compression parameters chosen. == Scope == * Proposal owners: The build environment should have support for adjusting the Lorax configuration file and -squashfs-only parameter. Lorax is a program that produces the LiveOS/squashfs.img file on the installation media. One of the ways to enable such customization is to introduce support in Pungi to pass -c option to Lorax. * Other developers: The pungi utility should support passing the custom configuration file location to the Lorax utility. This option should apply during buildInstall phase of pungi. * Release engineering: [https://pagure.io/releng/issue/9127]  * Policies and guidelines: Not required. * Trademark approval: N/A (not needed for this Change) == Upgrade/compatibility impact == N/A (not a System Wide Change) == How To Test == <pre> mkdir -p /mnt/new /mnt/old sudo mount -o loop,ro FedoraInstallationOld.iso sudo mount -o loop,ro FedoraInstallationNew.iso ls -l /mnt/{new,old}/LiveOS/squashfs.img </pre> And then calculate the size difference. == User Experience == * Decreasing the installation image size will reduce cost of mirroring and storing Fedora installation images. * Decreasing the installation image size will reduce the download time. * Increasing the block size on the current configuration with EXT4 file system, should increase latency while accessing the EXT4 filesystem. The exact impact is to be evaluated. * The impact of latency will be reduced, if the plain SquashFS option is be choosen. == Dependencies == Pungi, a utility that builds the compose, should include new functionality mentioned above. Alternatively, the /etc/lorax/lorax.conf should be altered in the environment where Lorax is running. == Contingency Plan == N/A == Documentation == https://pagure.io/releng/issue/9127. mksquashfs(1) lorax(1) https://docs.pagure.org/pungi Select ISOs can be downloaded at https://khomutsky.com/fedora-dvd/ Spreadsheet pictured above [[File:Comparison Table SquashFS.ods]] [https://github.com/rhinstaller/anaconda/pull/2292 Multi-core decompression of SquashFS merge request] -- Ben Cotton He / Him / His Senior Program Manager, Fedora & CentOS Stream Red Hat TZ=America/Indiana/Indianapolis

13 22

Orphaning 'hub' (the git wrapper for Github)
by Stephen Gallagher 01 Oct '20

01 Oct '20

Upstream is slowly dying out, since the official Github CLI[1] was announced. The latest upstream release doesn't pass the test suite on Golang 1.15[2] and I don't have the time to spare these days, so I'm going to orphan it and let someone else take over. [1] https://cli.github.com/ [2] https://github.com/github/hub/issues/2616

3 3

F33 update stuck for past 6 days in request for testing->stable
by Tony Asleson 01 Oct '20

01 Oct '20

This release: https://bodhi.fedoraproject.org/updates/FEDORA-2020-4da598e74b has been stuck waiting to get moved to stable. Is some error going on that isn't evident? This was a release that I did to hopefully correct what is discussed here: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org… Thanks, -Tony

7 14

← Newer
1
2
3
4
5
6
7
...
41
Older →

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

devel September 2020