Hi packagers,
rawhide rpmbuild contains various debuginfo improvements that hopefully will make various hacks in spec files redundant.
If you have your own way of handling debuginfo packages, calling find-debuginfo.sh directly, need hacks for working around debugedit limitations or split your debuginfo package by hand then please try out rpmbuild in rawhide and read below for some macros you can set to tweak debuginfo package generation.
If you still need hacks in your spec file because setting macros isn't enough to get the debuginfo packages you want then please let us know. Also please let us know about packages that need to set debuginfo rpm macros to non-default values because they would crash and burn with the default settings (best to file a bug against rpmbuild).
The improvements have been mainly driven by the following two change proposals for f27 (some inspired by what other distros do):
https://fedoraproject.org/wiki/Changes/ParallelInstallableDebuginfo https://fedoraproject.org/wiki/Changes/SubpackageAndSourceDebuginfo
The first is completely done and has been enabled by default for some months now in rawhide. The second introduces two new macros to enable separate debugsource and sub-debuginfo packages, but has not been enabled by default yet. If people like the change and no bugs are found (and fesco and releng agree) we can enable them for the f27 mass rebuild.
If your package already splits debuginfo packages in a (common) source package and/or sub-debuginfo packages, please try out the new macros introduced by the second change. You can enable the standard splitting by adding the following to your spec file:
%global _debugsource_packages 1 %global _debuginfo_subpackages 1
Besides the above two changes debuginfo packages can now (and are by default in rawhide) build by running debug extraction in parallel. This should speed up building with lots of binaries/libraries. If you do invoke find-debuginfo.sh by hand you most likely will want to add %{?_smp_mflags} as argument to get the parallel processing speedup.
If your package is invoking find-debuginfo.sh by hand also please take a look at all the new options that have been added. Also note that almost all options can be changed by setting (or undefining) rpm macros now. Using the rpm macros is preferred over invoking find-debuginfo.sh directly since it means you get any defaults and improvements that might need new find-debuginfo.sh arguments automatically.
Here is an overview of various debuginfo rpm macros that you can define undefine in your spec file with the latest rpmbuild:
# # Should an ELF file processed by find-debuginfo.sh having no build ID # terminate a build? This is left undefined to disable it and defined to # enable. # %_missing_build_ids_terminate_build 1
# # Include minimal debug information in build binaries. # Requires _enable_debug_packages. # %_include_minidebuginfo 1
# # Include a .gdb_index section in the .debug files. # Requires _enable_debug_packages and gdb-add-index installed. # %_include_gdb_index 1
# # Defines how and if build_id links are generated for ELF files. # The following settings are supported: # # - none # No build_id links are generated. # # - alldebug # build_id links are generated only when the __debug_package global is # defined. This will generate build_id links in the -debuginfo package # for both the main file as /usr/lib/debug/.build-id/xx/yyy and for # the .debug file as /usr/lib/debug/.build-id/xx/yyy.debug. # This is the old style build_id links as generated by the original # find-debuginfo.sh script. # # - separate # build_id links are generate for all binary packages. If this is a # main package (the __debug_package global isn't set) then the # build_id link is generated as /usr/lib/.build-id/xx/yyy. If this is # a -debuginfo package (the __debug_package global is set) then the # build_id link is generated as /usr/lib/debug/.build-id/xx/yyy. # # - compat # Same as for "separate" but if the __debug_package global is set then # the -debuginfo package will have a compatibility link for the main # ELF /usr/lib/debug/.build-id/xx/yyy -> /usr/lib/.build-id/xx/yyy %_build_id_links compat
# Whether build-ids should be made unique between package version/releases # when generating debuginfo packages. If set to 1 this will pass # --build-id-seed "%{VERSION}-%{RELEASE}" to find-debuginfo.sh which will # pass it onto debugedit --build-id-seed to be used to prime the build-id # note hash. %_unique_build_ids 1
# Do not recompute build-ids but keep whatever is in the ELF file already. # Cannot be used together with _unique_build_ids (which forces recomputation). # Defaults to undefined (unset). #%_no_recompute_build_ids 1
# Whether .debug files should be made unique between package version, # release and architecture. If set to 1 this will pass # --unique-debug-suffix "-%{VERSION}-%{RELEASE}.%{_arch} find-debuginfo.sh # to create debuginfo files which end in -<ver>-<rel>.<arch>.debug # Requires _unique_build_ids. %_unique_debug_names 1
# Whether the /usr/debug/src/<package> directories should be unique between # package version, release and architecture. If set to 1 this will pass # --unique-debug-src-base "%{name}-%{VERSION}-%{RELEASE}.%{_arch}" to # find-debuginfo.sh to name the directory under /usr/debug/src as # <name>-<ver>-<rel>.<arch>. %_unique_debug_srcs 1
# Whether rpm should put debug source files into its own subpackage #%_debugsource_packages 1
# Whether rpm should create extra debuginfo packages for each subpackage #%_debuginfo_subpackages 1
# Number of debugging information entries (DIEs) above which # dwz will stop considering file for multifile optimizations # and enter a low memory mode, in which it will optimize # in about half the memory needed otherwise. %_dwz_low_mem_die_limit 10000000 # Number of DIEs above which dwz will stop processing # a file altogether. %_dwz_max_die_limit 50000000
%_find_debuginfo_dwz_opts --run-dwz\\ --dwz-low-mem-die-limit %{_dwz_low_mem_die_limit}\\ --dwz-max-die-limit %{_dwz_max_die_limit}
If there are settings missing that would be useful, bugs with the default settings or defaults that should be changed please do file a bug report.
Thanks,
Mark
Hi packagers,
On Mon, Jun 26, 2017 at 04:13:02PM +0200, Mark Wielaard wrote:
The second introduces two new macros to enable separate debugsource and sub-debuginfo packages, but has not been enabled by default yet. If people like the change and no bugs are found (and fesco and releng agree) we can enable them for the f27 mass rebuild.
If your package already splits debuginfo packages in a (common) source package and/or sub-debuginfo packages, please try out the new macros introduced by the second change. You can enable the standard splitting by adding the following to your spec file:
%global _debugsource_packages 1 %global _debuginfo_subpackages 1
rel-eng is still figuring out whether any updates are necessary to pick up both the -debuginfo and -debugsource pacakges: https://pagure.io/releng/issue/6863
But if you tried out the above I would really like to hear your experiences.
If there are settings missing that would be useful, bugs with the default settings or defaults that should be changed please do file a bug report.
There was one request for keeping a special section for rust packages. "Need a way to exclude sections from eu-strip" https://bugzilla.redhat.com/show_bug.cgi?id=1465997 That was implemented and should be generally useful. It is now possible to add --keep-section=NAME or --remove-section=NAME as _find_debuginfo_opts. Which will make sure a section is explicitly kept in the main binary or explicitly (re)moved to the .debug file.
For completeness here are the different argument that a package can add when using %global _find_debuginfo_opts to tweak how rpm find-debuginfo.sh works which don't have a separate rpm macro:
# The -g flag says to use strip -g instead of full strip on DSOs or EXEs. # # The -r flag says to use eu-strip --reloc-debug-sections. # (this is only really useful for kernel modules) # # Use --keep-section SECTION or --remove-section SECTION to explicitly # keep a (non-allocated) section in the main executable or explicitly # remove it into the .debug file. SECTION is an extended wildcard pattern. # Both options can be given more than once.
The following options should normally not be given directly, but will be used when the corresponding rpm macros are defined:
# %_smp_mflags # The -j N option will spawn N processes to do the debuginfo extraction # in parallel. # # %_missing_build_ids_terminate_build # The --strict-build-id flag says to exit with failure status if # any ELF binary processed fails to contain a build-id note. # # %_no_recompute_build_ids # The -n flag says to not recompute the build-id. # # %_include_minidebuginfo # The -m flag says to include a .gnu_debugdata section in the main binary. # # %_include_gdb_index # The -i flag says to include a .gdb_index section in the .debug file. # # %_unique_build_ids # If --build-id-seed SEED is given then debugedit is called to # update the build-ids it finds adding the SEED as seed to recalculate # the build-id hash. This makes sure the build-ids in the ELF files # are unique between versions and releases of the same package. # (Use --build-id-seed "%{VERSION}-%{RELEASE}".) # # %_unique_debug_names # If --unique-debug-suffix SUFFIX is given then the debug files created # for <FILE> will be named <FILE>-<SUFFIX>.debug. This makes sure .debug # are unique between package version, release and architecture. # (Use --unique-debug-suffix "-%{VERSION}-%{RELEASE}.%{_arch}".) # # %_unique_debug_srcs # If --unique-debug-src-base BASE is given then the source directory # will be called /usr/debug/src/<BASE>. This makes sure the debug source # directories are unique between package version, release and architecture. # (Use --unique-debug-src-base "%{name}-%{VERSION}-%{RELEASE}.%{_arch}".) # # %_find_debuginfo_dwz_opts # %_dwz_low_mem_die_limit # %_dwz_max_die_limit # The --run-dwz flag instructs find-debuginfo.sh to run the dwz utility # if available, and --dwz-low-mem-die-limit and --dwz-max-die-limit # provide detailed limits. See dwz(1) -l and -L option for details. # # %_debugsource_packages # The -S debugsourcefiles.list option will create a debugsourcefiles.list # file containing all source files found in the .debug packages. # This will also exclude all source files from the debugfiles.list.
But please don't invoke find-debuginfo.sh by hand, but use the provided rpm macros. Finally when you are invoking find-debuginfo.sh by hand then the following can be also be used:
# A single -o switch before any -l or -p switches simply renames # the primary output file from debugfiles.list to something else. # A -o switch that follows a -p switch or some -l switches produces # an additional output file with the debuginfo for the files in # the -l filelist file, or whose names match the -p pattern. # The -p argument is an grep -E -style regexp matching the a file name, # and must not use anchors (^ or $).
I would be interested in feedback from packagers that do need to invoke find-debuginfo.sh by hand. And what they believe would be necessary to create their packages without needing to invokle find-debuginfo.sh by hand.
Thanks,
Mark
Could this be causing:
error: Empty %files file /builddir/build/BUILD/cduce-0.6.0/debugsourcefiles.list
in https://koji.fedoraproject.org/koji/taskinfo?taskID=20742651 ?
OCaml generates DWARF information which is picked up for debuginfo files (and is useful), but our debuginfo tools have never been able to locate the source files correctly -- perhaps they need to be told that *.ml files are source files?
Rich.
On Wed, 2017-07-26 at 11:37 +0100, Richard W.M. Jones wrote:
Could this be causing:
error: Empty %files file /builddir/build/BUILD/cduce-0.6.0/debugsourcefiles.list
in https://koji.fedoraproject.org/koji/taskinfo?taskID=20742651 ?
Yes, if %_debugsource_packages is defined and the rpm debugedit cannot find any debug sources then that error would be created. That is obviously a bug. Someone else also just reported it. But I haven't had time to look into it yet.
Note that that the fedora rawhide rpm package now has enabled %_debugsource_packages by default. So you will have to %undefine it in your spec file if you don't want a -debugsource package.
OCaml generates DWARF information which is picked up for debuginfo files (and is useful), but our debuginfo tools have never been able to locate the source files correctly -- perhaps they need to be told that *.ml files are source files?
The problem seems to be that the generated DWARF doesn't use a .debug_line directory table but only plain file names. Those file names are all relative to the debuginfo compile unit comp_dir attribute. But the comp_dir is always the full absolute path:
$ eu-readelf --debug-dump=info ./usr/lib/debug/usr/bin/cduce-0.6.0-23.fc27.x86_64.debug 2>&1 | grep comp_dir comp_dir (strp) "/builddir/build/BUILD/ocamlnet-4.1.2/src/netsys"
This confuses rpm debugedit which assumes anything with an absolute path is outside the build dir and so a system file belonging to another package (e.g. /usr/include/...) and so gets excluded.
That should be fixable in rpm debugedit. I'll try to come up with a patch.
Thanks for the report,
Mark
On Wed, Jul 26, 2017 at 01:21:29PM +0200, Mark Wielaard wrote:
OCaml generates DWARF information which is picked up for debuginfo files (and is useful), but our debuginfo tools have never been able to locate the source files correctly -- perhaps they need to be told that *.ml files are source files?
The problem seems to be that the generated DWARF doesn't use a .debug_line directory table but only plain file names. Those file names are all relative to the debuginfo compile unit comp_dir attribute. But the comp_dir is always the full absolute path:
$ eu-readelf --debug-dump=info ./usr/lib/debug/usr/bin/cduce-0.6.0-23.fc27.x86_64.debug 2>&1 | grep comp_dir comp_dir (strp) "/builddir/build/BUILD/ocamlnet-4.1.2/src/netsys"
This confuses rpm debugedit which assumes anything with an absolute path is outside the build dir and so a system file belonging to another package (e.g. /usr/include/...) and so gets excluded.
That should be fixable in rpm debugedit. I'll try to come up with a patch.
Thanks for the report,
Upstream OCaml doesn't generate that directive either, but I might have a go at adding it ...
Rich.
On Wed, Jul 26, 2017 at 01:21:29PM +0200, Mark Wielaard wrote:
On Wed, 2017-07-26 at 11:37 +0100, Richard W.M. Jones wrote:
Could this be causing:
error: Empty %files file /builddir/build/BUILD/cduce-0.6.0/debugsourcefiles.list
in https://koji.fedoraproject.org/koji/taskinfo?taskID=20742651 ?
Yes, if %_debugsource_packages is defined and the rpm debugedit cannot find any debug sources then that error would be created. That is obviously a bug. Someone else also just reported it. But I haven't had time to look into it yet.
Note that that the fedora rawhide rpm package now has enabled %_debugsource_packages by default. So you will have to %undefine it in your spec file if you don't want a -debugsource package.
OCaml generates DWARF information which is picked up for debuginfo files (and is useful), but our debuginfo tools have never been able to locate the source files correctly -- perhaps they need to be told that *.ml files are source files?
The problem seems to be that the generated DWARF doesn't use a .debug_line directory table but only plain file names. Those file names are all relative to the debuginfo compile unit comp_dir attribute. But the comp_dir is always the full absolute path:
$ eu-readelf --debug-dump=info ./usr/lib/debug/usr/bin/cduce-0.6.0-23.fc27.x86_64.debug 2>&1 | grep comp_dir comp_dir (strp) "/builddir/build/BUILD/ocamlnet-4.1.2/src/netsys"
This confuses rpm debugedit which assumes anything with an absolute path is outside the build dir and so a system file belonging to another package (e.g. /usr/include/...) and so gets excluded.
That should be fixable in rpm debugedit. I'll try to come up with a patch.
I think actually the compiler is correct, but there's another thing going on.
Firstly, here's an example of the compiler being (ISTM) correct:
$ pwd /var/tmp $ cat comp/test.ml print_endline "hello world" $ ocamlopt.opt -g ./comp/test.ml -o ./comp/test-ocaml $ eu-readelf --debug-dump=info ./comp/test-ocaml |& less Compilation unit at offset 0: ... name (strp) "pervasives.ml" comp_dir (strp) "/builddir/build/BUILD/ocaml-4.04.2/stdlib" Compilation unit at offset 46: ... name (strp) "./comp/test.ml" comp_dir (strp) "/var/tmp"
What's happened is that some functions from the pervasives.ml stdlib have been statically linked into the final binary.
The same happens in the cduce binary, with a bunch of stuff being statically linked.
However the problem is that cduce itself was compiled without the ‘-g’ flag, so none of the debuginfo refers to the main binary, only to statically linked libraries. That's why the paths appear to only refer to random external directories (the ones which the libraries were compiled under) and nothing under the cduce build directory.
That's a bug in cduce.spec which I'll fix shortly.
Rich.
On Wed, Jul 26, 2017 at 01:27:59PM +0100, Richard W.M. Jones wrote:
The same happens in the cduce binary, with a bunch of stuff being statically linked.
However the problem is that cduce itself was compiled without the ‘-g’ flag, so none of the debuginfo refers to the main binary, only to statically linked libraries. That's why the paths appear to only refer to random external directories (the ones which the libraries were compiled under) and nothing under the cduce build directory.
That's a bug in cduce.spec which I'll fix shortly.
So I fixed that:
http://pkgs.fedoraproject.org/cgit/rpms/cduce.git/commit/?id=7be2ca68241140b...
and it fixes the debuginfo problem, yay!
And we even get a debugsource package which contains sources, even better!
This isn't a problem in either ocaml or the debuginfo scripts after all.
Rich.
I should say there is still a kind of problem, although I don't think it's a real bug.
Because of the statically linked sources, during debuginfo generation it prints a bunch of warnings:
cpio: big_int.ml: Cannot stat: No such file or directory cpio: buffer.ml: Cannot stat: No such file or directory cpio: bytes.ml: Cannot stat: No such file or directory cpio: curl.ml: Cannot stat: No such file or directory [etc]
At least I now understand where those come from and why they can be ignored.
Rich.
On Monday, June 26, 2017 16:13:02 Mark Wielaard wrote:
Hi packagers,
rawhide rpmbuild contains various debuginfo improvements that hopefully will make various hacks in spec files redundant.
These improvements break build of packages that use the RemovePathPostfixes feature of RPM, such as coreutils or curl:
https://koji.fedoraproject.org/koji/getfile?taskID=20745024&volume=DEFAU... https://koji.fedoraproject.org/koji/getfile?taskID=20745984&volume=DEFAU...
Is this expected?
Kamil
If you have your own way of handling debuginfo packages, calling find-debuginfo.sh directly, need hacks for working around debugedit limitations or split your debuginfo package by hand then please try out rpmbuild in rawhide and read below for some macros you can set to tweak debuginfo package generation.
If you still need hacks in your spec file because setting macros isn't enough to get the debuginfo packages you want then please let us know. Also please let us know about packages that need to set debuginfo rpm macros to non-default values because they would crash and burn with the default settings (best to file a bug against rpmbuild).
The improvements have been mainly driven by the following two change proposals for f27 (some inspired by what other distros do):
https://fedoraproject.org/wiki/Changes/ParallelInstallableDebuginfo https://fedoraproject.org/wiki/Changes/SubpackageAndSourceDebuginfo
The first is completely done and has been enabled by default for some months now in rawhide. The second introduces two new macros to enable separate debugsource and sub-debuginfo packages, but has not been enabled by default yet. If people like the change and no bugs are found (and fesco and releng agree) we can enable them for the f27 mass rebuild.
If your package already splits debuginfo packages in a (common) source package and/or sub-debuginfo packages, please try out the new macros introduced by the second change. You can enable the standard splitting by adding the following to your spec file:
%global _debugsource_packages 1 %global _debuginfo_subpackages 1
Besides the above two changes debuginfo packages can now (and are by default in rawhide) build by running debug extraction in parallel. This should speed up building with lots of binaries/libraries. If you do invoke find-debuginfo.sh by hand you most likely will want to add %{?_smp_mflags} as argument to get the parallel processing speedup.
If your package is invoking find-debuginfo.sh by hand also please take a look at all the new options that have been added. Also note that almost all options can be changed by setting (or undefining) rpm macros now. Using the rpm macros is preferred over invoking find-debuginfo.sh directly since it means you get any defaults and improvements that might need new find-debuginfo.sh arguments automatically.
Here is an overview of various debuginfo rpm macros that you can define undefine in your spec file with the latest rpmbuild:
# # Should an ELF file processed by find-debuginfo.sh having no build ID # terminate a build? This is left undefined to disable it and defined to # enable. # %_missing_build_ids_terminate_build 1
# # Include minimal debug information in build binaries. # Requires _enable_debug_packages. # %_include_minidebuginfo 1
# # Include a .gdb_index section in the .debug files. # Requires _enable_debug_packages and gdb-add-index installed. # %_include_gdb_index 1
# # Defines how and if build_id links are generated for ELF files. # The following settings are supported: # # - none # No build_id links are generated. # # - alldebug # build_id links are generated only when the __debug_package global is # defined. This will generate build_id links in the -debuginfo package # for both the main file as /usr/lib/debug/.build-id/xx/yyy and for # the .debug file as /usr/lib/debug/.build-id/xx/yyy.debug. # This is the old style build_id links as generated by the original # find-debuginfo.sh script. # # - separate # build_id links are generate for all binary packages. If this is a # main package (the __debug_package global isn't set) then the # build_id link is generated as /usr/lib/.build-id/xx/yyy. If this is # a -debuginfo package (the __debug_package global is set) then the # build_id link is generated as /usr/lib/debug/.build-id/xx/yyy. # # - compat # Same as for "separate" but if the __debug_package global is set then # the -debuginfo package will have a compatibility link for the main # ELF /usr/lib/debug/.build-id/xx/yyy -> /usr/lib/.build-id/xx/yyy %_build_id_links compat
# Whether build-ids should be made unique between package version/releases # when generating debuginfo packages. If set to 1 this will pass # --build-id-seed "%{VERSION}-%{RELEASE}" to find-debuginfo.sh which will # pass it onto debugedit --build-id-seed to be used to prime the build-id # note hash. %_unique_build_ids 1
# Do not recompute build-ids but keep whatever is in the ELF file already. # Cannot be used together with _unique_build_ids (which forces recomputation). # Defaults to undefined (unset). #%_no_recompute_build_ids 1
# Whether .debug files should be made unique between package version, # release and architecture. If set to 1 this will pass # --unique-debug-suffix "-%{VERSION}-%{RELEASE}.%{_arch} find-debuginfo.sh # to create debuginfo files which end in -<ver>-<rel>.<arch>.debug # Requires _unique_build_ids. %_unique_debug_names 1
# Whether the /usr/debug/src/<package> directories should be unique between # package version, release and architecture. If set to 1 this will pass # --unique-debug-src-base "%{name}-%{VERSION}-%{RELEASE}.%{_arch}" to # find-debuginfo.sh to name the directory under /usr/debug/src as # <name>-<ver>-<rel>.<arch>. %_unique_debug_srcs 1
# Whether rpm should put debug source files into its own subpackage #%_debugsource_packages 1
# Whether rpm should create extra debuginfo packages for each subpackage #%_debuginfo_subpackages 1
# Number of debugging information entries (DIEs) above which # dwz will stop considering file for multifile optimizations # and enter a low memory mode, in which it will optimize # in about half the memory needed otherwise. %_dwz_low_mem_die_limit 10000000 # Number of DIEs above which dwz will stop processing # a file altogether. %_dwz_max_die_limit 50000000
%_find_debuginfo_dwz_opts --run-dwz\\ --dwz-low-mem-die-limit %{_dwz_low_mem_die_limit}\\ --dwz-max-die-limit %{_dwz_max_die_limit}
If there are settings missing that would be useful, bugs with the default settings or defaults that should be changed please do file a bug report.
Thanks,
Mark
On Wed, 2017-07-26 at 14:38 +0200, Kamil Dudka wrote:
On Monday, June 26, 2017 16:13:02 Mark Wielaard wrote:
rawhide rpmbuild contains various debuginfo improvements that hopefully will make various hacks in spec files redundant.
These improvements break build of packages that use the RemovePathPostfixes feature of RPM, such as coreutils or curl:
https://koji.fedoraproject.org/koji/getfile?taskID=20745024&volume=DEFAU... https://koji.fedoraproject.org/koji/getfile?taskID=20745984&volume=DEFAU...
Is this expected?
No, please file a bug and don't enable (or disable) any features that cause trouble for your package. Note that since rpm-4.13.0.1-37 both %_debugsource_packages and %_debuginfo_subpackages have been enabled by default in rawhide.
Thanks,
Mark
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256
On Wed, 2017-07-26 at 14:38 +0200, Kamil Dudka wrote:
On Monday, June 26, 2017 16:13:02 Mark Wielaard wrote:
Hi packagers,
rawhide rpmbuild contains various debuginfo improvements that hopefully will make various hacks in spec files redundant.
These improvements break build of packages that use the RemovePathPostfixes feature of RPM, such as coreutils or curl:
https://koji.fedoraproject.org/koji/getfile?taskID=20745024&volume=DE FAULT&name=build.log&offset=-4000 https://koji.fedoraproject.org/koji/getfile?taskID=20745984&volume=DE FAULT&name=build.log&offset=-4000
Is this expected?
Definitely not, but thanks a lot for reporting!
I've created issue for rpm[0] to track this problem. For now, feel free to add %undefine _debuginfo_subpackages on top of your spec files.
Sorry for breakage.
[0]https://github.com/rpm-software-management/rpm/issues/280
Kamil
If you have your own way of handling debuginfo packages, calling find-debuginfo.sh directly, need hacks for working around debugedit limitations or split your debuginfo package by hand then please try out rpmbuild in rawhide and read below for some macros you can set to tweak debuginfo package generation.
If you still need hacks in your spec file because setting macros isn't enough to get the debuginfo packages you want then please let us know. Also please let us know about packages that need to set debuginfo rpm macros to non-default values because they would crash and burn with the default settings (best to file a bug against rpmbuild).
The improvements have been mainly driven by the following two change proposals for f27 (some inspired by what other distros do):
https://fedoraproject.org/wiki/Changes/ParallelInstallableDebuginfo https://fedoraproject.org/wiki/Changes/SubpackageAndSourceDebuginfo
The first is completely done and has been enabled by default for some months now in rawhide. The second introduces two new macros to enable separate debugsource and sub-debuginfo packages, but has not been enabled by default yet. If people like the change and no bugs are found (and fesco and releng agree) we can enable them for the f27 mass rebuild.
If your package already splits debuginfo packages in a (common) source package and/or sub-debuginfo packages, please try out the new macros introduced by the second change. You can enable the standard splitting by adding the following to your spec file:
%global _debugsource_packages 1 %global _debuginfo_subpackages 1
Besides the above two changes debuginfo packages can now (and are by default in rawhide) build by running debug extraction in parallel. This should speed up building with lots of binaries/libraries. If you do invoke find-debuginfo.sh by hand you most likely will want to add %{?_smp_mflags} as argument to get the parallel processing speedup.
If your package is invoking find-debuginfo.sh by hand also please take a look at all the new options that have been added. Also note that almost all options can be changed by setting (or undefining) rpm macros now. Using the rpm macros is preferred over invoking find-debuginfo.sh directly since it means you get any defaults and improvements that might need new find-debuginfo.sh arguments automatically.
Here is an overview of various debuginfo rpm macros that you can define undefine in your spec file with the latest rpmbuild:
# # Should an ELF file processed by find-debuginfo.sh having no build ID # terminate a build? This is left undefined to disable it and defined to # enable. # %_missing_build_ids_terminate_build 1
# # Include minimal debug information in build binaries. # Requires _enable_debug_packages. # %_include_minidebuginfo 1
# # Include a .gdb_index section in the .debug files. # Requires _enable_debug_packages and gdb-add-index installed. # %_include_gdb_index 1
# # Defines how and if build_id links are generated for ELF files. # The following settings are supported: # # - none # No build_id links are generated. # # - alldebug # build_id links are generated only when the __debug_package global is # defined. This will generate build_id links in the -debuginfo package # for both the main file as /usr/lib/debug/.build-id/xx/yyy and for # the .debug file as /usr/lib/debug/.build-id/xx/yyy.debug. # This is the old style build_id links as generated by the original # find-debuginfo.sh script. # # - separate # build_id links are generate for all binary packages. If this is a # main package (the __debug_package global isn't set) then the # build_id link is generated as /usr/lib/.build-id/xx/yyy. If this is # a -debuginfo package (the __debug_package global is set) then the # build_id link is generated as /usr/lib/debug/.build-id/xx/yyy. # # - compat # Same as for "separate" but if the __debug_package global is set then # the -debuginfo package will have a compatibility link for the main # ELF /usr/lib/debug/.build-id/xx/yyy -> /usr/lib/.build- id/xx/yyy %_build_id_links compat
# Whether build-ids should be made unique between package version/releases # when generating debuginfo packages. If set to 1 this will pass # --build-id-seed "%{VERSION}-%{RELEASE}" to find-debuginfo.sh which will # pass it onto debugedit --build-id-seed to be used to prime the build-id # note hash. %_unique_build_ids 1
# Do not recompute build-ids but keep whatever is in the ELF file already. # Cannot be used together with _unique_build_ids (which forces recomputation). # Defaults to undefined (unset). #%_no_recompute_build_ids 1
# Whether .debug files should be made unique between package version, # release and architecture. If set to 1 this will pass # --unique-debug-suffix "-%{VERSION}-%{RELEASE}.%{_arch} find- debuginfo.sh # to create debuginfo files which end in -<ver>-<rel>.<arch>.debug # Requires _unique_build_ids. %_unique_debug_names 1
# Whether the /usr/debug/src/<package> directories should be unique between # package version, release and architecture. If set to 1 this will pass # --unique-debug-src-base "%{name}-%{VERSION}-%{RELEASE}.%{_arch}" to # find-debuginfo.sh to name the directory under /usr/debug/src as # <name>-<ver>-<rel>.<arch>. %_unique_debug_srcs 1
# Whether rpm should put debug source files into its own subpackage #%_debugsource_packages 1
# Whether rpm should create extra debuginfo packages for each subpackage #%_debuginfo_subpackages 1
# Number of debugging information entries (DIEs) above which # dwz will stop considering file for multifile optimizations # and enter a low memory mode, in which it will optimize # in about half the memory needed otherwise. %_dwz_low_mem_die_limit 10000000 # Number of DIEs above which dwz will stop processing # a file altogether. %_dwz_max_die_limit 50000000
%_find_debuginfo_dwz_opts --run-dwz\\ --dwz-low-mem-die-limit %{_dwz_low_mem_die_limit}\\ --dwz-max-die-limit %{_dwz_max_die_limit}
If there are settings missing that would be useful, bugs with the default settings or defaults that should be changed please do file a bug report.
Thanks,
Mark
devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-leave@lists.fedoraproject.org
- -- - -Igor Gnatenko
On Wednesday, July 26, 2017 14:55:34 Igor Gnatenko wrote:
On Wed, 2017-07-26 at 14:38 +0200, Kamil Dudka wrote:
On Monday, June 26, 2017 16:13:02 Mark Wielaard wrote:
Hi packagers,
rawhide rpmbuild contains various debuginfo improvements that hopefully will make various hacks in spec files redundant.
These improvements break build of packages that use the RemovePathPostfixes feature of RPM, such as coreutils or curl:
https://koji.fedoraproject.org/koji/getfile?taskID=20745024&volume=DE FAULT&name=build.log&offset=-4000 https://koji.fedoraproject.org/koji/getfile?taskID=20745984&volume=DE FAULT&name=build.log&offset=-4000
Is this expected?
Definitely not, but thanks a lot for reporting!
I've created issue for rpm[0] to track this problem. For now, feel free to add %undefine _debuginfo_subpackages on top of your spec files.
Thanks for the quick reply on this! Using the above trick helped me to make both coreutils and curl build successfully again.
Kamil
Sorry for breakage.
[0]https://github.com/rpm-software-management/rpm/issues/280
Kamil
If you have your own way of handling debuginfo packages, calling find-debuginfo.sh directly, need hacks for working around debugedit limitations or split your debuginfo package by hand then please try out rpmbuild in rawhide and read below for some macros you can set to tweak debuginfo package generation.
If you still need hacks in your spec file because setting macros isn't enough to get the debuginfo packages you want then please let us know. Also please let us know about packages that need to set debuginfo rpm macros to non-default values because they would crash and burn with the default settings (best to file a bug against rpmbuild).
The improvements have been mainly driven by the following two change proposals for f27 (some inspired by what other distros do):
https://fedoraproject.org/wiki/Changes/ParallelInstallableDebuginfo https://fedoraproject.org/wiki/Changes/SubpackageAndSourceDebuginfo
The first is completely done and has been enabled by default for some months now in rawhide. The second introduces two new macros to enable separate debugsource and sub-debuginfo packages, but has not been enabled by default yet. If people like the change and no bugs are found (and fesco and releng agree) we can enable them for the f27 mass rebuild.
If your package already splits debuginfo packages in a (common) source package and/or sub-debuginfo packages, please try out the new macros introduced by the second change. You can enable the standard splitting by adding the following to your spec file:
%global _debugsource_packages 1 %global _debuginfo_subpackages 1
Besides the above two changes debuginfo packages can now (and are by default in rawhide) build by running debug extraction in parallel. This should speed up building with lots of binaries/libraries. If you do invoke find-debuginfo.sh by hand you most likely will want to add %{?_smp_mflags} as argument to get the parallel processing speedup.
If your package is invoking find-debuginfo.sh by hand also please take a look at all the new options that have been added. Also note that almost all options can be changed by setting (or undefining) rpm macros now. Using the rpm macros is preferred over invoking find-debuginfo.sh directly since it means you get any defaults and improvements that might need new find-debuginfo.sh arguments automatically.
Here is an overview of various debuginfo rpm macros that you can define undefine in your spec file with the latest rpmbuild:
# # Should an ELF file processed by find-debuginfo.sh having no build ID # terminate a build? This is left undefined to disable it and defined to # enable. # %_missing_build_ids_terminate_build 1
# # Include minimal debug information in build binaries. # Requires _enable_debug_packages. # %_include_minidebuginfo 1
# # Include a .gdb_index section in the .debug files. # Requires _enable_debug_packages and gdb-add-index installed. # %_include_gdb_index 1
# # Defines how and if build_id links are generated for ELF files. # The following settings are supported: # # - none # No build_id links are generated. # # - alldebug # build_id links are generated only when the __debug_package global is # defined. This will generate build_id links in the -debuginfo package # for both the main file as /usr/lib/debug/.build-id/xx/yyy and for # the .debug file as /usr/lib/debug/.build-id/xx/yyy.debug. # This is the old style build_id links as generated by the original # find-debuginfo.sh script. # # - separate # build_id links are generate for all binary packages. If this is a # main package (the __debug_package global isn't set) then the # build_id link is generated as /usr/lib/.build-id/xx/yyy. If this is # a -debuginfo package (the __debug_package global is set) then the # build_id link is generated as /usr/lib/debug/.build-id/xx/yyy. # # - compat # Same as for "separate" but if the __debug_package global is set then # the -debuginfo package will have a compatibility link for the main # ELF /usr/lib/debug/.build-id/xx/yyy -> /usr/lib/.build- id/xx/yyy %_build_id_links compat
# Whether build-ids should be made unique between package version/releases # when generating debuginfo packages. If set to 1 this will pass # --build-id-seed "%{VERSION}-%{RELEASE}" to find-debuginfo.sh which will # pass it onto debugedit --build-id-seed to be used to prime the build-id # note hash. %_unique_build_ids 1
# Do not recompute build-ids but keep whatever is in the ELF file already. # Cannot be used together with _unique_build_ids (which forces recomputation). # Defaults to undefined (unset). #%_no_recompute_build_ids 1
# Whether .debug files should be made unique between package version, # release and architecture. If set to 1 this will pass # --unique-debug-suffix "-%{VERSION}-%{RELEASE}.%{_arch} find- debuginfo.sh # to create debuginfo files which end in -<ver>-<rel>.<arch>.debug # Requires _unique_build_ids. %_unique_debug_names 1
# Whether the /usr/debug/src/<package> directories should be unique between # package version, release and architecture. If set to 1 this will pass # --unique-debug-src-base "%{name}-%{VERSION}-%{RELEASE}.%{_arch}" to # find-debuginfo.sh to name the directory under /usr/debug/src as # <name>-<ver>-<rel>.<arch>. %_unique_debug_srcs 1
# Whether rpm should put debug source files into its own subpackage #%_debugsource_packages 1
# Whether rpm should create extra debuginfo packages for each subpackage #%_debuginfo_subpackages 1
# Number of debugging information entries (DIEs) above which # dwz will stop considering file for multifile optimizations # and enter a low memory mode, in which it will optimize # in about half the memory needed otherwise. %_dwz_low_mem_die_limit 10000000 # Number of DIEs above which dwz will stop processing # a file altogether. %_dwz_max_die_limit 50000000
%_find_debuginfo_dwz_opts --run-dwz\\
--dwz-low-mem-die-limit %{_dwz_low_mem_die_limit}\\ --dwz-max-die-limit %{_dwz_max_die_limit}
If there are settings missing that would be useful, bugs with the default settings or defaults that should be changed please do file a bug report.
Thanks,
Mark
devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-leave@lists.fedoraproject.org
-- -Igor Gnatenko
devel@lists.stg.fedoraproject.org
-
Igor Gnatenko -
Kamil Dudka -
Mark Wielaard -
Richard W.M. Jones