kexec March 2020

kexec@lists.fedoraproject.org

6 participants
17 discussions

[PATCH] mkdumprd: allow spaces after 'path' config phrase when network dump
by Kazuhito Hagio 27 May '22

27 May '22

Without this patch, when there are two or more spaces after 'path' configuration phrase with ssh or nfs setting, SAVE_PATH is set to '/var/crash' in mkdumprd, and in most cases kdump service fails to start. ssh kdump(a)192.168.122.1 path /kdump ^^ This behavior would be too sensitive and different from the other configurations. With this patch, mkdumprd allows such spaces. Signed-off-by: Kazuhito Hagio <k-hagio(a)ab.jp.nec.com> --- mkdumprd | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mkdumprd b/mkdumprd index a6f7fe8..aa0abfd 100644 --- a/mkdumprd +++ b/mkdumprd @@ -13,7 +13,7 @@ export IN_KDUMP=1 conf_file="/etc/kdump.conf" SSH_KEY_LOCATION="/root/.ssh/kdump_id_rsa" -SAVE_PATH=$(grep ^path $conf_file| cut -d' ' -f2) +SAVE_PATH=$(awk '/^path/ {print $2}' $conf_file) [ -z "$SAVE_PATH" ] && SAVE_PATH=$DEFAULT_PATH # strip the duplicated "/" SAVE_PATH=$(echo $SAVE_PATH | tr -s /) -- 2.18.0

4 3

[RFC PATCH 0/6] Support for unmounted dump target
by Kairui Song 09 Apr '20

09 Apr '20

This is based on previous clean up patches in: [PATCH 0/5] Simplify bind mount handling and clean up Currently kexec-tools always depend on dump target to be mounted, which caused some inconvenience for setup. So for user configured target, allow kdump to start and build initramfs even if target is not mounted. When a mounted user configured target is used, the behavior is not changed. When a unmounted user configured target is used, mkdumprd will look for corresponding mount info in fstab, and a entry with noauto option is founded, mkdumprd will try to mount it inplace with optoins specified in fstab and do basic checks on the device, then umount it. If there is no fstab entry, mkdumprd will try to mount it in temporary path with defaults option, do same basic check and umount it. If there is a fstab entry but "noauto" option is not used, then there must be some reason that the target device is not mounted, mkdumprd will error out. When path based target is used, there is no behavior change. Some examples: Example 1, target is mounted kdump.conf: nfs 192.168.122.1:/srv/local/nfs path /var/crash core_collector makedumpfile -l --message-level 1 -d 31 mount info (from mount command): 192.168.122.1:/srv/local/nfs on /var/mnt/nfs2 type nfs4 (rw,relatime,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.122.249,local_lock=none,addr=192.168.122.1) mkdumprd will just do do basic kdump checks (fs size, path accisibility), and then build the initramfs using current mount info (mount point, target and options from current system). Example 2, target is not mounted, and marked noauto in fstab: kdump.conf: nfs 192.168.122.1:/srv/local/nfs path /var/crash core_collector makedumpfile -l --message-level 1 -d 31 fstab: 192.168.122.1:/srv/local/nfs /mnt/nfs nfs defaults,noauto 0 0 mkdumprd will try to mount 192.168.122.1:/srv/local/nfs on /mnt/nfs, do basic kdump checks, and then build the initramfs using mount info in fstab. Example 3, target is not mounted, and no entry in fstab: kdump.conf: nfs 192.168.122.1:/srv/local/nfs path /var/crash core_collector makedumpfile -l --message-level 1 -d 31 fstab or current mount: No info related to target. mkdumprd will try to mount 192.168.122.1:/srv/local/nfs on /tmp/mkdumprd.XXXXXX/target with "mount -o defaults", do basic kdump checks, and then build the initramfs using defaults options. In second kernel, target will be mounted on /kdumproot using default options. Kairui Song (6): Fix the problem that kdump prints redundant / Add a is_mounted helper Allow calling mkdumprd from kdumpctl even if targat not mounted kdump-lib.sh: add fstab failback helper for getting mount info User get_mount_info to replace findmnt calls mkdumprd: generate usable kdump initramfs even target is not mounted kdump-lib-initramfs.sh | 54 ++++++++--------- kdump-lib.sh | 58 +++++++++--------- kdumpctl | 9 +-- mkdumprd | 130 +++++++++++++++++++++++++++++------------ 4 files changed, 153 insertions(+), 98 deletions(-) -- 2.24.1

2 9

[PATCH] dracut-module-setup.sh: ensure cluster info is ready before query
by Pingfan Liu 02 Apr '20

02 Apr '20

There is a race issue between "pcs" and "kdumpctl restart" -1. set up cluster -2. Then here comes the command _immediately_ in kdumpctl But due to some pcs internal mechanism, "pcs cluster cib" can not fetch the updated info in time. Fix these issue by forcing the upgrade of cib. Signed-off-by: Pingfan Liu <piliu(a)redhat.com> --- dracut-module-setup.sh | 1 + 1 file changed, 1 insertion(+) diff --git a/dracut-module-setup.sh b/dracut-module-setup.sh index 7a2c7b4..5f91c89 100755 --- a/dracut-module-setup.sh +++ b/dracut-module-setup.sh @@ -712,6 +712,7 @@ is_localhost() { get_pcs_fence_kdump_nodes() { local nodes + pcs cluster sync && pcs cluster cib-upgrade # get cluster nodes from cluster cib, get interface and ip address nodelist=`pcs cluster cib | xmllint --xpath "/cib/status/node_state/@uname" -` -- 2.7.5

3 4

[PATCH 0/5] Simplify bind mount handling and clean up
by Kairui Song 30 Mar '20

30 Mar '20

With this series, bind mount will be treated equally, and no longer have special workaround for atomic. And also cleaned up user configured target handling code. There should be no function change by this fat. Another fix applied here is that in atomic/silverblue enviroment, the path is always resolved as current bind mounted path under root device, but for user configured targets, path should be just used as it is else the behavior will be uncontrollable and go wrong easily. Kairui Song (5): Remove is_dump_target_configured mkdumprd: Use get_save_path instead of parsing config mkdumprd: Simplify handling of user specified target No longer treat atomic/silverblue specially Remove adjust_bind_mount_path call dracut-module-setup.sh | 39 +--------------- kdump-lib.sh | 102 +++++++++++++---------------------------- kdumpctl | 18 ++------ mkdumprd | 37 ++++----------- 4 files changed, 48 insertions(+), 148 deletions(-) -- 2.24.1

2 7

[PATCH] dracut-module-setup.sh: improve get_alias()
by Pingfan Liu 24 Mar '20

24 Mar '20

In /etc/hosts, the alias name can come at the 2nd column, regardless of the recommendation. E.g. the following format is valid although not recommended cat /etc/hosts 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 192.168.22.21 fastvm-rhel-7-6-21 fastvm-rhel-7-6-21.localdomain 192.168.22.22 fastvm-rhel-7-6-22 fastvm-rhel-7-6-22.localdomain 192.168.22.21 node1_hb 192.168.22.22 node2_hb So filtering out both 2nd and 3rd column for matching. Signed-off-by: Pingfan Liu <piliu(a)redhat.com> --- dracut-module-setup.sh | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/dracut-module-setup.sh b/dracut-module-setup.sh index 1237896..767be0a 100755 --- a/dracut-module-setup.sh +++ b/dracut-module-setup.sh @@ -683,7 +683,8 @@ get_alias() { ips=$(hostname -I) for ip in $ips do - entries=$(grep $ip /etc/hosts | awk '{ $1=$2=""; print $0 }') + # in /etc/hosts, alias can come at the 2nd column + entries=$(grep $ip /etc/hosts | awk '{ $1=""; print $0 }') if [ $? -eq 0 ]; then alias_set="$alias_set $entries" fi -- 2.7.5

3 3

[PATCH] fadump: update fadump-howto.txt with some troubleshooting help
by Hari Bathini 24 Mar '20

24 Mar '20

Add recommendations on how much memory is required for FADump. Also, mention the optimizations applied to default initrd when FADump is used and how to workaround it. Signed-off-by: Hari Bathini <hbathini(a)linux.ibm.com> --- * The 4th point added in "Things to remember" section can be dropped, if the fix proposed in Redhat Bug 1796941 is accepted upstream. fadump-howto.txt | 86 +++++++++++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 78 insertions(+), 8 deletions(-) diff --git a/fadump-howto.txt b/fadump-howto.txt index 7061f02..89675a1 100644 --- a/fadump-howto.txt +++ b/fadump-howto.txt @@ -84,19 +84,25 @@ kexec-tools installed. If not, you install it via the following command: # yum install kexec-tools -To be able to do much of anything interesting in the way of debug analysis, -you'll also need to install the kernel-debuginfo package, of the same arch -as your running kernel, and the crash utility: +Make the kernel to be configured with FADump as the default boot entry, if +it isn't already: + + # grubby --set-default=/boot/vmlinuz-<kver> + +Boot into the kernel to be configured for FADump. To be able to do much of +anything interesting in the way of debug analysis, you'll also need to install +the kernel-debuginfo package, of the same arch as your running kernel, and the +crash utility: # yum --enablerepo=\*debuginfo install kernel-debuginfo.$(uname -m) crash Next up, we need to modify some boot parameters to enable firmware assisted dump. With the help of grubby, it's very easy to append "fadump=on" to the end -of your kernel boot parameters. Optionally, user can also append -'fadump_reserve_mem=X' kernel cmdline to specify size of the memory to reserve -for boot memory dump preservation. +of your kernel boot parameters. To reserve the appropriate amount of memory +for boot memory preservation, pass 'crashkernel=X' kernel cmdline parameter. +For the recommended value of X, see 'FADump Memory Requirements' section. - # grubby --args="fadump=on" --update-kernel=/boot/vmlinuz-`uname -r` + # grubby --args="fadump=on crashkernel=6G" --update-kernel=/boot/vmlinuz-`uname -r` The term 'boot memory' means size of the low memory chunk that is required for a kernel to boot successfully when booted with restricted memory. By default, @@ -145,7 +151,7 @@ perform postmortem analysis: and so on... Saving vmcore-dmesg.txt ----------------------- +----------------------- Kernel log bufferes are one of the most important information available in vmcore. Now before saving vmcore, kernel log bufferes are extracted from /proc/vmcore and saved into a file vmcore-dmesg.txt. After @@ -153,6 +159,57 @@ vmcore-dmesg.txt, vmcore is saved. Destination disk and directory for vmcore-dmesg.txt is same as vmcore. Note that kernel log buffers will not be available if dump target is raw device. +FADump Memory Requirements: + + System Memory Recommended memory +--------------------- ---------------------- + 4 GB - 16 GB : 768 MB + 16 GB - 64 GB : 1024 MB + 64 GB - 128 GB : 2 GB + 128 GB - 1 TB : 4 GB + 1 TB - 2 TB : 6 GB + 2 TB - 4 TB : 12 GB + 4 TB - 8 TB : 20 GB + 8 TB - 16 TB : 36 GB + 16 TB - 32 TB : 64 GB + 32 TB - 64 TB : 128 GB + 64 TB & above : 180 GB + +Things to remember: + +1) The memory required to boot capture Kernel is a moving target that depends + on many factors like hardware attached to the system, kernel and modules in + use, packages installed and services enabled, there is no one-size-fits-all. + But the above recommendations are based on system memory. So, the above + recommendations for FADump come with a few assumptions, based on available + system memory, about the resources the system could have. So, please take + the recommendations with a pinch of salt and remember to try capturing dump + a few times to confirm that the system is configured successfully with dump + capturing support. + +2) Though the memory requirements for FADump seem high, this memory is not + completely set aside but made available for userspace applications to use, + through the CMA allocator. + +3) As the same initrd is used for booting production kernel as well as capture + kernel and with dump being captured in a restricted memory environment, few + optimizations (like not inclding network dracut module, disabling multipath + and such) are applied while building the initrd. In case, the production + environment needs these optimizations to be avoided, dracut_args option in + /etc/kdump.conf file could be leveraged. For example, if a user wishes for + network module to be included in the initrd, adding the below entry in + /etc/kdump.conf file and restarting kdump service would take care of it. + + dracut_args --add "network" + +4) If FADump is configured to capture vmcore to a remote dump target using SSH + or NFS protocol, the network interface is renamed to kdump-<interface-name> + if <interface-name> is generic, for example, *eth#, or net#. This problem + occurs because the vmcore capture scripts in the initial RAM disk (initrd) + add the kdump- prefix to the network interface name to secure persistent + naming. As the same initrd is used for production kernel boot, the interface + name is changed for the production kernel too. + Dump Triggering methods: This section talks about the various ways, other than a Kernel Panic, in which @@ -250,3 +307,16 @@ initramfs for the time being. If you need modify "KDUMP_COMMANDLINE=" in options are copied from /proc/cmdline. In general it is best to append command line options using "KDUMP_COMMANDLINE_APPEND=" instead of replacing the original command line completely. + +How to disable FADump: + +Remove "fadump=on" from kernel cmdline parameters: + + # grubby --update-kernel=/boot/vmlinuz-`uname -r` --remove-args="fadump=on" + +If KDump is to be used as the dump capturing mechanism, update the crashkernel +parameter (Else, remove "crashkernel=" parameter too, using grubby): + + # grubby --update-kernel=/boot/vmlinuz-$kver --args="crashkernl=auto" + +Reboot the system for the settings to take effect.

2 1

[PATCH] Fix a potential syntax error
by Kairui Song 23 Mar '20

23 Mar '20

Process substitution is not POSIX standard syntax, so if bash is configured to strictly follow POSIC, this will fail. Just use a POSIX friendly syntax instead. Signed-off-by: Kairui Song <kasong(a)redhat.com> --- dracut-module-setup.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/dracut-module-setup.sh b/dracut-module-setup.sh index 8691f9c..b18f348 100755 --- a/dracut-module-setup.sh +++ b/dracut-module-setup.sh @@ -188,7 +188,7 @@ kdump_handle_mulitpath_route() { _rule="" _max_weight=0 _weight=0 fi done >> ${initdir}/etc/cmdline.d/45route-static.conf\ - < <(/sbin/ip $_ipv6_flag route show) + <<< "$(/sbin/ip $_ipv6_flag route show)" [[ -n $_rule ]] && echo $_rule >> ${initdir}/etc/cmdline.d/45route-static.conf } -- 2.24.1

2 1

[PATCH] Use read_strip_comments to filter the installed kdump.conf
by Kairui Song 23 Mar '20

23 Mar '20

This help remove redundant spaces and tailing comment in installed kdump.conf, currently installed kdump.conf always contain extra empty lines. Signed-off-by: Kairui Song <kasong(a)redhat.com> --- dracut-module-setup.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/dracut-module-setup.sh b/dracut-module-setup.sh index b18f348..e2315d5 100755 --- a/dracut-module-setup.sh +++ b/dracut-module-setup.sh @@ -487,7 +487,7 @@ adjust_bind_mount_path() #install kdump.conf and what user specifies in kdump.conf kdump_install_conf() { local _opt _val _pdev - sed -ne '/^#/!p' /etc/kdump.conf > ${initdir}/tmp/$$-kdump.conf + (read_strip_comments /etc/kdump.conf) > ${initdir}/tmp/$$-kdump.conf while read _opt _val; do -- 2.24.1

2 1

[PATCH] kdumpctl: fix driver change detection on latest Fedora
by Kairui Song 19 Mar '20

19 Mar '20

Now modinfo will return "(builtin)" instead of empty string for builtin module. Sync the code logic. Signed-off-by: Kairui Song <kasong(a)redhat.com> --- kdumpctl | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/kdumpctl b/kdumpctl index 55e333b..2da3846 100755 --- a/kdumpctl +++ b/kdumpctl @@ -404,7 +404,7 @@ check_dump_fs_modified() local _old_dev _old_mntpoint _old_fstype local _new_dev _new_mntpoint _new_fstype local _target _path _dracut_args - local _target_drivers _module_name + local _target_drivers _module_name _module_filename local _old_drivers="$(lsinitrd $TARGET_INITRD -f /usr/lib/dracut/hostonly-kernel-modules.txt | tr '\n' ' ')" @@ -453,10 +453,10 @@ check_dump_fs_modified() check_block_and_slaves_all _record_block_drivers "$(get_maj_min "$_target")" for _driver in $_target_drivers; do - # Target is mounted already, if module is not included by current kernel, - # could be a deprecated/invalid driver name or a built-in module + # Skip deprecated/invalid driver name or built-in module _module_name=$(modinfo --set-version "$kdump_kver" -F name $_driver 2>/dev/null) - if [ $? -ne 0 ] || [ -z "$_module_name" ]; then + _module_filename=$(modinfo --set-version "$kdump_kver" -n $_driver 2>/dev/null) + if [ $? -ne 0 ] || [ -z "$_module_name" ] || [[ "$_module_filename" = *"(builtin)"* ]]; then continue fi if ! [[ " $_old_drivers " == *" $_module_name "* ]]; then -- 2.24.1

2 1

[PATCH] dracut-module-setup.sh: Ensure initrd.target.wants dir exists
by Kairui Song 17 Mar '20

17 Mar '20

Latest dracut release stopped creating $systemdsystemunitdir/initrd.target.wants dir for us, so ensure it exists before creating the symlink. Signed-off-by: Kairui Song <kasong(a)redhat.com> --- dracut-module-setup.sh | 1 + 1 file changed, 1 insertion(+) diff --git a/dracut-module-setup.sh b/dracut-module-setup.sh index 8691f9c..a18d765 100755 --- a/dracut-module-setup.sh +++ b/dracut-module-setup.sh @@ -827,6 +827,7 @@ install() { inst "/lib/kdump/kdump-lib-initramfs.sh" "/lib/kdump-lib-initramfs.sh" inst "$moddir/kdump.sh" "/usr/bin/kdump.sh" inst "$moddir/kdump-capture.service" "$systemdsystemunitdir/kdump-capture.service" + mkdir -p "$initdir/$systemdsystemunitdir/initrd.target.wants" ln_r "$systemdsystemunitdir/kdump-capture.service" "$systemdsystemunitdir/initrd.target.wants/kdump-capture.service" inst "$moddir/kdump-error-handler.sh" "/usr/bin/kdump-error-handler.sh" inst "$moddir/kdump-error-handler.service" "$systemdsystemunitdir/kdump-error-handler.service" -- 2.24.1

2 1

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

kexec March 2020