kexec December 2018

kexec@lists.fedoraproject.org

9 participants
7 discussions

[PATCH] mkdumprd: allow spaces after 'path' config phrase when network dump
by Kazuhito Hagio 26 May '22

26 May '22

Without this patch, when there are two or more spaces after 'path' configuration phrase with ssh or nfs setting, SAVE_PATH is set to '/var/crash' in mkdumprd, and in most cases kdump service fails to start. ssh kdump(a)192.168.122.1 path /kdump ^^ This behavior would be too sensitive and different from the other configurations. With this patch, mkdumprd allows such spaces. Signed-off-by: Kazuhito Hagio <k-hagio(a)ab.jp.nec.com> --- mkdumprd | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mkdumprd b/mkdumprd index a6f7fe8..aa0abfd 100644 --- a/mkdumprd +++ b/mkdumprd @@ -13,7 +13,7 @@ export IN_KDUMP=1 conf_file="/etc/kdump.conf" SSH_KEY_LOCATION="/root/.ssh/kdump_id_rsa" -SAVE_PATH=$(grep ^path $conf_file| cut -d' ' -f2) +SAVE_PATH=$(awk '/^path/ {print $2}' $conf_file) [ -z "$SAVE_PATH" ] && SAVE_PATH=$DEFAULT_PATH # strip the duplicated "/" SAVE_PATH=$(echo $SAVE_PATH | tr -s /) -- 2.18.0

4 3

[PATCH] Avoid falling into infinite loop restart when using a problematic system
by Lianbo Jiang 08 Jan '19

08 Jan '19

By default, early kdump reboots the system after capturing the vmcore. If the problematic system is continuously crashing due to some issue during early boot stage, the system may fall into infinite loop restart like this: boot -----> crash -----> early kdump (dump vmcore) ^ | '.........(reboot).............' But now, the system crash at early stage is only captured by early kdump, and the rest is captured by normal kdump. That to say, when normal kdump service starts, it will load it again and override early kdump. It is helpful to control the logic of early kdump and normal kdump separately in final action(it is called by kdump-capture.service). For example, early kdump always passes the 'rd.earlykdump' to the second kernel when early kdump is enabled, but normal kdump doesn't pass the 'rd.earlykdump' to the second kernel at any time. So they can be distinguished in the second kernel. Signed-off-by: Lianbo Jiang <lijiang(a)redhat.com> --- dracut-early-kdump.sh | 2 +- dracut-kdump-error-handler.sh | 7 ++++++- dracut-kdump.sh | 6 +++++- kdump-lib-initramfs.sh | 16 +++++++++++++--- kdump.sysconfig | 2 +- kdump.sysconfig.aarch64 | 2 +- kdump.sysconfig.i386 | 2 +- kdump.sysconfig.ppc64 | 2 +- kdump.sysconfig.ppc64le | 2 +- kdump.sysconfig.s390x | 2 +- kdump.sysconfig.x86_64 | 2 +- kdumpctl | 15 +++++++++++++-- 12 files changed, 45 insertions(+), 15 deletions(-) diff --git a/dracut-early-kdump.sh b/dracut-early-kdump.sh index 34a9909..a799dbf 100755 --- a/dracut-early-kdump.sh +++ b/dracut-early-kdump.sh @@ -58,7 +58,7 @@ early_kdump_load() fi $KEXEC ${EARLY_KEXEC_ARGS} $standard_kexec_args \ - --command-line="$EARLY_KDUMP_CMDLINE" \ + --command-line="$EARLY_KDUMP_CMDLINE rd.earlykdump" \ --initrd=$EARLY_KDUMP_INITRD $EARLY_KDUMP_KERNEL if [ $? == 0 ]; then echo "kexec: loaded early-kdump kernel" diff --git a/dracut-kdump-error-handler.sh b/dracut-kdump-error-handler.sh index 2f0f1d1..4f0e58c 100755 --- a/dracut-kdump-error-handler.sh +++ b/dracut-kdump-error-handler.sh @@ -1,5 +1,6 @@ #!/bin/sh +. /lib/dracut-lib.sh . /lib/kdump-lib-initramfs.sh set -o pipefail @@ -7,4 +8,8 @@ export PATH=$PATH:$KDUMP_SCRIPT_DIR get_kdump_confs do_default_action -do_final_action +if getargbool 0 rd.earlykdump; then + do_earlykdump_final_action +else + do_final_action +fi diff --git a/dracut-kdump.sh b/dracut-kdump.sh index b75c2a5..119b006 100755 --- a/dracut-kdump.sh +++ b/dracut-kdump.sh @@ -201,4 +201,8 @@ if [ $DUMP_RETVAL -ne 0 ]; then exit 1 fi -do_final_action +if getargbool 0 rd.earlykdump; then + do_earlykdump_final_action +else + do_final_action +fi diff --git a/kdump-lib-initramfs.sh b/kdump-lib-initramfs.sh index 7ba99b6..f95cbcf 100755 --- a/kdump-lib-initramfs.sh +++ b/kdump-lib-initramfs.sh @@ -6,7 +6,7 @@ KDUMP_PATH="/var/crash" CORE_COLLECTOR="" DEFAULT_CORE_COLLECTOR="makedumpfile -l --message-level 1 -d 31" DMESG_COLLECTOR="/sbin/vmcore-dmesg" -DEFAULT_ACTION="systemctl reboot -f" +DEFAULT_ACTION="" DATEDIR=`date +%Y-%m-%d-%T` HOST_IP='127.0.0.1' DUMP_INSTRUCTION="" @@ -14,6 +14,7 @@ SSH_KEY_LOCATION="/root/.ssh/kdump_id_rsa" KDUMP_SCRIPT_DIR="/kdumpscripts" DD_BLKSIZE=512 FINAL_ACTION="systemctl reboot -f" +EARLYKDUMP_FINAL_ACTION="systemctl poweroff" KDUMP_CONF="/etc/kdump.conf" KDUMP_PRE="" KDUMP_POST="" @@ -155,11 +156,20 @@ kdump_emergency_shell() do_default_action() { - echo "Kdump: Executing default action $DEFAULT_ACTION" - eval $DEFAULT_ACTION + if [ $DEFAULT_ACTION == "" ]; then + echo "Kdump: default action string is null." + else + echo "Kdump: Executing default action $DEFAULT_ACTION" + eval $DEFAULT_ACTION + fi } do_final_action() { eval $FINAL_ACTION } + +do_earlykdump_final_action() +{ + eval $EARLYKDUMP_FINAL_ACTION +} diff --git a/kdump.sysconfig b/kdump.sysconfig index ffe1df8..b011c1c 100644 --- a/kdump.sysconfig +++ b/kdump.sysconfig @@ -17,7 +17,7 @@ KDUMP_COMMANDLINE="" # This variable lets us remove arguments from the current kdump commandline # as taken from either KDUMP_COMMANDLINE above, or from /proc/cmdline # NOTE: some arguments such as crashkernel will always be removed -KDUMP_COMMANDLINE_REMOVE="hugepages hugepagesz slub_debug quiet" +KDUMP_COMMANDLINE_REMOVE="hugepages hugepagesz slub_debug quiet rd.earlykdump" # This variable lets us append arguments to the current kdump commandline # after processed by KDUMP_COMMANDLINE_REMOVE diff --git a/kdump.sysconfig.aarch64 b/kdump.sysconfig.aarch64 index 0a6b14c..b8b8865 100644 --- a/kdump.sysconfig.aarch64 +++ b/kdump.sysconfig.aarch64 @@ -17,7 +17,7 @@ KDUMP_COMMANDLINE="" # This variable lets us remove arguments from the current kdump commandline # as taken from either KDUMP_COMMANDLINE above, or from /proc/cmdline # NOTE: some arguments such as crashkernel will always be removed -KDUMP_COMMANDLINE_REMOVE="hugepages hugepagesz slub_debug quiet" +KDUMP_COMMANDLINE_REMOVE="hugepages hugepagesz slub_debug quiet rd.earlykdump" # This variable lets us append arguments to the current kdump commandline # after processed by KDUMP_COMMANDLINE_REMOVE diff --git a/kdump.sysconfig.i386 b/kdump.sysconfig.i386 index 18c407e..b9a2835 100644 --- a/kdump.sysconfig.i386 +++ b/kdump.sysconfig.i386 @@ -17,7 +17,7 @@ KDUMP_COMMANDLINE="" # This variable lets us remove arguments from the current kdump commandline # as taken from either KDUMP_COMMANDLINE above, or from /proc/cmdline # NOTE: some arguments such as crashkernel will always be removed -KDUMP_COMMANDLINE_REMOVE="hugepages hugepagesz slub_debug quiet" +KDUMP_COMMANDLINE_REMOVE="hugepages hugepagesz slub_debug quiet rd.earlykdump" # This variable lets us append arguments to the current kdump commandline # after processed by KDUMP_COMMANDLINE_REMOVE diff --git a/kdump.sysconfig.ppc64 b/kdump.sysconfig.ppc64 index 55a01cc..e1ea9c6 100644 --- a/kdump.sysconfig.ppc64 +++ b/kdump.sysconfig.ppc64 @@ -17,7 +17,7 @@ KDUMP_COMMANDLINE="" # This variable lets us remove arguments from the current kdump commandline # as taken from either KDUMP_COMMANDLINE above, or from /proc/cmdline # NOTE: some arguments such as crashkernel will always be removed -KDUMP_COMMANDLINE_REMOVE="hugepages hugepagesz slub_debug quiet" +KDUMP_COMMANDLINE_REMOVE="hugepages hugepagesz slub_debug quiet rd.earlykdump" # This variable lets us append arguments to the current kdump commandline # after processed by KDUMP_COMMANDLINE_REMOVE diff --git a/kdump.sysconfig.ppc64le b/kdump.sysconfig.ppc64le index 55a01cc..e1ea9c6 100644 --- a/kdump.sysconfig.ppc64le +++ b/kdump.sysconfig.ppc64le @@ -17,7 +17,7 @@ KDUMP_COMMANDLINE="" # This variable lets us remove arguments from the current kdump commandline # as taken from either KDUMP_COMMANDLINE above, or from /proc/cmdline # NOTE: some arguments such as crashkernel will always be removed -KDUMP_COMMANDLINE_REMOVE="hugepages hugepagesz slub_debug quiet" +KDUMP_COMMANDLINE_REMOVE="hugepages hugepagesz slub_debug quiet rd.earlykdump" # This variable lets us append arguments to the current kdump commandline # after processed by KDUMP_COMMANDLINE_REMOVE diff --git a/kdump.sysconfig.s390x b/kdump.sysconfig.s390x index b3aec3c..4c5fc00 100644 --- a/kdump.sysconfig.s390x +++ b/kdump.sysconfig.s390x @@ -17,7 +17,7 @@ KDUMP_COMMANDLINE="" # This variable lets us remove arguments from the current kdump commandline # as taken from either KDUMP_COMMANDLINE above, or from /proc/cmdline # NOTE: some arguments such as crashkernel will always be removed -KDUMP_COMMANDLINE_REMOVE="hugepages hugepagesz slub_debug quiet" +KDUMP_COMMANDLINE_REMOVE="hugepages hugepagesz slub_debug quiet rd.earlykdump" # This variable lets us append arguments to the current kdump commandline # after processed by KDUMP_COMMANDLINE_REMOVE diff --git a/kdump.sysconfig.x86_64 b/kdump.sysconfig.x86_64 index f269d02..3cccbfc 100644 --- a/kdump.sysconfig.x86_64 +++ b/kdump.sysconfig.x86_64 @@ -17,7 +17,7 @@ KDUMP_COMMANDLINE="" # This variable lets us remove arguments from the current kdump commandline # as taken from either KDUMP_COMMANDLINE above, or from /proc/cmdline # NOTE: some arguments such as crashkernel will always be removed -KDUMP_COMMANDLINE_REMOVE="hugepages hugepagesz slub_debug quiet" +KDUMP_COMMANDLINE_REMOVE="hugepages hugepagesz slub_debug quiet rd.earlykdump" # This variable lets us append arguments to the current kdump commandline # after processed by KDUMP_COMMANDLINE_REMOVE diff --git a/kdumpctl b/kdumpctl index fe6af22..a8fd53d 100755 --- a/kdumpctl +++ b/kdumpctl @@ -26,7 +26,7 @@ image_time=0 standard_kexec_args="-p" # Some default values in case /etc/sysconfig/kdump doesn't include -KDUMP_COMMANDLINE_REMOVE="hugepages hugepagesz slub_debug" +KDUMP_COMMANDLINE_REMOVE="hugepages hugepagesz slub_debug rd.earlykdump" if [ -f /etc/sysconfig/kdump ]; then . /etc/sysconfig/kdump @@ -942,6 +942,11 @@ check_default_config() fi } +check_rd_earlykdump() +{ + egrep "rd.earlykdump" /proc/cmdline +} + start() { check_dump_feasibility @@ -969,7 +974,13 @@ start() check_current_status if [ $? == 0 ]; then echo "Kdump already running: [WARNING]" - return 0 + check_rd_earlykdump + #if earlykdump loaded, it will stop and start. + if [ $? -eq 0 ]; then + stop + else + return 0 + fi fi if check_ssh_config; then -- 2.17.1

8 29

[PATCH] mkdumprd: allow spaces after 'path' config phrase with network dump setting
by Kazuhito Hagio 08 Jan '19

08 Jan '19

Without this patch, when there are two or more spaces after 'path' configuration phrase with ssh or nfs setting, SAVE_PATH is set to '/var/crash' in mkdumprd, and in most cases kdump service fails to start by checking the /var/crash directory regardless of the path value. ssh kdump(a)192.168.122.1 path /kdump ^^ This behavior would be too sensitive and different from the other configurations. With this patch, mkdumprd allows such spaces. Signed-off-by: Kazuhito Hagio <k-hagio(a)ab.jp.nec.com> --- mkdumprd | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mkdumprd b/mkdumprd index a6f7fe8..aa0abfd 100644 --- a/mkdumprd +++ b/mkdumprd @@ -13,7 +13,7 @@ export IN_KDUMP=1 conf_file="/etc/kdump.conf" SSH_KEY_LOCATION="/root/.ssh/kdump_id_rsa" -SAVE_PATH=$(grep ^path $conf_file| cut -d' ' -f2) +SAVE_PATH=$(awk '/^path/ {print $2}' $conf_file) [ -z "$SAVE_PATH" ] && SAVE_PATH=$DEFAULT_PATH # strip the duplicated "/" SAVE_PATH=$(echo $SAVE_PATH | tr -s /) -- 2.18.0

2 2

[PATCH] dracut-module-setup: Don't build squashed image if required modules are missing
by Kairui Song 25 Dec '18

25 Dec '18

When someone is using a minimal kernel without squash module installed, including squash dracut module will either either fail to build or fail to boot the initramfs. As kdump always build the image for one single kernel, we can safely just use modprobe to check if a modules is already built in, or it exists and loadable for the kernel we are using for kdump image, and don't include the squash module if they are missing. Everything will still work just fine without squash module. We do the check in kdump dracut modules not in squash dracut module because kdump dracut module could leverage of the KDUMP_KERNELVER variable to know which kernel it should check against, squash dracut module may be used to build for a generic image. And we only check for the kernel module dependency, other binary dependencies are either well checked or well declared in dracut. Signed-off-by: Kairui Song <kasong(a)redhat.com> --- dracut-module-setup.sh | 18 +++++++++++++++++- 1 file changed, 17 insertions(+), 1 deletion(-) diff --git a/dracut-module-setup.sh b/dracut-module-setup.sh index 7499678..64d475f 100755 --- a/dracut-module-setup.sh +++ b/dracut-module-setup.sh @@ -18,7 +18,23 @@ check() { } depends() { - local _dep="base shutdown squash" + local _dep="base shutdown" + + is_squash_available() { + for kmodule in squashfs overlay loop; do + if [ -z "$KDUMP_KERNELVER" ]; then + modprobe --dry-run $kmodule &>/dev/null || return 1 + else + modprobe -S $KDUMP_KERNELVER --dry-run $kmodule &>/dev/null || return 1 + fi + done + } + + if is_squash_available; then + _dep="$_dep squash" + else + dwarning "Required modules to build a squashed kdump image is missing!" + fi if [ -n "$( find /sys/devices -name drm )" ] || [ -d /sys/module/hyperv_fb ]; then _dep="$_dep drm" -- 2.20.0.rc2

2 1

[PATCH] kdump-lib-initramfs.sh: using -force option when poweroff
by Pingfan Liu 10 Dec '18

10 Dec '18

If default action is poweroff, we can observe that the machine is rebooted, instead of poweroff. That is due to the following two race processes: systemctl poweroff systemctl reboot -f which is launched by kdump-error-handle.sh. Unfortunately, although both of them are executed in systemd block mode, but due to poweroff will tear down some internal things in systemd, there is no guarantee for the block mode. As we can see the msg "Failed to execute operation: Connection reset by peer", which is thrown by "systemctl reboot -f" poweroff and reboot share most of code, if one fails, then the other should also fails, so it is meaningless to use reboot as the backup of poweroff. Using "systemctl poweroff -f", the sdbus will teared down immediately, which prevent the following "systemctl reboot -f" from executing. Meanwhile, as man systemctl says: -f, --force When used with enable, overwrite any existing conflicting symlinks. When used with halt, poweroff, reboot or kexec, execute the selected operation without shutting down all units. However, all processes will be killed forcibly and all file systems are unmounted or remounted read-only. Hence, replacing the 'poweroff' with 'systemctl poweroff -f' Signed-off-by: Pingfan Liu <piliu(a)redhat.com> --- kdump-lib-initramfs.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/kdump-lib-initramfs.sh b/kdump-lib-initramfs.sh index c7f874f..f5155a4 100755 --- a/kdump-lib-initramfs.sh +++ b/kdump-lib-initramfs.sh @@ -63,7 +63,7 @@ get_kdump_confs() DEFAULT_ACTION="halt" ;; poweroff) - DEFAULT_ACTION="poweroff" + DEFAULT_ACTION="systemctl poweroff -f" ;; dump_to_rootfs) DEFAULT_ACTION="dump_to_rootfs" -- 2.7.4

2 1

[PATCH v2] Make udev reload rules quiet during bootup
by Kairui Song 06 Dec '18

06 Dec '18

In commit 1c97aee and commit 227c185 udev rules was rewritten to use systemd-run to run in a non-blocking mode. The problem is that it's a bit noise, especially on machine bootup, systemd will always generate extra logs for service start, you might see your journal full of lines like these if you have many CPUs (each CPU generates a udev event on boot): ... Nov 22 22:23:05 localhost systemd[1]: Started /usr/lib/udev/kdump-udev-throttler. Nov 22 22:23:05 localhost systemd[1]: Started /usr/lib/udev/kdump-udev-throttler. Nov 22 22:23:05 localhost systemd[1]: Started /usr/lib/udev/kdump-udev-throttler. Nov 22 22:23:05 localhost systemd[1]: Started /usr/lib/udev/kdump-udev-throttler. ... While system is still booting up, kdump service is not started yet, so systemd-run calls will end up doing nothing, the throttler being called by systemd-run will just exit if kdump is not loaded. This patch avoid systemd-run from being called at first place if kdump service is not running by checking kdump.service status in udev rule, so there won't be unnecessary logs. Also remove the kdump service checking logic in kdump-udev-throttler as udev is the only expected callee of this script, if it's not being called at first place when kdump service is running, this checking will be redundant. And even if any user called this script manually, it will still work well as this script will call 'kdumpctl reload', it reload the kdump resource only if kdump is loaded already. Signed-off-by: Kairui Song <kasong(a)redhat.com> --- Update from V1: - Embed the conditional invocation logic in udev rule as suggested by Dave Young and Bhupesh Sharma - Add some comment in udev rule about why the conditional invocation is needed. - Remove the service status detect code in kdump-udev-throttle as it's no longer needed. 98-kexec.rules | 6 +++++- kdump-udev-throttler | 5 ----- 2 files changed, 5 insertions(+), 6 deletions(-) diff --git a/98-kexec.rules b/98-kexec.rules index 2f88c77..b73b701 100644 --- a/98-kexec.rules +++ b/98-kexec.rules @@ -7,6 +7,10 @@ GOTO="kdump_reload_end" LABEL="kdump_reload" -RUN+="/usr/bin/systemd-run --no-block /usr/lib/udev/kdump-udev-throttler" +# If kdump is not loaded, calling kdump-udev-throttle will end up +# doing nothing, but systemd-run will always generate extra logs for +# each call, so trigger the kdump-udev-throttler only if kdump +# service is active to avoid unnecessary logs +RUN+="/bin/sh -c '/usr/bin/systemctl is-active kdump.service || exit 0; /usr/bin/systemd-run --quiet --no-block /usr/lib/udev/kdump-udev-throttler'" LABEL="kdump_reload_end" diff --git a/kdump-udev-throttler b/kdump-udev-throttler index 6cbb99a..cd77a31 100755 --- a/kdump-udev-throttler +++ b/kdump-udev-throttler @@ -13,12 +13,7 @@ # In this way, we can make sure kdump service is restarted immediately # and for exactly once after udev events are settled. - throttle_lock="/var/lock/kdump-udev-throttle" -interval=2 - -# Don't reload kdump service if kdump service is not started by systemd -systemctl is-active kdump.service &>/dev/null || exit 0 exec 9>$throttle_lock if [ $? -ne 0 ]; then -- 2.19.1

2 2

[PATCH] Make udev reload rules quiet during bootup
by Kairui Song 05 Dec '18

05 Dec '18

In commit 1c97aee and commit 227c185 udev rules was rewritten to use systemd-run to run in a non-blocking mode. The problem is that it's a bit noise, especially on machine bootup, systemd will always generate extra logs for service start, you might see your journal full of lines like these if you have many CPUs (each CPU generates a udev event on boot): ... Nov 22 22:23:05 localhost systemd[1]: Started /usr/lib/udev/kdump-udev-throttler. Nov 22 22:23:05 localhost systemd[1]: Started /usr/lib/udev/kdump-udev-throttler. Nov 22 22:23:05 localhost systemd[1]: Started /usr/lib/udev/kdump-udev-throttler. Nov 22 22:23:05 localhost systemd[1]: Started /usr/lib/udev/kdump-udev-throttler. ... While system is still booting up, kdump service is not started yet, so systemd-run calls will end up doing nothing, the throttler being called by systemd-run will just exit if kdump is not loaded. This patch avoid systemd-run from being called at first place if kdump service is not running, so there won't be unnecessary logs. As udev rules don't support shell expressions, this patch reuse the kdump-udev-throttler script, adding a --no-block option. kdump-udev-throttler will call it self with systemd-run to run in non blocking mode if --no-block is given, and exit before doing anything if kdump is not started. Udev just call "kdump-udev-throttler --no-block" directly. Signed-off-by: Kairui Song <kasong(a)redhat.com> --- 98-kexec.rules | 2 +- kdump-udev-throttler | 13 ++++++++++--- 2 files changed, 11 insertions(+), 4 deletions(-) diff --git a/98-kexec.rules b/98-kexec.rules index 2f88c77..eca909e 100644 --- a/98-kexec.rules +++ b/98-kexec.rules @@ -7,6 +7,6 @@ GOTO="kdump_reload_end" LABEL="kdump_reload" -RUN+="/usr/bin/systemd-run --no-block /usr/lib/udev/kdump-udev-throttler" +RUN+="/usr/lib/udev/kdump-udev-throttler --no-block" LABEL="kdump_reload_end" diff --git a/kdump-udev-throttler b/kdump-udev-throttler index 6cbb99a..6899379 100755 --- a/kdump-udev-throttler +++ b/kdump-udev-throttler @@ -13,13 +13,20 @@ # In this way, we can make sure kdump service is restarted immediately # and for exactly once after udev events are settled. - throttle_lock="/var/lock/kdump-udev-throttle" -interval=2 -# Don't reload kdump service if kdump service is not started by systemd +# Don't do anything if kdump service is not started by systemd systemctl is-active kdump.service &>/dev/null || exit 0 +if [[ $1 == '--no-block' ]]; then + systemd-run --no-block --quiet $0 + if [[ $? -ne 0 ]]; then + echo "kdump-udev-throttler: Failed to call systemd-run" + exit 1 + fi + exit 0 +fi + exec 9>$throttle_lock if [ $? -ne 0 ]; then echo "Failed to create the lock file! Fallback to non-throttled kdump service restart" -- 2.19.1

4 5

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

kexec December 2018