kexec February 2021

kexec@lists.fedoraproject.org

8 participants
17 discussions

[PATCH] mkdumprd: allow spaces after 'path' config phrase when network dump
by Kazuhito Hagio 26 May '22

26 May '22

Without this patch, when there are two or more spaces after 'path' configuration phrase with ssh or nfs setting, SAVE_PATH is set to '/var/crash' in mkdumprd, and in most cases kdump service fails to start. ssh kdump(a)192.168.122.1 path /kdump ^^ This behavior would be too sensitive and different from the other configurations. With this patch, mkdumprd allows such spaces. Signed-off-by: Kazuhito Hagio <k-hagio(a)ab.jp.nec.com> --- mkdumprd | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mkdumprd b/mkdumprd index a6f7fe8..aa0abfd 100644 --- a/mkdumprd +++ b/mkdumprd @@ -13,7 +13,7 @@ export IN_KDUMP=1 conf_file="/etc/kdump.conf" SSH_KEY_LOCATION="/root/.ssh/kdump_id_rsa" -SAVE_PATH=$(grep ^path $conf_file| cut -d' ' -f2) +SAVE_PATH=$(awk '/^path/ {print $2}' $conf_file) [ -z "$SAVE_PATH" ] && SAVE_PATH=$DEFAULT_PATH # strip the duplicated "/" SAVE_PATH=$(echo $SAVE_PATH | tr -s /) -- 2.18.0

4 3

[PATCH] kdumpctl: Write to `/var/lib/kdump` if $KDUMP_BOOTDIR not writable
by Kelvin Fan 06 Apr '21

06 Apr '21

The /boot directory on some operating systems might be read-only. If we cannot write to $KDUMP_BOOTDIR when generating the kdump initrd, attempt to place the generated initrd at `/var/lib/kdump` instead. Signed-off by: Kelvin Fan <kfan(a)redhat.com> --- kdumpctl | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/kdumpctl b/kdumpctl index 24f5cf7..2107e3e 100755 --- a/kdumpctl +++ b/kdumpctl @@ -152,8 +152,15 @@ rebuild_kdump_initrd() rebuild_initrd() { if [[ ! -w "$KDUMP_BOOTDIR" ]];then - derror "$KDUMP_BOOTDIR does not have write permission. Can not rebuild $TARGET_INITRD" - return 1 + VAR_TARGET_INITRD_DIR="/var/lib/kdump" + mkdir -p "$VAR_TARGET_INITRD_DIR" + TARGET_INITRD_BASE="$(basename $TARGET_INITRD)" + TARGET_INITRD="$VAR_TARGET_INITRD_DIR/$TARGET_INITRD_BASE" + dwarn "$KDUMP_BOOTDIR does not have write permission. Rebuilding initrd at $TARGET_INITRD" + if [[ ! -w "$VAR_TARGET_INITRD_DIR" ]];then + derror "$VAR_TARGET_INITRD_DIR does not have write permission. Cannot rebuild $TARGET_INITRD" + return 1 + fi fi if [ $DEFAULT_DUMP_MODE == "fadump" ]; then @@ -656,6 +663,9 @@ load_kdump() KEXEC_ARGS="$KEXEC_ARGS -s" fi + # Relabel the target initrd for SELinux. + chcon -t boot_t $TARGET_INITRD + ddebug "$KEXEC $KEXEC_ARGS $standard_kexec_args --command-line=$KDUMP_COMMANDLINE --initrd=$TARGET_INITRD $KDUMP_KERNEL" # The '12' represents an intermediate temporary file descriptor -- 2.29.2

2 2

[PATCH] mkdumprd: prompt the user to install nfs-utils when mounting NFS fs failed
by Coiby Xu 24 Mar '21

24 Mar '21

When nfs-utils is not installed, mounting as NFS fs would fail. Currently, the error message is not user-friendly, mount: /tmp/mkdumprd.HyPGpS/target: bad option; for several filesystems (e.g. nfs, cifs) you might need a /sbin/mount.<type> helper program. kdump: Failed to mount on xxx for kdump preflight check. kdump: mkdumprd: failed to make kdump initrd Prompt the user to install nfs-utilsa in the error message, kdump: Failed to mount on xxx for kdump preflight check. Please make sure nfs-utils has been installed. Signed-off-by: Coiby Xu <coxu(a)redhat.com> --- mkdumprd | 24 ++++++++++++++++++++++-- 1 file changed, 22 insertions(+), 2 deletions(-) diff --git a/mkdumprd b/mkdumprd index c34b79c..50ebdc8 100644 --- a/mkdumprd +++ b/mkdumprd @@ -185,6 +185,26 @@ check_save_path_fs() fi } +mount_failure() +{ + local _target=$1 + local _mnt=$2 + local _fstype=$3 + local msg="Failed to mount $_target" + + if [ -n "$_mnt" ]; then + msg="$msg on $_mnt" + fi + + msg="$msg for kdump preflight check." + + if [[ $_fstype = "nfs" ]]; then + msg="$msg Please make sure nfs-utils has been installed." + fi + + perror_exit "$msg" +} + check_user_configured_target() { local _target=$1 _cfg_fs_type=$2 _mounted @@ -210,7 +230,7 @@ check_user_configured_target() if ! is_mounted "$_mnt"; then if [[ $_opt = *",noauto"* ]]; then mount $_mnt - [ $? -ne 0 ] && perror_exit "Failed to mount $_target on $_mnt for kdump preflight check." + [ $? -ne 0 ] && mount_failure "$_target" "$_mnt" "$_fstype" _mounted=$_mnt else perror_exit "Dump target \"$_target\" is neither mounted nor configured as \"noauto\"" @@ -220,7 +240,7 @@ check_user_configured_target() _mnt=$MKDUMPRD_TMPMNT mkdir -p $_mnt mount $_target $_mnt -t $_fstype -o defaults - [ $? -ne 0 ] && perror_exit "Failed to mount $_target for kdump preflight check." + [ $? -ne 0 ] && mount_failure "$_target" "" "$_fstype" _mounted=$_mnt fi -- 2.30.0

2 1

[PATCH 0/3] Preparing for running kexec-tools from dracut
by Coiby Xu 23 Mar '21

23 Mar '21

kexec-tools heavily depends on dracut and occasionally breaks because the change of dracut. I'm now adding kdump test [1] to dracut which will clone kexec-tools' rawhide branch and run kexec-tools' basic functionality tests. This patch set prepares kexec-tools for running the tests by dracut. [1] https://github.com/coiby/dracut/tree/kexec_tools_tests Coiby Xu (3): selftest: Fix bug of collecting test RPMs from argument selftest: add DRACUT_RPMs so dracut RPMs can be installed onto the image to run the tests selftest: replace qemu-kvm with one based on dracut's run-qemu tests/Makefile | 2 +- .../scripts/build-scripts/test-base-image.sh | 2 +- tests/scripts/run-qemu | 23 +++++++++++++++++++ tests/scripts/test-lib.sh | 2 +- 4 files changed, 26 insertions(+), 3 deletions(-) create mode 100755 tests/scripts/run-qemu -- 2.30.1

2 6

[PATCH v2 0/2] Fix failure to dump vmcore to nfs target for s390x RHEL9
by Coiby Xu 16 Mar '21

16 Mar '21

On s390x RHEL9, vmcore can't be dumped to a nfs target for the following reasons, 1. ifcfg is retired and we can't get znet config info from /etc/sysconfig/network-scripts any more. 2. ipcalc is not installed by default, thus static route info can't be generated correctly. 3. rpc-statd.service can't be started because of a bug in dracut These two patches address the first two issues and the third issue has been addressed by Kairui [1]. [1] https://github.com/dracutdevs/dracut/pull/1063 Changes since v1: - use regex to extract value from nmcli output and avoid using array to simplify code [Kairui] - take care of the case that nmcli could output "802-3-ethernet.s390-nettype: --" Coiby Xu (2): read znet config from nmcli directly add dependency on ipcalc dracut-module-setup.sh | 41 +++++++++++++++++++++++++++++++++++++---- kexec-tools.spec | 1 + 2 files changed, 38 insertions(+), 4 deletions(-) -- 2.30.1

2 7

[PATCH] mkdumprd: support using an fs overlay for building initrd
by Kairui Song 10 Mar '21

10 Mar '21

Currently kdump simply uses dracut and pulls all the required binaries from the host directly. This keep things simple as kdump is using same binaries and libs from the system of fisrt kernel. But some binaries, like systemd, is growing too large, which increase the memory usage by a lot, and most of its features are never used in kdump. So it's more reasonable to use a rebuilt-minimized version. So for such binaries, introduce a "build root" overlay here. Before calling dracut to build initramfs, setup an overlay to let the build root override current rootfs, so dracut can just work as usual, but the binaries used are from the "build root". All binaries/libs that need to be rebuild for smaller size can be put in the build root. --- kexec-tools.spec | 1 + mkdumprd | 20 +++++++++++++++++++- 2 files changed, 20 insertions(+), 1 deletion(-) diff --git a/kexec-tools.spec b/kexec-tools.spec index 91bc0f3..1a06b62 100644 --- a/kexec-tools.spec +++ b/kexec-tools.spec @@ -171,6 +171,7 @@ mkdir -p $RPM_BUILD_ROOT%{_unitdir} mkdir -p -m755 $RPM_BUILD_ROOT%{_bindir} mkdir -p -m755 $RPM_BUILD_ROOT%{_libdir} mkdir -p -m755 $RPM_BUILD_ROOT%{_prefix}/lib/kdump +mkdir -p -m755 $RPM_BUILD_ROOT%{_prefix}/lib/kdump/buildroot install -m 755 %{SOURCE1} $RPM_BUILD_ROOT%{_bindir}/kdumpctl install -m 755 build/sbin/kexec $RPM_BUILD_ROOT/usr/sbin/kexec diff --git a/mkdumprd b/mkdumprd index c34b79c..5f8d2da 100644 --- a/mkdumprd +++ b/mkdumprd @@ -23,6 +23,7 @@ if [ $? -ne 0 ]; then exit 1 fi +KDUMP_BUILDROOT="/usr/lib/kdump/buildroot" conf_file="/etc/kdump.conf" SSH_KEY_LOCATION="/root/.ssh/kdump_id_rsa" SAVE_PATH=$(get_save_path) @@ -33,12 +34,14 @@ dracut_args="--add kdumpbase --quiet --hostonly --hostonly-cmdline --hostonly-i1 readonly MKDUMPRD_TMPDIR="$(mktemp -d -t mkdumprd.XXXXXX)" [ -d "$MKDUMPRD_TMPDIR" ] || perror_exit "dracut: mktemp -p -d -t dracut.XXXXXX failed." +readonly MKDUMPRD_TMPROOT="$MKDUMPRD_TMPDIR/root" readonly MKDUMPRD_TMPMNT="$MKDUMPRD_TMPDIR/target" trap ' ret=$?; is_mounted $MKDUMPRD_TMPMNT && umount -f $MKDUMPRD_TMPMNT; [[ -d $MKDUMPRD_TMPDIR ]] && rm --one-file-system -rf -- "$MKDUMPRD_TMPDIR"; + [[ -d $MKDUMPRD_TMPROOT ]] && rm --one-file-system -rf -- "$MKDUMPRD_TMPROOT"; exit $ret; ' EXIT @@ -460,7 +463,22 @@ if ! is_fadump_capable; then add_dracut_arg "--no-hostonly-default-device" fi -echo "$dracut_args $@" | xargs dracut +if [ -n "$(ls -A1 $KDUMP_BUILDROOT | grep -E "etc|usr")" ]; then + # Run in a new namespace, so overlay on root is safe + # everything is automatically cleaned up upon exit + export KDUMP_BUILDROOT + export MKDUMPRD_TMPROOT + export dracut_args="$dracut_args $@" + + cp -r /usr/lib/kdump/buildroot $MKDUMPRD_TMPROOT + unshare --mount bash -c ' + for _d in etc usr; do + [ -d "$MKDUMPRD_TMPROOT/$_d" ] && mount overlay -t overlay -o ro,lowerdir=$MKDUMPRD_TMPROOT/$_d:/$_d /$_d; + done + echo "$dracut_args $@" | xargs -t dracut;' +else + echo "$dracut_args $@" | xargs -t dracut +fi _rc=$? sync -- 2.29.2

3 6

[PATCH] Stop reloading kdump service on CPU hotplug event for FADump
by Sourabh Jain 06 Mar '21

06 Mar '21

As FADump does not require an explicit elfcorehdr update whenever there is CPU hotplug event so let's stop kdump service reload for FADump when CPU hotplug event is triggered. A new label is added to handle CPU and memory hotplug events separately. The updated CPU hotplug event handler make sure that kdump service should not be reloaded when FADump is configured. Signed-off-by: Sourabh Jain <sourabhjain(a)linux.ibm.com> --- 98-kexec.rules.ppc64 | 17 ++++++++++++----- 1 file changed, 12 insertions(+), 5 deletions(-) diff --git a/98-kexec.rules.ppc64 b/98-kexec.rules.ppc64 index 1a91220..a1c00a9 100644 --- a/98-kexec.rules.ppc64 +++ b/98-kexec.rules.ppc64 @@ -1,15 +1,22 @@ -SUBSYSTEM=="cpu", ACTION=="online", GOTO="kdump_reload" -SUBSYSTEM=="memory", ACTION=="online", GOTO="kdump_reload" -SUBSYSTEM=="memory", ACTION=="offline", GOTO="kdump_reload" +SUBSYSTEM=="cpu", ACTION=="online", GOTO="kdump_reload_cpu" +SUBSYSTEM=="memory", ACTION=="online", GOTO="kdump_reload_mem" +SUBSYSTEM=="memory", ACTION=="offline", GOTO="kdump_reload_mem" GOTO="kdump_reload_end" -LABEL="kdump_reload" - # If kdump is not loaded, calling kdump-udev-throttle will end up # doing nothing, but systemd-run will always generate extra logs for # each call, so trigger the kdump-udev-throttler only if kdump # service is active to avoid unnecessary logs + +LABEL="kdump_reload_mem" + RUN+="/bin/sh -c '/usr/bin/systemctl is-active kdump.service || exit 0; /usr/bin/systemd-run --quiet --no-block /usr/lib/udev/kdump-udev-throttler'" +GOTO="kdump_reload_end" + +LABEL="kdump_reload_cpu" + +RUN+="/bin/sh -c '/usr/bin/systemctl is-active kdump.service || exit 0; ! test -f /sys/kernel/fadump_enabled || cat /sys/kernel/fadump_enabled | grep 0 || exit 0; /usr/bin/systemd-run --quiet --no-block /usr/lib/udev/kdump-udev-throttler'" + LABEL="kdump_reload_end" -- 2.26.2

3 4

[PATCH] fadump: improve fadump-howto.txt about remote dump target setup
by Hari Bathini 23 Feb '21

23 Feb '21

While fadump-howto.txt talks about what happens to network interface name on setting up a remote dump target in FADump mode, it doesn't explicitly specify the negative consequences of it. Make it explicit and provide a recommendation to overcome the same. Signed-off-by: Hari Bathini <hbathini(a)linux.ibm.com> --- fadump-howto.txt | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-) diff --git a/fadump-howto.txt b/fadump-howto.txt index 5360f3d..433e9a6 100644 --- a/fadump-howto.txt +++ b/fadump-howto.txt @@ -224,12 +224,16 @@ Things to remember: dracut_args --add "network" 4) If FADump is configured to capture vmcore to a remote dump target using SSH - or NFS protocol, the network interface is renamed to kdump-<interface-name> - if <interface-name> is generic, for example, *eth#, or net#. This problem - occurs because the vmcore capture scripts in the initial RAM disk (initrd) - add the kdump- prefix to the network interface name to secure persistent - naming. As the same initrd is used for production kernel boot, the interface - name is changed for the production kernel too. + or NFS protocol, the corresponding network interface '<interface-name>' is + renamed to 'kdump-<interface-name>', if it is generic (like *eth# or net#). + It happens because vmcore capture scripts in the initial RAM disk (initrd) + add the 'kdump-' prefix to the network interface name to secure persistent + naming. And as capture kernel and production kernel use the same initrd in + case of FADump, the interface name is changed for the production kernel too. + This is likely to impact network configuration setup for production kernel. + So, it is recommended to use a non-generic name for a network interface, + before setting up FADump to capture vmcore to a remote dump target based on + that network interface, to avoid running into network configuration issues. Dump Triggering methods:

2 1

[PATCH] kdumpctl: enable secure boot on ppc64le LPARs
by Pingfan Liu 22 Feb '21

22 Feb '21

On ppc64le LPAR, secure-boot is a little different from bare metal, Where host secure boot: /ibm,secure-boot/os-secureboot-enforcing DT property exists while guest secure boot: /ibm,secure-boot >= 2 Make kexec-tools adapt to LPAR Signed-off-by: Pingfan Liu <piliu(a)redhat.com> --- kdump-lib.sh | 9 +++++++-- kdumpctl | 32 ++++++++++++++++++++++++++++++++ 2 files changed, 39 insertions(+), 2 deletions(-) diff --git a/kdump-lib.sh b/kdump-lib.sh index a4bab39..3d57f98 100755 --- a/kdump-lib.sh +++ b/kdump-lib.sh @@ -621,11 +621,16 @@ is_secure_boot_enforced() local secure_boot_file setup_mode_file local secure_boot_byte setup_mode_byte - # On powerpc, os-secureboot-enforcing DT property indicates whether secureboot - # is enforced. Return success, if it is found. + # On powerpc, secure boot is enforced if: + # host secure boot: /ibm,secure-boot/os-secureboot-enforcing DT property exists + # guest secure boot: /ibm,secure-boot >= 2 if [ -f /proc/device-tree/ibm,secureboot/os-secureboot-enforcing ]; then return 0 fi + if [ -f /proc/device-tree/ibm,secure-boot ] && \ + [ $(lsprop /proc/device-tree/ibm,secure-boot | tail -1) -ge 2 ]; then + return 0 + fi # Detect secure boot on x86 and arm64 secure_boot_file=$(find /sys/firmware/efi/efivars -name SecureBoot-* 2>/dev/null) diff --git a/kdumpctl b/kdumpctl index 24f5cf7..c3311ad 100755 --- a/kdumpctl +++ b/kdumpctl @@ -638,6 +638,35 @@ check_rebuild() return $? } +# On ppc64le LPARs, the keys trusted by firmware do not end up in +# .builtin_trusted_keys. So instead, add the key to the .ima keyring +function load_kdump_kernel_key() +{ + # this is only called inside is_secure_boot_enforced, + # no need to retest + + # this is only required if DT /ibm,secure-boot is a file. + # if it is a dir, we are on OpenPower and don't need this. + if ! [ -f /proc/device-tree/ibm,secure-boot ]; then + return + fi + + KDUMP_KEY_ID=$(cat /usr/share/doc/kernel-keys/$KDUMP_KERNELVER/kernel-signing-ppc.cer | + keyctl padd asymmetric kernelkey-$RANDOM %:.ima) +} + +# remove a previously loaded key. There's no real security implication +# to leaving it around, we choose to do this because it makes it easier +# to be idempotent and so as to reduce the potential for confusion. +function remove_kdump_kernel_key() +{ + if [ -z "$KDUMP_KEY_ID" ]; then + return + fi + + keyctl unlink $KDUMP_KEY_ID %:.ima +} + # Load the kdump kernel specified in /etc/sysconfig/kdump # If none is specified, try to load a kdump kernel with the same version # as the currently running kernel. @@ -654,6 +683,7 @@ load_kdump() if is_secure_boot_enforced; then dinfo "Secure Boot is enabled. Using kexec file based syscall." KEXEC_ARGS="$KEXEC_ARGS -s" + load_kdump_kernel_key fi ddebug "$KEXEC $KEXEC_ARGS $standard_kexec_args --command-line=$KDUMP_COMMANDLINE --initrd=$TARGET_INITRD $KDUMP_KERNEL" @@ -675,6 +705,8 @@ load_kdump() set +x exec 2>&12 12>&- + remove_kdump_kernel_key + if [ $ret == 0 ]; then dinfo "kexec: loaded kdump kernel" return 0 -- 2.29.2

2 1

[PATCHv2] kdump-lib.sh: introduce functions to return recommened mem size
by Pingfan Liu 16 Feb '21

16 Feb '21

There is requirement to decide the recommended memory size for the current system. Ant the algorithm is based on /proc/iomem, so it can align with the algorithm used by reserve_crashkernel() in kernel. Signed-off-by: Pingfan Liu <piliu(a)redhat.com> --- kdump-lib.sh | 69 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 69 insertions(+) diff --git a/kdump-lib.sh b/kdump-lib.sh index d2801da..a4bab39 100755 --- a/kdump-lib.sh +++ b/kdump-lib.sh @@ -823,3 +823,72 @@ prepare_cmdline() echo ${cmdline} } + +#get system memory size in the unit of GB +get_system_size() +{ + result=$(cat /proc/iomem | grep "System RAM" | awk -F ":" '{ print $1 }' | tr [:lower:] [:upper:] | paste -sd+) + result="+$result" + # replace '-' with '+0x' and '+' with '-0x' + sum=$( echo $result | sed -e 's/-/K0x/g' | sed -e 's/+/-0x/g' | sed -e 's/K/+/g' ) + size=$(printf "%d\n" $(($sum))) + let size=$size/1024/1024/1024 + + echo $size +} + +get_recommend_size() +{ + local mem_size=$1 + local _ck_cmdline=$2 + local OLDIFS="$IFS" + + last_sz="" + last_unit="" + + IFS=',' + for i in $_ck_cmdline + do + end=$(echo $i | awk -F "-" '{ print $2 }' | awk -F ":" '{ print $1 }') + recommend=$(echo $i | awk -F "-" '{ print $2 }' | awk -F ":" '{ print $2 }') + size=${end: : -1} + unit=${end: -1} + if [ $unit == 'T' ]; then + let size=$size*1024 + fi + if [ $mem_size -lt $size ]; then + echo $recommend + IFS="$OLDIFS" + return + fi + done + IFS="$OLDIFS" +} + +# return recommended size based on current system RAM size +kdump_get_arch_recommend_size() +{ + if ! [[ -r "/proc/iomem" ]] ; then + echo "Error, can not access /proc/iomem." + return 1 + fi + arch=$(lscpu | grep Architecture | awk -F ":" '{ print $2 }' | tr [:lower:] [:upper:]) + + if [ $arch == "X86_64" ] || [ $arch == "S390" ]; then + ck_cmdline="1G-4G:160M,4G-64G:192M,64G-1T:256M,1T-:512M" + elif [ $arch == "ARM64" ]; then + ck_cmdline="2G-:448M" + elif [ $arch == "PPC64LE" ]; then + if is_fadump_capable; then + ck_cmdline="4G-16G:768M,16G-64G:1G,64G-128G:2G,128G-1T:4G,1T-2T:6G,2T-4T:12G,4T-8T:20G,8T-16T:36G,16T-32T:64G,32T-64T:128G,64T-:180G" + else + ck_cmdline="2G-4G:384M,4G-16G:512M,16G-64G:1G,64G-128G:2G,128G-:4G" + fi + fi + + ck_cmdline=$( echo $ck_cmdline | sed -e 's/-:/-102400T:/g' ) + sys_mem=$(get_system_size) + result=$( get_recommend_size $sys_mem "$ck_cmdline" ) + echo $result + return 0 +} -- 2.29.2

3 2

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

kexec February 2021