kexec August 2020

kexec@lists.fedoraproject.org

6 participants
9 discussions

[PATCH] mkdumprd: allow spaces after 'path' config phrase when network dump
by Kazuhito Hagio 26 May '22

26 May '22

Without this patch, when there are two or more spaces after 'path' configuration phrase with ssh or nfs setting, SAVE_PATH is set to '/var/crash' in mkdumprd, and in most cases kdump service fails to start. ssh kdump(a)192.168.122.1 path /kdump ^^ This behavior would be too sensitive and different from the other configurations. With this patch, mkdumprd allows such spaces. Signed-off-by: Kazuhito Hagio <k-hagio(a)ab.jp.nec.com> --- mkdumprd | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mkdumprd b/mkdumprd index a6f7fe8..aa0abfd 100644 --- a/mkdumprd +++ b/mkdumprd @@ -13,7 +13,7 @@ export IN_KDUMP=1 conf_file="/etc/kdump.conf" SSH_KEY_LOCATION="/root/.ssh/kdump_id_rsa" -SAVE_PATH=$(grep ^path $conf_file| cut -d' ' -f2) +SAVE_PATH=$(awk '/^path/ {print $2}' $conf_file) [ -z "$SAVE_PATH" ] && SAVE_PATH=$DEFAULT_PATH # strip the duplicated "/" SAVE_PATH=$(echo $SAVE_PATH | tr -s /) -- 2.18.0

4 3

[PATCH] Rework check_config and warn on any duplicated option
by Kairui Song 27 Oct '20

27 Oct '20

Instead of read and parse the kdump.conf multiple times, only read once and use a single loop to handle the error check, which is faster. Also check for any duplicated config otion, and error out if there are duplicated ones. Now it checks for following errors, most are unchanged from before: - Any duplicated config options. (New added) - Deprecated/Invalid kdump config option. - Duplicated kdump target, will have a different error message of other duplicated config options. - Duplicated --mount options in dracut_args. - Empty config values. All kdump configs should be in "<config_opt> <config_value>" format. - Check If raw target is used in fadump mode. And removed detect of lines start with space, it will not break kdump anyway. The performance is measurable better than before for the check_config function. Signed-off-by: Kairui Song <kasong(a)redhat.com> --- kdumpctl | 72 ++++++++++++++++++++++++++------------------------------ 1 file changed, 34 insertions(+), 38 deletions(-) diff --git a/kdumpctl b/kdumpctl index 2248da4..f191d9a 100755 --- a/kdumpctl +++ b/kdumpctl @@ -217,47 +217,34 @@ restore_default_initrd() check_config() { - local nr - - nr=$(awk 'BEGIN{cnt=0} /^raw|^ssh[[:blank:]]|^nfs|^ext[234]|^xfs|^btrfs|^minix|^dracut_args .*\-\-mount/{cnt++} END{print cnt}' $KDUMP_CONFIG_FILE) - [ $nr -gt 1 ] && { - echo "More than one dump targets specified." - return 1 - } - - # Check if path option is set more than once. - nr=$(awk 'BEGIN{cnt=0} /^path /{cnt++} END{print cnt}' $KDUMP_CONFIG_FILE) - [ $nr -gt 1 ] && { - echo "Mutiple paths specifed in $KDUMP_CONFIG_FILE" - return 1 - } - - nr=$(grep "^dracut_args .*\-\-mount" $KDUMP_CONFIG_FILE | grep -o "\-\-mount" | wc -l) - [ $nr -gt 1 ] && { - echo "Multiple mount targets specified in one \"dracut_args\"." - return 1 - } - - # Check if we have any leading spaces (or tabs) before the - # variable name in the kdump conf file - if grep -E -q '^[[:blank:]]+[a-z]' $KDUMP_CONFIG_FILE; then - echo "No whitespaces are allowed before a kdump option name in $KDUMP_CONFIG_FILE" - return 1 - fi - + local -A _opt_rec while read config_opt config_val; do + if [ -z "$config_val" ]; then + echo "Invalid kdump config value for option $config_opt" + return 1 + fi + case "$config_opt" in - \#* | "") + dracut_args) + if [[ $config_val == *--mount* ]]; then + if [ $(echo $config_val | grep -o "\-\-mount" | wc -l) -ne 1 ]; then + echo "Multiple mount targets specified in one \"dracut_args\"." + return 1 + fi + config_opt=_target + fi ;; - raw|ext2|ext3|ext4|minix|btrfs|xfs|nfs|ssh|sshkey|path|core_collector|kdump_post|kdump_pre|extra_bins|extra_modules|failure_action|default|final_action|force_rebuild|force_no_rebuild|dracut_args|fence_kdump_args|fence_kdump_nodes) - # remove inline comments after the end of a directive. - [ -z "$config_val" ] && { - echo "Invalid kdump config value for option $config_opt." - return 1; - } - if [ -d "/proc/device-tree/ibm,opal/dump" ] && [ "$config_opt" == "raw" ]; then + raw) + if [ -d "/proc/device-tree/ibm,opal/dump" ]; then echo "WARNING: Won't capture opalcore when 'raw' dump target is used." + return 1 fi + config_opt=_target + ;; + ext[234]|minix|btrfs|xfs|nfs|ssh) + config_opt=_target + ;; + sshkey|path|core_collector|kdump_post|kdump_pre|extra_bins|extra_modules|failure_action|default|final_action|force_rebuild|force_no_rebuild|fence_kdump_args|fence_kdump_nodes) ;; net|options|link_delay|disk_timeout|debug_mem_level|blacklist) echo "Deprecated kdump config option: $config_opt. Refer to kdump.conf manpage for alternatives." @@ -265,14 +252,23 @@ check_config() ;; *) echo "Invalid kdump config option $config_opt" - return 1; + return 1 ;; esac + + if [ -n "${_opt_rec[$config_opt]}" ]; then + if [ $config_opt == _target ]; then + echo "More than one dump targets specified" + else + echo "Duplicated kdump config value of option $config_opt" + fi + return 1 + fi + _opt_rec[$config_opt]="$config_val" done <<< "$(read_strip_comments $KDUMP_CONFIG_FILE)" check_failure_action_config || return 1 check_final_action_config || return 1 - check_fence_kdump_config || return 1 return 0 -- 2.26.2

2 1

[PATCH 0/2] Don't drop journalctl content if failure action is "shell"
by Kairui Song 08 Sep '20

08 Sep '20

If failure action is set to "shell", user will need more debug info available in kdump kernel. Especially when serial console is not available, manually retrieve the log from journalctl is very useful for debugging kdump issue. Else, we can still drop journalctl content to save memory assuming nothing will use it. Kairui Song (2): dracut-module-install: Move systemd conf install code to a function Don't drop journalctl content if failure action is "shell" dracut-module-setup.sh | 44 +++++++++++++++++++++++++----------------- 1 file changed, 26 insertions(+), 18 deletions(-) -- 2.26.2

2 3

[PATCH v2 0/2] Add additional debugging messages for the kdump service
by Lianbo Jiang 24 Aug '20

24 Aug '20

Sometimes, debugging the kdump service failure becomes very challenging because there is no complete debugging information, which requires modification of the options or the scripts like kdumpctl, mkdumprd, etc to collect the information for troubleshooting. That means users have to wait for the next failure so that they can capture the additional information, which could waste valuable time. This patch series will add the debugging messages and save them to a file. It includes the following patches: [1] [PATCH 1/2] kdumpctl/mkdumprd: add 'set -x' to output debugging information to a file [2] [PATCH 2/2] kdumpctl: add the '-d' option to enable the kexec loading debugging messages Changes since v1: [1] Set default log path to '/var/log/kdump' instead of '/var/crash' [2] Add a timestamp to the debug log messages [3] Print a message to differentiate between the kdumpctl and mkdumprd log Lianbo Jiang (2): kdumpctl/mkdumprd: add 'set -x' to output debugging information to a file kdumpctl: add the '-d' option to enable the kexec loading debugging messages kdumpctl | 12 +++++++++++- mkdumprd | 10 ++++++++++ 2 files changed, 21 insertions(+), 1 deletion(-) -- 2.17.1

6 9

[PATCH v3 0/4] Rework kernel image and initrd detection code
by Kairui Song 19 Aug '20

19 Aug '20

This patch series refactor kernel image and initrd detection, fix kdump service failure on some Fedora machines which use /boot/<machine-id>/<kver>/vmlinuz as kernel installation destination. Also cover the atomic corner case properly by refering BOOT_IMAGE= cmdline param for kernel image location. Also introduce a kernel install hook to clean up the initramfs properly on kernel uninstall. Update from V2: - Don't error out if DEFAULT_INITRD is not found, so if the kernel is installed without initramfs kdump will also work. Update from V1: - Add the kdump initramfs clean up hook to make sure it work with different kernel installation style. Kairui Song (4): Add a kernel install hook to clean up kdump initramfs early-kdump: Use consistent symbol link for kernel and initramfs Refactor kernel image and initrd detection code kdump-lib.sh: Remove is_atomic 60-kdump.install | 30 ++++++++++++ dracut-early-kdump-module-setup.sh | 21 ++++----- dracut-early-kdump.sh | 6 +-- kdump-lib.sh | 73 ++++++++++++++++++++++-------- kdumpctl | 48 ++++++++++---------- kexec-tools.spec | 3 ++ 6 files changed, 122 insertions(+), 59 deletions(-) create mode 100755 60-kdump.install -- 2.26.2

2 8

[PATCH v2 0/4] Add selftest for kexec-tools
by Kairui Song 14 Aug '20

14 Aug '20

This patch series adds the VM based selftest framework for Fedora kexec-tools. Currently there are 3 test cases, more could be added later. And the usage and details are included in the document for selftest. This help do quick sanity check of new patches, also help debug issues when developing kexec-tools. The test output looks like this: ==== Starting all tests: ==== local-kdump nfs-kdump ssh-kdump ======== Running Test Case local-kdump ======== ---- Building image for: 0-local.sh ---- -------- Output image is: /home/kasong/fedpkg/kexec-tools/tests/output/local-kdump/0-local.img -------- Building log is: /home/kasong/fedpkg/kexec-tools/tests/output/local-kdump/0-local.img.log ---- Starting test VM: 0-local.sh ---- -------- Qemu cmdline: /home/kasong/fedpkg/kexec-tools/tests/output/local-kdump/0-local.qemu_cmd -------- Console log: /home/kasong/fedpkg/kexec-tools/tests/output/local-kdump/0-local.console -------- Test log: /home/kasong/fedpkg/kexec-tools/tests/output/local-kdump/0-local.output --------Test finished: local-kdump TEST PASSED You can retrive the verify the vmcore file using following command: ./scripts/copy-from-image.sh \ /home/kasong/fedpkg/kexec-tools/tests/output/local-kdump/0-local.img \ /var/crash/127.0.0.1-2020-07-31-06:58:44/vmcore ./ ======== Running Test Case nfs-kdump ======== ---- Building image for: 1-client.sh ---- -------- Output image is: /home/kasong/fedpkg/kexec-tools/tests/output/nfs-kdump/1-client.img -------- Building log is: /home/kasong/fedpkg/kexec-tools/tests/output/nfs-kdump/1-client.img.log ---- Building image for: 0-server.sh ---- -------- Output image is: /home/kasong/fedpkg/kexec-tools/tests/output/nfs-kdump/0-server.img -------- Building log is: /home/kasong/fedpkg/kexec-tools/tests/output/nfs-kdump/0-server.img.log ---- Starting VM: 0-server.sh ---- -------- Qemu cmdline: /home/kasong/fedpkg/kexec-tools/tests/output/nfs-kdump/0-server.qemu_cmd -------- Console log: /home/kasong/fedpkg/kexec-tools/tests/output/nfs-kdump/0-server.console -------- Test log: /home/kasong/fedpkg/kexec-tools/tests/output/nfs-kdump/0-server.output ---- Starting test VM: 1-client.sh ---- -------- Qemu cmdline: /home/kasong/fedpkg/kexec-tools/tests/output/nfs-kdump/1-client.qemu_cmd -------- Console log: /home/kasong/fedpkg/kexec-tools/tests/output/nfs-kdump/1-client.console -------- Test log: /home/kasong/fedpkg/kexec-tools/tests/output/nfs-kdump/1-client.output --------Test finished: nfs-kdump TEST PASSED You can retrive the verify the vmcore file using following command: ./scripts/copy-from-image.sh \ /home/kasong/fedpkg/kexec-tools/tests/output/nfs-kdump/0-server.img \ /srv/nfs/var/crash/192.168.77.62-2020-07-31-07:03:12/vmcore ./ ======== Running Test Case ssh-kdump ======== ---- Building image for: 1-client.sh ---- -------- Output image is: /home/kasong/fedpkg/kexec-tools/tests/output/ssh-kdump/1-client.img -------- Building log is: /home/kasong/fedpkg/kexec-tools/tests/output/ssh-kdump/1-client.img.log ---- Building image for: 0-server.sh ---- -------- Output image is: /home/kasong/fedpkg/kexec-tools/tests/output/ssh-kdump/0-server.img -------- Building log is: /home/kasong/fedpkg/kexec-tools/tests/output/ssh-kdump/0-server.img.log ---- Starting VM: 0-server.sh ---- -------- Qemu cmdline: /home/kasong/fedpkg/kexec-tools/tests/output/ssh-kdump/0-server.qemu_cmd -------- Console log: /home/kasong/fedpkg/kexec-tools/tests/output/ssh-kdump/0-server.console -------- Test log: /home/kasong/fedpkg/kexec-tools/tests/output/ssh-kdump/0-server.output ---- Starting test VM: 1-client.sh ---- -------- Qemu cmdline: /home/kasong/fedpkg/kexec-tools/tests/output/ssh-kdump/1-client.qemu_cmd -------- Console log: /home/kasong/fedpkg/kexec-tools/tests/output/ssh-kdump/1-client.console -------- Test log: /home/kasong/fedpkg/kexec-tools/tests/output/ssh-kdump/1-client.output --------Test finished: ssh-kdump TEST PASSED You can retrive the verify the vmcore file using following command: ./scripts/copy-from-image.sh \ /home/kasong/fedpkg/kexec-tools/tests/output/ssh-kdump/0-server.img \ /var/crash/192.168.77.62-2020-07-31-07:05:57/vmcore.flat ./ ======== Test results ======== ---------------- nfs-kdump: TEST PASSED ---------------- ssh-kdump: TEST PASSED ---------------- local-kdump: TEST PASSED Update from V1: - Add a tool and log output to let user know how to retrive the dumped vmcore - Fix usage of makedumpfile when extracting dmesg from vmcore to verify vmcore is valid - Fix some misc problems found during testing patches using selftest framework, including Makefile dependency issue Kairui Song (4): selftest: Add basic infrastructure to build test image selftest: Add basic test framework selftest: Add document for selftests selftest: Show the path of dumped vmcore on test end tests/Makefile | 88 ++++++ tests/README | 65 +++++ tests/scripts/build-image.sh | 57 ++++ tests/scripts/build-scripts/base-image.sh | 10 + .../scripts/build-scripts/test-base-image.sh | 21 ++ tests/scripts/build-scripts/test-image.sh | 21 ++ tests/scripts/copy-from-image.sh | 25 ++ tests/scripts/image-init-lib.sh | 252 ++++++++++++++++++ tests/scripts/kexec-kdump-test/init.sh | 113 ++++++++ .../kexec-kdump-test/kexec-kdump-test.service | 9 + tests/scripts/kexec-kdump-test/test.sh | 15 ++ tests/scripts/run-test.sh | 146 ++++++++++ tests/scripts/spawn-image-shell.sh | 16 ++ tests/scripts/test-lib.sh | 177 ++++++++++++ .../scripts/testcases/local-kdump/0-local.sh | 32 +++ tests/scripts/testcases/nfs-kdump/0-server.sh | 38 +++ tests/scripts/testcases/nfs-kdump/1-client.sh | 30 +++ tests/scripts/testcases/ssh-kdump/0-server.sh | 34 +++ tests/scripts/testcases/ssh-kdump/1-client.sh | 40 +++ 19 files changed, 1189 insertions(+) create mode 100644 tests/Makefile create mode 100644 tests/README create mode 100755 tests/scripts/build-image.sh create mode 100755 tests/scripts/build-scripts/base-image.sh create mode 100755 tests/scripts/build-scripts/test-base-image.sh create mode 100755 tests/scripts/build-scripts/test-image.sh create mode 100755 tests/scripts/copy-from-image.sh create mode 100644 tests/scripts/image-init-lib.sh create mode 100755 tests/scripts/kexec-kdump-test/init.sh create mode 100644 tests/scripts/kexec-kdump-test/kexec-kdump-test.service create mode 100755 tests/scripts/kexec-kdump-test/test.sh create mode 100755 tests/scripts/run-test.sh create mode 100755 tests/scripts/spawn-image-shell.sh create mode 100644 tests/scripts/test-lib.sh create mode 100755 tests/scripts/testcases/local-kdump/0-local.sh create mode 100755 tests/scripts/testcases/nfs-kdump/0-server.sh create mode 100755 tests/scripts/testcases/nfs-kdump/1-client.sh create mode 100755 tests/scripts/testcases/ssh-kdump/0-server.sh create mode 100755 tests/scripts/testcases/ssh-kdump/1-client.sh -- 2.26.2

2 6

[PATCH 0/2] Add additional debugging messages for the kdump service
by Lianbo Jiang 11 Aug '20

11 Aug '20

Sometimes, debugging the kdump service failure becomes very challenging because there is no complete debugging information, which requires modification of the options or the scripts like kdumpctl, mkdumprd, etc to collect the information for troubleshooting. That means users have to wait for the next failure so that they can capture the additional information, which could waste valuable time. This patch series will add the debugging messages and save them to a file. It includes the following patches: [1] [PATCH 1/2] kdumpctl/mkdumprd: add 'set -x' to output debugging information to a file [2] [PATCH 2/2] kdumpctl: add the '-d' option to enable the kexec loading debugging messages Lianbo Jiang (2): kdumpctl/mkdumprd: add 'set -x' to output debugging information to a file kdumpctl: add the '-d' option to enable the kexec loading debugging messages kdumpctl | 11 ++++++++++- mkdumprd | 9 +++++++++ 2 files changed, 19 insertions(+), 1 deletion(-) -- 2.17.1

3 5

[PATCH v2 0/4] Rework kernel image and initrd detection code and install hook
by Kairui Song 06 Aug '20

06 Aug '20

This patch series refactor kernel image and initrd detection, fix kdump service failure on some Fedora machines which use /boot/<machine-id>/<kver>/vmlinuz as kernel installation destination. Also cover the atomic corner case properly by refering BOOT_IMAGE= cmdline param for kernel image location. Also introduce a kernel install hook to clean up the initramfs properly on kernel uninstall. Update from V1: - Add the kdump initramfs clean up hook to make sure it work with different kernel installation style. Kairui Song (4): Add a kernel install hook to clean up kdump initramfs early-kdump: Use consistent symbol link for kernel and initramfs Refactor kernel image and initrd detection code kdump-lib.sh: Remove is_atomic 60-kdump.install | 30 ++++++++++++ dracut-early-kdump-module-setup.sh | 21 +++----- dracut-early-kdump.sh | 6 +-- kdump-lib.sh | 78 +++++++++++++++++++++++------- kdumpctl | 44 ++++++++--------- kexec-tools.spec | 3 ++ 6 files changed, 123 insertions(+), 59 deletions(-) create mode 100755 60-kdump.install -- 2.26.2

2 8

[PATCH] mkdumprd: Improve the warning message when using encrypted target
by Kairui Song 06 Aug '20

06 Aug '20

Encrypted target have many issues, so let user check kexec-kdump-howto.txt, which have more details. Signed-off-by: Kairui Song <kasong(a)redhat.com> --- mkdumprd | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mkdumprd b/mkdumprd index 91ce3d5..36f71b7 100644 --- a/mkdumprd +++ b/mkdumprd @@ -400,7 +400,7 @@ if ! check_resettable; then fi if ! check_crypt; then - echo "Warning: Encrypted device is in dump path. User will prompted for password during second kernel boot." + echo "Warning: Encrypted device is in dump path, which is not recommended, see kexec-kdump-howto.txt for more details." fi # firstly get right SSH_KEY_LOCATION -- 2.26.2

2 1

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

kexec August 2020