Currently kexec-tools always depend on dump target to be mounted, which caused some inconvenience for setup.
So for user configured target, allow kdump to start and build initramfs even if target is not mounted.
Behavior is not changed for "path" based dump target config.
When a unmounted dump target is explcitely configured, mkdumprd will look for corresponding mount info in fstab for the mount options, and failback to use "defaults" as mount option.
Then mkdumprd will try to mount it and do basic checks on the device, then umount it on exit.
NOTE: If there is a fstab entry but "noauto" option is not used, then there must be some reason that the target device is not mounted, mkdumprd will error out and ask user to double check.
Update from v1: Rebase and add some document.
Kairui Song (7): Add a is_mounted helper Allow calling mkdumprd from kdumpctl even if targat not mounted kdump-lib.sh: add fstab failback helper for getting mount info User get_mount_info to replace findmnt calls mkdumprd: generate usable kdump initramfs even target is not mounted kexec-kdump-howto.txt: Add some format to the document Update docs for the new noauto dump target support
kdump-lib-initramfs.sh | 23 ++- kdump-lib.sh | 58 ++++--- kdumpctl | 9 +- kexec-kdump-howto.txt | 363 ++++++++++++++++++++++++++--------------- mkdumprd | 130 +++++++++++---- 5 files changed, 371 insertions(+), 212 deletions(-)
Use is_mounted helper instaed of calling findmnt directly or checking if "mount" value is empty.
If findmnt looks for fstab as well, some non mounted entry will also return value. Required to support non-mounted target.
Signed-off-by: Kairui Song kasong@redhat.com --- kdump-lib-initramfs.sh | 2 +- kdump-lib.sh | 5 +++++ kdumpctl | 2 +- mkdumprd | 2 +- 4 files changed, 8 insertions(+), 3 deletions(-)
diff --git a/kdump-lib-initramfs.sh b/kdump-lib-initramfs.sh index 49b12dc..c374c4b 100755 --- a/kdump-lib-initramfs.sh +++ b/kdump-lib-initramfs.sh @@ -101,7 +101,7 @@ dump_fs() local _mp=$(findmnt -k -f -n -r -o TARGET $1) local _op=$(findmnt -k -f -n -r -o OPTIONS $1)
- if [ -z "$_mp" ]; then + if ! is_mounted "$_mp"; then _dev=$(findmnt -s -f -n -r -o SOURCE $1) _mp=$(findmnt -s -f -n -r -o TARGET $1) _op=$(findmnt -s -f -n -r -o OPTIONS $1) diff --git a/kdump-lib.sh b/kdump-lib.sh index 2157c34..a5951ad 100755 --- a/kdump-lib.sh +++ b/kdump-lib.sh @@ -246,6 +246,11 @@ get_target_from_path() echo $_target }
+is_mounted() +{ + findmnt -k -n $1 &>/dev/null +} + get_fs_type_from_target() { findmnt -k -f -n -r -o FSTYPE $1 diff --git a/kdumpctl b/kdumpctl index 081720e..c840b28 100755 --- a/kdumpctl +++ b/kdumpctl @@ -475,7 +475,7 @@ check_dump_fs_modified() fi fi
- if ! findmnt $_target >/dev/null; then + if ! is_mounted $_target; then echo "Dump target $_target is probably not mounted." return 2 fi diff --git a/mkdumprd b/mkdumprd index c2da275..4fc0a5b 100644 --- a/mkdumprd +++ b/mkdumprd @@ -366,7 +366,7 @@ do extra_modules="$extra_modules $config_val" ;; ext[234]|xfs|btrfs|minix|nfs) - if ! findmnt $config_val >/dev/null; then + if ! is_mounted $config_val; then perror_exit "Dump target $config_val is probably not mounted." fi
Ignore mount check in kdumpctl, mkdumprd will still fail building and exit if target is not mounted.
Signed-off-by: Kairui Song kasong@redhat.com --- kdumpctl | 5 ----- 1 file changed, 5 deletions(-)
diff --git a/kdumpctl b/kdumpctl index c840b28..23c0c38 100755 --- a/kdumpctl +++ b/kdumpctl @@ -475,11 +475,6 @@ check_dump_fs_modified() fi fi
- if ! is_mounted $_target; then - echo "Dump target $_target is probably not mounted." - return 2 - fi - _new_mntpoint="$(get_kdump_mntpoint_from_target $_target)" _dracut_args=$(lsinitrd $TARGET_INITRD -f usr/lib/dracut/build-parameter.txt) if [[ -z "$_dracut_args" ]];then
This allows look up mount info even if target is not mounted.
Signed-off-by: Kairui Song kasong@redhat.com --- kdump-lib.sh | 21 ++++++++++++++++++--- 1 file changed, 18 insertions(+), 3 deletions(-)
diff --git a/kdump-lib.sh b/kdump-lib.sh index a5951ad..95162cd 100755 --- a/kdump-lib.sh +++ b/kdump-lib.sh @@ -236,7 +236,7 @@ get_bind_mount_source() echo $_mntpoint$_path }
-# Return the real underlaying device of a path, ignore bind mounts +# Return the current underlaying device of a path, ignore bind mounts get_target_from_path() { local _target @@ -251,16 +251,31 @@ is_mounted() findmnt -k -n $1 &>/dev/null }
+get_mount_info() +{ + local _info_type=$1 _src_type=$2 _src=$3; shift 3 + local _info=$(findmnt --real -k -n -r -o $_info_type --$_src_type $_src $@) + + [ -z "$_info" ] && [ -e "/etc/fstab" ] && _info=$(findmnt --real -s -n -r -o $_info_type --$_src_type $_src $@) + + echo $_info +} + get_fs_type_from_target() { - findmnt -k -f -n -r -o FSTYPE $1 + get_mount_info FSTYPE source $1 -f +} + +get_mntopt_from_target() +{ + get_mount_info OPTIONS source $1 -f }
# Find the general mount point of a dump target, not the bind mount point get_mntpoint_from_target() { # Expcilitly specify --source to findmnt could ensure non-bind mount is returned - findmnt -k -f -n -r -o TARGET --source $1 + get_mount_info TARGET source $1 -f }
# Get the path where the target will be mounted in kdump kernel
Use get_mount_info so that fstab is used as a failback when look for mount info.
Signed-off-by: Kairui Song kasong@redhat.com --- kdump-lib-initramfs.sh | 23 ++++++++++------------- kdump-lib.sh | 1 - kdumpctl | 4 ++-- mkdumprd | 42 +++++++++++++++--------------------------- 4 files changed, 27 insertions(+), 43 deletions(-)
diff --git a/kdump-lib-initramfs.sh b/kdump-lib-initramfs.sh index c374c4b..ab78be3 100755 --- a/kdump-lib-initramfs.sh +++ b/kdump-lib-initramfs.sh @@ -94,19 +94,16 @@ get_kdump_confs() fi }
-# dump_fs <mount point| device> +# dump_fs <mount point> dump_fs() { - local _dev=$(findmnt -k -f -n -r -o SOURCE $1) - local _mp=$(findmnt -k -f -n -r -o TARGET $1) - local _op=$(findmnt -k -f -n -r -o OPTIONS $1) + local _mp=$1 + local _dev=$(get_mount_info SOURCE target $_mp -f) + local _op=$(get_mount_info OPTIONS target $_mp -f)
- if ! is_mounted "$_mp"; then - _dev=$(findmnt -s -f -n -r -o SOURCE $1) - _mp=$(findmnt -s -f -n -r -o TARGET $1) - _op=$(findmnt -s -f -n -r -o OPTIONS $1) - - if [ -n "$_dev" ] && [ -n "$_mp" ]; then + # If dump path have a corresponding device entry but not mounted, mount it. + if [ -n "$_dev" ]; then + if ! is_mounted "$_mp"; then echo "kdump: dump target $_dev is not mounted, trying to mount..." mkdir -p $_mp mount -o $_op $_dev $_mp @@ -115,11 +112,10 @@ dump_fs() echo "kdump: mounting failed (mount point: $_mp, option: $_op)" return 1 fi - else - echo "kdump: error: Dump target $_dev is not usable" fi else - echo "kdump: dump target is $_dev" + echo "kdump: failed to dump to "$_mp", it's not a mount point!" + return 1 fi
# Remove -F in makedumpfile case. We don't want a flat format dump here. @@ -260,6 +256,7 @@ read_kdump_conf() [ -n "$config_val" ] && add_dump_code "dump_fs $config_val" ;; ext[234]|xfs|btrfs|minix|nfs) + config_val=$(get_mntpoint_from_target "$config_val") add_dump_code "dump_fs $config_val" ;; raw) diff --git a/kdump-lib.sh b/kdump-lib.sh index 95162cd..691a14d 100755 --- a/kdump-lib.sh +++ b/kdump-lib.sh @@ -270,7 +270,6 @@ get_mntopt_from_target() { get_mount_info OPTIONS source $1 -f } - # Find the general mount point of a dump target, not the bind mount point get_mntpoint_from_target() { diff --git a/kdumpctl b/kdumpctl index 23c0c38..4a674f2 100755 --- a/kdumpctl +++ b/kdumpctl @@ -907,8 +907,8 @@ path_to_be_relabeled()
_target=$(local_fs_dump_target) if [[ -n "$_target" ]]; then - _mnt=$(findmnt -k -f -n -r -o TARGET $_target) - if [ -z "$_mnt" ]; then + _mnt=$(get_mntpoint_from_target $_target) + if ! is_mounted "$_mnt"; then return fi else diff --git a/mkdumprd b/mkdumprd index 4fc0a5b..d08a34d 100644 --- a/mkdumprd +++ b/mkdumprd @@ -51,21 +51,18 @@ add_dracut_sshkey() {
# caller should ensure $1 is valid and mounted in 1st kernel to_mount() { - local _dev=$1 _source _new_mntpoint _fstype _options _mntopts _pdev - - _source=$(findmnt -k -f -n -r -o SOURCE $_dev) - _fstype=$(findmnt -k -f -n -r -o FSTYPE $_dev) - _new_mntpoint=$(get_kdump_mntpoint_from_target $_dev) - - [[ -e /etc/fstab ]] && _options=$(findmnt --fstab -f -n -r -o OPTIONS $_dev) - if [ -z "$_options" ]; then - _options=$(findmnt -k -f -n -r -o OPTIONS $_dev) - if [[ $_fstype == "nfs"* ]]; then - _options=$(echo $_options | sed 's/,addr=[^,]*//') - _options=$(echo $_options | sed 's/,proto=[^,]*//') - _options=$(echo $_options | sed 's/,clientaddr=[^,]*//') - fi + local _target=$1 _new_mntpoint _fstype _options _mntopts _pdev + + _fstype=$(get_fs_type_from_target $_target) + _options=$(get_mntopt_from_target $_target) + _new_mntpoint=$(get_kdump_mntpoint_from_target $_target) + + if [[ "$_fstype" == "nfs"* ]]; then + _options=$(echo $_options | sed 's/,addr=[^,]*//') + _options=$(echo $_options | sed 's/,proto=[^,]*//') + _options=$(echo $_options | sed 's/,clientaddr=[^,]*//') fi + # mount fs target as rw in 2nd kernel _options=$(echo $_options | sed 's/(^|,)ro($|,)/\1rw\2/g') # with 'noauto' in fstab nfs and non-root disk mount will fail in 2nd @@ -79,28 +76,19 @@ to_mount() { _options="$_options,nofail,x-systemd.before=kdump-capture.service"
_mntopts="$_new_mntpoint $_fstype $_options" - # for non-nfs _dev converting to use udev persistent name - if [ -b "$_source" ]; then - _pdev="$(get_persistent_dev $_source)" + # for non-nfs _target converting to use udev persistent name + if [ -b "$_target" ]; then + _pdev="$(get_persistent_dev $_target)" if [ -z "$_pdev" ]; then return 1 fi else - _pdev=$_dev + _pdev=$_target fi
echo "$_pdev $_mntopts" }
-is_readonly_mount() { - local _mnt - _mnt=$(findmnt -k -f -n -r -o OPTIONS $1) - - #fs/proc_namespace.c: show_mountinfo(): - #seq_puts(m, mnt->mnt_flags & MNT_READONLY ? " ro" : " rw"); - [[ "$_mnt" =~ ^ro ]] -} - #Function: get_ssh_size #$1=dump target #called from while loop and shouldn't read from stdin, so we're using "ssh -n"
Currently kexec-tools always depend on dump target to be mounted, which caused some inconvenience for setup.
So for user configured target, allow kdump to start and build initramfs even if target is not mounted.
When a mounted user configured target is used, the behavior is not changed.
When a unmounted user configured target is used, mkdumprd will look for corresponding mount info in fstab, and a entry with noauto option is founded, mkdumprd will try to mount it inplace with optoins specified in fstab and do basic checks on the device, then umount it.
If there is no fstab entry, mkdumprd will try to mount it in temporary path with defaults option, do same basic check and umount it.
If there is a fstab entry but "noauto" option is not used, then there must be some reason that the target device is not mounted, mkdumprd will error out.
When path based target is used, there is no behavior change.
Signed-off-by: Kairui Song kasong@redhat.com --- kdump-lib.sh | 33 ++++++------------ mkdumprd | 94 +++++++++++++++++++++++++++++++++++++++++++++------- 2 files changed, 92 insertions(+), 35 deletions(-)
diff --git a/kdump-lib.sh b/kdump-lib.sh index 691a14d..74dac4c 100755 --- a/kdump-lib.sh +++ b/kdump-lib.sh @@ -284,12 +284,17 @@ get_kdump_mntpoint_from_target() local _mntpoint=$(get_mntpoint_from_target $1)
# mount under /sysroot if dump to root disk or mount under - # /kdumproot/$_mntpoint in other cases in 2nd kernel. systemd - # will be in charge to umount it. - if [ "$_mntpoint" = "/" ];then - _mntpoint="/sysroot" + # mount under /kdumproot if dump target is not mounted in first kernel + # mount under /kdumproot/$_mntpoint in other cases in 2nd kernel. + # systemd will be in charge to umount it. + if [ -z "$_mntpoint" ];then + _mntpoint="/kdumproot" else - _mntpoint="/kdumproot/$_mntpoint" + if [ "$_mntpoint" = "/" ];then + _mntpoint="/sysroot" + else + _mntpoint="/kdumproot/$_mntpoint" + fi fi
# strip duplicated "/" @@ -302,24 +307,6 @@ get_option_value() { strip_comments `grep "^$1[[:space:]]+" /etc/kdump.conf | tail -1 | cut -d\ -f2-` }
-check_save_path_fs() -{ - local _path=$1 - - if [ ! -d $_path ]; then - perror_exit "Dump path $_path does not exist." - fi -} - -# Check if path exists within dump target -check_save_path_user_configured() -{ - local _target=$1 _path=$2 - local _mnt=$(get_mntpoint_from_target $_target) - - check_save_path_fs "$_mnt/$_path" -} - is_atomic() { grep -q "ostree" /proc/cmdline diff --git a/mkdumprd b/mkdumprd index d08a34d..dc4e0a5 100644 --- a/mkdumprd +++ b/mkdumprd @@ -19,6 +19,20 @@ OVERRIDE_RESETTABLE=0 extra_modules="" dracut_args="--quiet --hostonly --hostonly-cmdline --hostonly-i18n --hostonly-mode strict -o "plymouth dash resume ifcfg earlykdump""
+readonly MKDUMPRD_TMPDIR="$(mktemp -d -t mkdumprd.XXXXXX)" +[ -d "$MKDUMPRD_TMPDIR" ] || perror_exit "dracut: mktemp -p -d -t dracut.XXXXXX failed." +readonly MKDUMPRD_TMPMNT="$MKDUMPRD_TMPDIR/target" + +trap ' + ret=$?; + is_mounted $MKDUMPRD_TMPMNT && umount -f $MKDUMPRD_TMPMNT; + [[ -d $MKDUMPRD_TMPDIR ]] && rm --one-file-system -rf -- "$MKDUMPRD_TMPDIR"; + exit $ret; + ' EXIT + +# clean up after ourselves no matter how we die. +trap 'exit 1;' SIGINT + is_wdt_addition_needed() { local active
@@ -51,11 +65,12 @@ add_dracut_sshkey() {
# caller should ensure $1 is valid and mounted in 1st kernel to_mount() { - local _target=$1 _new_mntpoint _fstype _options _mntopts _pdev + local _target=$1 _fstype=$2 _options=$3 _new_mntpoint _mntopts _pdev
- _fstype=$(get_fs_type_from_target $_target) - _options=$(get_mntopt_from_target $_target) _new_mntpoint=$(get_kdump_mntpoint_from_target $_target) + _fstype="${_fstype:-$(get_fs_type_from_target $_target)}" + _options="${_options:-$(get_mntopt_from_target $_options)}" + _options="${_options:-defaults}"
if [[ "$_fstype" == "nfs"* ]]; then _options=$(echo $_options | sed 's/,addr=[^,]*//') @@ -174,6 +189,67 @@ check_size() { fi }
+check_save_path_fs() +{ + local _path=$1 + + if [ ! -d $_path ]; then + perror_exit "Dump path $_path does not exist." + fi +} + +check_user_configured_target() +{ + local _target=$1 _cfg_fs_type=$2 _mounted + local _mnt=$(get_mntpoint_from_target $_target) + local _opt=$(get_mntopt_from_target $_target) + local _fstype=$(get_fs_type_from_target $_target) + + if [ -n "$_fstype" ]; then + # In case of nfs4, nfs should be used instead, nfs* options is deprecated in kdump.conf + [[ $_fstype = "nfs"* ]] && _fstype=nfs + + if [ -n "$_cfg_fs_type" ] && [ "$_fstype" != "$_cfg_fs_type" ]; then + perror_exit ""$_target" have a wrong type config "$_cfg_fs_type", expected "$_fstype"" + fi + else + _fstype="$_cfg_fs_type" + _fstype="$_cfg_fs_type" + fi + + # For noauto mount, mount it inplace with default value. + # Else use the temporary target directory + if [ -n "$_mnt" ]; then + if ! is_mounted "$_mnt"; then + if [[ $_opt = *",noauto"* ]]; then + mount $_mnt + [ $? -ne 0 ] && perror_exit "Failed to mount $_target on $_mnt for kdump preflight check." + _mounted=$_mnt + else + perror_exit "$_target is configured in fstab but not mounted, please check its usability." + fi + fi + else + _mnt=$MKDUMPRD_TMPMNT + mkdir -p $_mnt + mount $_target $_mnt -t $_fstype -o defaults + [ $? -ne 0 ] && perror_exit "Failed to mount $_target for kdump preflight check." + _mounted=$_mnt + fi + + # For user configured target, use $SAVE_PATH as the dump path within the target + if [ ! -d "$_mnt/$SAVE_PATH" ]; then + perror_exit "Dump path "$SAVE_PATH" does not exist in dump target "$_target"" + fi + + check_size fs "$_target" + + # Unmount it early, if function is interrupted and didn't reach here, the shell trap will clear it up anyway + if [ -n "$_mounted" ]; then + umount -f -- $_mounted + fi +} + # $1: core_collector config value verify_core_collector() { local _cmd="${1%% *}" @@ -201,7 +277,7 @@ verify_core_collector() { }
add_mount() { - local _mnt=$(to_mount "$1") + local _mnt=$(to_mount $@)
if [ $? -ne 0 ]; then exit 1 @@ -354,14 +430,8 @@ do extra_modules="$extra_modules $config_val" ;; ext[234]|xfs|btrfs|minix|nfs) - if ! is_mounted $config_val; then - perror_exit "Dump target $config_val is probably not mounted." - fi - - # User configured target, use $SAVE_PATH as the dump path within the target - check_save_path_user_configured "$config_val" "$SAVE_PATH" - check_size fs "$config_val" - add_mount "$config_val" + check_user_configured_target "$config_val" "$config_opt" + add_mount "$config_val" "$config_opt" ;; raw) # checking raw disk writable
When adding doc for the non-mounted dump target support, I found the document are a bit uneasy to read due to lack of a proper format, this commit should it make looks better.
Signed-off-by: Kairui Song kasong@redhat.com --- kexec-kdump-howto.txt | 210 +++++++++++++++++++++++++----------------- 1 file changed, 127 insertions(+), 83 deletions(-)
diff --git a/kexec-kdump-howto.txt b/kexec-kdump-howto.txt index 8e6cc4c..0d2b6cb 100644 --- a/kexec-kdump-howto.txt +++ b/kexec-kdump-howto.txt @@ -1,15 +1,22 @@ +================= Kexec/Kdump HOWTO +================= +
Introduction +============
Kexec and kdump are new features in the 2.6 mainstream kernel. These features are included in Red Hat Enterprise Linux 5. The purpose of these features is to ensure faster boot up and creation of reliable kernel vmcores for diagnostic purposes.
+ Overview +========
Kexec +-----
Kexec is a fastboot mechanism which allows booting a Linux kernel from the context of already running kernel without going through BIOS. BIOS can be very @@ -17,6 +24,7 @@ time consuming especially on the big servers with lots of peripherals. This can save a lot of time for developers who end up booting a machine numerous times.
Kdump +-----
Kdump is a new kernel crash dumping mechanism and is very reliable because the crash dump is captured from the context of a freshly booted kernel and @@ -52,7 +60,8 @@ Now reboot your system, taking note that it should bypass the BIOS: # reboot
-How to configure kdump: +How to configure kdump +======================
Again, we assume if you're reading this document, you should already have kexec-tools installed. If not, you install it via the following command: @@ -136,7 +145,9 @@ perform postmortem analysis:
and so on...
-Notes: + +Notes on kdump +==============
When kdump starts, the kdump kernel is loaded together with the kdump initramfs. To save memory usage and disk space, the kdump initramfs is @@ -152,8 +163,10 @@ recommended to rebuild the initramfs manually with following command:
# kdumpctl rebuild
+ Saving vmcore-dmesg.txt ----------------------- +======================= + Kernel log bufferes are one of the most important information available in vmcore. Now before saving vmcore, kernel log bufferes are extracted from /proc/vmcore and saved into a file vmcore-dmesg.txt. After @@ -161,7 +174,9 @@ vmcore-dmesg.txt, vmcore is saved. Destination disk and directory for vmcore-dmesg.txt is same as vmcore. Note that kernel log buffers will not be available if dump target is raw device.
-Dump Triggering methods: + +Dump Triggering methods +=======================
This section talks about the various ways, other than a Kernel Panic, in which Kdump can be triggered. The following methods assume that Kdump is configured @@ -268,7 +283,11 @@ to initate the dump and then click "Restart blade with NMI". This issues a system reset and invokes xmon debugger.
-Advanced Setups: +Advanced Setups +=============== + +Dump targets +------------
In addition to being able to capture a vmcore to your system's local file system, kdump can be configured to capture a vmcore to a number of other @@ -295,7 +314,6 @@ which out of the box, is fairly well documented itself. Any alterations to the changes can be incorporated in the kdump initrd. Restarting the kdump service is as simple as '/sbin/systemctl restart kdump.service'.
- Note that kdump.conf is used as a configuration mechanism for capturing dump files from the initramfs (in the interests of safety), the root file system is mounted, and the init process is started, only as a last resort if the @@ -314,7 +332,9 @@ someone uses the filesystem for something else other than dumping vmcore you can mount it as read-only. Mkdumprd will still remount it as read-write for creating dump directory and will move it back to read-only afterwards.
-Raw partition +Followings are the available config options for dump targets. + +1) Raw partition
Raw partition dumping requires that a disk partition in the system, at least as large as the amount of memory in the system, be left unformatted. Assuming @@ -325,7 +345,7 @@ onto partition /dev/vg/lv_kdump. Restart the kdump service via initrd. Dump target should be persistent device name, such as lvm or device mapper canonical name.
-Dedicated file system +2) Dedicated file system
Similar to raw partition dumping, you can format a partition with the file system of your choice, Again, it should be at least as large as the amount @@ -349,7 +369,7 @@ Be careful of your filesystem selection when using this target. It is recommended to use persistent device names or UUID/LABEL for file system dumps. One example of persistent device is /dev/vg/<devname>.
-NFS mount +3) NFS mount
Dumping over NFS requires an NFS server configured to export a file system with full read/write access for the root user. All operations done within @@ -367,7 +387,7 @@ mount the NFS mount and copy out the vmcore to your NFS server. Restart the kdump service via '/sbin/systemctl restart kdump.service' to commit this change to your kdump initrd.
-Special mount via "dracut_args" +4) Special mount via "dracut_args"
You can utilize "dracut_args" to pass "--mount" to kdump, see dracut manpage about the format of "--mount" for details. If there is any "--mount" specified @@ -385,7 +405,7 @@ dracut_args --mount "192.168.1.1:/share /mnt/test nfs4 defaults" NOTE: - <mountpoint> must be specified as an absolute path.
-Remote system via ssh/scp +5) Remote system via ssh/scp
Dumping over ssh/scp requires setting up passwordless ssh keys for every machine you wish to have dump via this method. First up, configure kdump.conf @@ -404,7 +424,8 @@ to send over the necessary ssh key file. Restart the kdump service via '/sbin/systemctl restart kdump.service' to commit this change to your kdump initrd.
Path -==== +---- + "path" represents the file system path in which vmcore will be saved. In fact kdump creates a directory $hostip-$date with-in "path" and saves vmcore there. So practically dump is saved in $path/$hostip-$date/. To @@ -427,25 +448,26 @@ at automatically depending on what's mounted in the current system.
Following are few examples.
-path /var/crash/ ----------------- -Assuming there is no disk mounted on /var/ or on /var/crash, dump will -be saved on disk backing rootfs in directory /var/crash. +- path /var/crash/ + + Assuming there is no disk mounted on /var/ or on /var/crash, dump will + be saved on disk backing rootfs in directory /var/crash. + +- path /var/crash/ (A separate disk mounted on /var)
-path /var/crash/ (A separate disk mounted on /var) --------------------------------------------------- -Say a disk /dev/sdb is mouted on /var. In this case dump target will -become /dev/sdb and path will become "/crash" and dump will be saved -on "sdb:/crash/" directory. + Say a disk /dev/sdb is mouted on /var. In this case dump target will + become /dev/sdb and path will become "/crash" and dump will be saved + on "sdb:/crash/" directory.
-path /var/crash/ (NFS mounted on /var) -------------------------------------- -Say foo.com:/export/tmp is mounted on /var. In this case dump target is -nfs server and path will be adjusted to "/crash" and dump will be saved to -foo.com:/export/tmp/crash/ directory. +- path /var/crash/ (NFS mounted on /var) + + Say foo.com:/export/tmp is mounted on /var. In this case dump target is + nfs server and path will be adjusted to "/crash" and dump will be saved to + foo.com:/export/tmp/crash/ directory.
Kdump boot directory -==================== +-------------------- + Usually kdump kernel is the same as 1st kernel. So kdump will try to find kdump kernel under /boot according to /proc/cmdline. E.g we execute below command and get an output: @@ -456,6 +478,7 @@ However a variable KDUMP_BOOTDIR in /etc/sysconfig/kdump is provided to user if kdump kernel is put in a different directory.
Kdump Post-Capture Executable +-----------------------------
It is possible to specify a custom script or binary you wish to run following an attempt to capture a vmcore. The executable is passed an exit code from @@ -463,6 +486,7 @@ the capture process, which can be used to trigger different actions from within your post-capture executable.
Kdump Pre-Capture Executable +----------------------------
It is possible to specify a custom script or binary you wish to run before capturing a vmcore. Exit status of this binary is interpreted: @@ -470,6 +494,7 @@ capturing a vmcore. Exit status of this binary is interpreted: non 0 - reboot the system
Extra Binaries +--------------
If you have specific binaries or scripts you want to have made available within your kdump initrd, you can specify them by their full path, and they @@ -478,6 +503,7 @@ This may be particularly useful for those running post-capture scripts that rely on other binaries.
Extra Modules +-------------
By default, only the bare minimum of kernel modules will be included in your kdump initrd. Should you wish to capture your vmcore files to a non-boot-path @@ -486,7 +512,8 @@ need to manually specify additional kernel modules to load into your kdump initrd.
Failure action -============== +-------------- + Failure action specifies what to do when dump to configured dump target fails. By default, failure action is "reboot" and that is system reboots if attempt to save dump to dump target fails. @@ -494,21 +521,24 @@ if attempt to save dump to dump target fails. There are other failure actions available though.
- dump_to_rootfs - This option tries to mount root and save dump on root filesystem - in a path specified by "path". This option will generally make - sense when dump target is not root filesystem. For example, if - dump is being saved over network using "ssh" then one can specify - failure action to "dump_to_rootfs" to try saving dump to root - filesystem if dump over network fails. + This option tries to mount root and save dump on root filesystem + in a path specified by "path". This option will generally make + sense when dump target is not root filesystem. For example, if + dump is being saved over network using "ssh" then one can specify + failure action to "dump_to_rootfs" to try saving dump to root + filesystem if dump over network fails.
- shell - Drop into a shell session inside initramfs. + Drop into a shell session inside initramfs. + - halt - Halt system after failure + Halt system after failure + - poweroff - Poweroff system after failure. + Poweroff system after failure.
Compression and filtering +-------------------------
The 'core_collector' parameter in kdump.conf allows you to specify a custom dump capture method. The most common alternate method is makedumpfile, which @@ -526,22 +556,21 @@ Core collector command format depends on dump target type. Typically for filesystem (local/remote), core_collector should accept two arguments. First one is source file and second one is target file. For ex.
-ex1. ---- -core_collector "cp --sparse=always" +- ex1. + + core_collector "cp --sparse=always"
-Above will effectively be translated to: + Above will effectively be translated to:
-cp --sparse=always /proc/vmcore <dest-path>/vmcore + cp --sparse=always /proc/vmcore <dest-path>/vmcore
-ex2. ---- -core_collector "makedumpfile -l --message-level 1 -d 31" +- ex2.
-Above will effectively be translated to: + core_collector "makedumpfile -l --message-level 1 -d 31"
-makedumpfile -l --message-level 1 -d 31 /proc/vmcore <dest-path>/vmcore + Above will effectively be translated to:
+ makedumpfile -l --message-level 1 -d 31 /proc/vmcore <dest-path>/vmcore
For dump targets like raw and ssh, in general, core collector should expect one argument (source file) and should output the processed core on standard @@ -549,55 +578,56 @@ output (There is one exception of "scp", discussed later). This standard output will be saved to destination using appropriate commands.
raw dumps core_collector examples: ---------- -ex3. ---- -core_collector "cat"
-Above will effectively be translated to. +- ex3. + + core_collector "cat"
-cat /proc/vmcore | dd of=<target-device> + Above will effectively be translated to.
-ex4. ---- -core_collector "makedumpfile -F -l --message-level 1 -d 31" + cat /proc/vmcore | dd of=<target-device>
-Above will effectively be translated to. +- ex4.
-makedumpfile -F -l --message-level 1 -d 31 | dd of=<target-device> + core_collector "makedumpfile -F -l --message-level 1 -d 31" + + Above will effectively be translated to. + + makedumpfile -F -l --message-level 1 -d 31 | dd of=<target-device>
ssh dumps core_collector examples: ---------- -ex5. ---- -core_collector "cat"
-Above will effectively be translated to. +- ex5. + + core_collector "cat"
-cat /proc/vmcore | ssh <options> <remote-location> "dd of=path/vmcore" + Above will effectively be translated to.
-ex6. ---- -core_collector "makedumpfile -F -l --message-level 1 -d 31" + cat /proc/vmcore | ssh <options> <remote-location> "dd of=path/vmcore"
-Above will effectively be translated to. +- ex6.
-makedumpfile -F -l --message-level 1 -d 31 | ssh <options> <remote-location> "dd of=path/vmcore" + core_collector "makedumpfile -F -l --message-level 1 -d 31" + + Above will effectively be translated to. + + makedumpfile -F -l --message-level 1 -d 31 | ssh <options> <remote-location> "dd of=path/vmcore"
There is one exception to standard output rule for ssh dumps. And that is scp. As scp can handle ssh destinations for file transfers, one can specify "scp" as core collector for ssh targets (no output on stdout).
-ex7. ----- -core_collector "scp" +- ex7.
-Above will effectively be translated to. + core_collector "scp"
-scp /proc/vmcore user@host:path/vmcore + Above will effectively be translated to. + + scp /proc/vmcore user@host:path/vmcore
About default core collector ---------------------------- + Default core_collector for ssh/raw dump is: "makedumpfile -F -l --message-level 1 -d 31". Default core_collector for other targets is: @@ -615,7 +645,9 @@ dump data from stdard input to a normal dumpfile (readable with analysis tools). For example: "makedumpfile -R vmcore < vmcore.flat"
-Caveats: + +Caveats +=======
Console frame-buffers and X are not properly supported. If you typically run with something along the lines of "vga=791" in your kernel config line or @@ -624,7 +656,11 @@ kexec. Note that the kdump kernel should still be able to create a dump, and when the system reboots, video should be restored to normal.
+Notes +===== + Notes on resetting video: +-------------------------
Video is a notoriously difficult issue with kexec. Video cards contain ROM code that controls their initial configuration and setup. This code is nominally @@ -646,7 +682,9 @@ Secondly, it may be worth trying to add vga15fb.ko to the extra_modules list in /etc/kdump.conf. This will attempt to use the video card in framebuffer mode, which can blank the screen prior to the start of a dump capture.
-Notes on rootfs mount: +Notes on rootfs mount +--------------------- + Dracut is designed to mount rootfs by default. If rootfs mounting fails it will refuse to go on. So kdump leaves rootfs mounting to dracut currently. We make the assumtion that proper root= cmdline is being passed to dracut @@ -656,7 +694,8 @@ options are copied from /proc/cmdline. In general it is best to append command line options using "KDUMP_COMMANDLINE_APPEND=" instead of replacing the original command line completely.
-Notes on watchdog module handling: +Notes on watchdog module handling +---------------------------------
If a watchdog is active in first kernel then, we must have it's module loaded in crash kernel, so that either watchdog is deactivated or started @@ -670,7 +709,8 @@ not been written in watchdog-core framework then this option will not have any effect and module will not be added. Please note that only systemd watchdog daemon is supported as watchdog kick application.
-Notes for disk images: +Notes for disk images +---------------------
Kdump initramfs is a critical component for capturing the crash dump. But it's strictly generated for the machine it will run on, and have @@ -684,7 +724,8 @@ a machine with a disk image which have kdump initramfs embedded, you should rebuild the initramfs using "kdumpctl rebuild" command manually, or else kdump may not work as expeceted.
-Notes on encrypted dump target: +Notes on encrypted dump target +------------------------------
Currently, kdump is not working well with encrypted dump target. First, user have to give the password manually in capture kernel, @@ -698,7 +739,8 @@ crash kernel according, or update your encryption setup. It's recommanded to use a non-encrypted target (eg. remote target) instead.
-Notes on device dump: +Notes on device dump +--------------------
Device dump allows drivers to append dump data to vmcore, so you can collect driver specified debug info. The drivers could append the @@ -715,8 +757,10 @@ the dump target setup will be included. To ensure the device dump data will be included in the vmcore, you need to force include related device drivers by using "extra_modules" option in /etc/kdump.conf
+ Parallel Dumping Operation ========================== + Kexec allows kdump using multiple cpus. So parallel feature can accelerate dumping substantially, especially in executing compression and filter. For example: @@ -746,8 +790,10 @@ may lead to panic due to Out Of Memory. hang, system reset or power-off at boot, depending on your system and runtime situation at the time of crash.
+ Debugging Tips --------------- +============== + - One can drop into a shell before/after saving vmcore with the help of using kdump_pre/kdump_post hooks. Use following in one of the pre/post scripts to drop into a shell. @@ -772,5 +818,3 @@ Debugging Tips minicom -C /tmp/console-logs
Now minicom should be logging serial console in file console-logs. - -
Signed-off-by: Kairui Song kasong@redhat.com --- kexec-kdump-howto.txt | 195 +++++++++++++++++++++++++++--------------- 1 file changed, 127 insertions(+), 68 deletions(-)
diff --git a/kexec-kdump-howto.txt b/kexec-kdump-howto.txt index 0d2b6cb..06dfbcf 100644 --- a/kexec-kdump-howto.txt +++ b/kexec-kdump-howto.txt @@ -283,11 +283,8 @@ to initate the dump and then click "Restart blade with NMI". This issues a system reset and invokes xmon debugger.
-Advanced Setups -=============== - Dump targets ------------- +============
In addition to being able to capture a vmcore to your system's local file system, kdump can be configured to capture a vmcore to a number of other @@ -295,7 +292,8 @@ locations, including a raw disk partition, a dedicated file system, an NFS mounted file system, or a remote system via ssh/scp. Additional options exist for specifying the relative path under which the dump is captured, what to do if the capture fails, and for compressing and filtering the dump -(so as to produce smaller, more manageable, vmcore files). +(so as to produce smaller, more manageable, vmcore files, see "Advanced Setups" +for more detail on these options).
In theory, dumping to a location other than the local file system should be safer than kdump's default setup, as its possible the default setup will try @@ -308,31 +306,131 @@ as allowing for the centralization of vmcore files, should you have several systems from which you'd like to obtain vmcore files. Of course, note that these configurations could present problems if your network is unreliable.
-Advanced setups are configured via modifications to /etc/kdump.conf, -which out of the box, is fairly well documented itself. Any alterations to -/etc/kdump.conf should be followed by a restart of the kdump service, so -the changes can be incorporated in the kdump initrd. Restarting the kdump -service is as simple as '/sbin/systemctl restart kdump.service'. - -Note that kdump.conf is used as a configuration mechanism for capturing dump -files from the initramfs (in the interests of safety), the root file system is -mounted, and the init process is started, only as a last resort if the -initramfs fails to capture the vmcore. As such, configuration made in -/etc/kdump.conf is only applicable to capture recorded in the initramfs. If -for any reason the init process is started on the root file system, only a -simple copying of the vmcore from /proc/vmcore to /var/crash/$DATE/vmcore will -be preformed. - -For both local filesystem and nfs dump the dump target must be mounted before -building kdump initramfs. That means one needs to put an entry for the dump -file system in /etc/fstab so that after reboot when kdump service starts, -it can find the dump target and build initramfs instead of failing. +Kdump target and advanced setups are configured via modifications to +/etc/kdump.conf, which out of the box, is fairly well documented itself. +Any alterations to /etc/kdump.conf should be followed by a restart of the +kdump service, so the changes can be incorporated in the kdump initrd. +Restarting the kdump service is as simple as '/sbin/systemctl restart kdump.service'. + +There are two ways to config the dump target, config dump target only +using "path", and config dump target explicitely. Interpretation of "path" +also differs in two config styples. + +Config dump target only using "path" +------------------------------------ + +You can change the dump target by setting "path" to a mount point where +dump target is mounted. When there is no explicitly configured dump target, +"path" in kdump.conf represents the current file system path in which vmcore +will be saved. Kdump will automatically detect the underlying device of +"path" and use that as the dump target. + +In fact, upon dump, kdump creates a directory $hostip-$date with-in "path" +and saves vmcore there. So practically dump is saved in $path/$hostip-$date/. + +Kdump will only check current mount status for mount entry corresponding to +"path". So please ensure the dump target is mounted on "path" before kdump +service starts. + +NOTES: + +- It's strongly recommanded to put an mount entry for "path" in /etc/fstab + and have it auto mounted on boot. This make sure the dump target is + reachable from the machine and kdump's configuration is stable. + +EXAMPLES: + +- path /var/crash/ + + This is the default configuration. Assuming there is no disk mounted + on /var/ or on /var/crash, dump will be saved on disk backing rootfs + in directory /var/crash. + +- path /var/crash/ (A separate disk mounted on /var/crash) + + Say a disk /dev/sdb is mounted on /var. In this case dump target will + become /dev/sdb and path will become "/" and dump will be saved + on "sdb:/var/crash/" directory. + +- path /var/crash/ (NFS mounted on /var) + + Say foo.com:/export/tmp is mounted on /var. In this case dump target is + nfs server and path will be adjusted to "/crash" and dump will be saved to + foo.com:/export/tmp/crash/ directory. + +Config dump target explicitely +------------------------------ + +You can set the dump target explicitly in kdump.conf, and "path" will be +the relative path in the specified dump target. For example, if dump +target is "ext4 /dev/sda", then dump will be saved in "path" directory +on /dev/sda. + +Same is the case for nfs dump. If user specified "nfs foo.com:/export/tmp/" +as dump target, then dump will effectively be saved in +"foo.com:/export/tmp/var/crash/" directory. + +If the dump target is "raw", then "path" is ignored. + +If it's a filesystemd target, kdump will need to know the right mount option. +Kdump will check current mount status, and then /etc/fstab for mount options +corresponding to the specified dump target and use it. If there are +special mount option required for the dump target, it could be set by put +an entry in fstab. + +If there are no related mount entry, mount option is set to "defaults". + +NOTES: + +- It's recommended to put an entry for the dump target in /etc/fstab + and have it auto mounted on boot. This make sure the dump target is + reachable from the machine and kdump won't fail. + +- Kdump ignores some mount options, including "noauto", "ro". This + make it possible to keep the dump target unmounted or read-only + when not used. + +EXAMPLES: + +- ext4 /dev/sda (mounted) + path /var/crash/ + + In this case dump target is set to /dev/sdb, path is the absolute path + "/var/crash" in /dev/sda, vmcore path will saved on + "sda:/var/crash" directory. + +- nfs foo.com:/export/tmp (mounted) + path /var/crash/ + + In this case dump target is nfs server, path is the absolute path + "/var/crash", vmcore path will saved on "foo.com:/export/tmp/crash/" directory. + +- nfs foo.com:/export/tmp (not mounted) + path /var/crash/ + + Same with above case, kdump will use "defaults" as the mount option + for the dump target. + +- nfs foo.com:/export/tmp (not mounted, entry with option "noauto,nolock" exists in /etc/fstab) + path /var/crash/ + + In this case dump target is nfs server, vmcore path will saved on + "foo.com:/export/tmp/crash/" directory, and kdump will inherit "nolock" option. + +Dump target and mkdumprd +------------------------ + +MKdumprd is the tool used to create kdump initramfs, and it may change +the mount status of the dump target in some condition. + Usually the dump target should be used only for kdump. If you worry about someone uses the filesystem for something else other than dumping vmcore -you can mount it as read-only. Mkdumprd will still remount it as read-write -for creating dump directory and will move it back to read-only afterwards. +you can mount it as read-only or make it a noauto mount. Mkdumprd will +mount/remount it as read-write for creating dump directory and will +move it back to it's original state afterwards.
-Followings are the available config options for dump targets. +Supported dump target types and requirements +--------------------------------------------
1) Raw partition
@@ -423,47 +521,8 @@ you've connected to it, and then input the target system user's password to send over the necessary ssh key file. Restart the kdump service via '/sbin/systemctl restart kdump.service' to commit this change to your kdump initrd.
-Path ----- - -"path" represents the file system path in which vmcore will be saved. In -fact kdump creates a directory $hostip-$date with-in "path" and saves -vmcore there. So practically dump is saved in $path/$hostip-$date/. To -simplify discussion further, if we say dump will be saved in $path, it -is implied that kdump will create another directory inside path and -save vmcore there. - -If a dump target is specified in kdump.conf, then "path" is relative to the -specified dump target. For example, if dump target is "ext4 /dev/sda", then -dump will be saved in "$path" directory on /dev/sda. - -Same is the case for nfs dump. If user specified "nfs foo.com:/export/tmp/" -as dump target, then dump will effectively be saved in -"foo.com:/export/tmp/var/crash/" directory. - -Interpretation of path changes a bit if user has not specified a dump -target explicitly in kdump.conf. In this case, "path" represents the -absolute path from root. And dump target and adjusted path are arrived -at automatically depending on what's mounted in the current system. - -Following are few examples. - -- path /var/crash/ - - Assuming there is no disk mounted on /var/ or on /var/crash, dump will - be saved on disk backing rootfs in directory /var/crash. - -- path /var/crash/ (A separate disk mounted on /var) - - Say a disk /dev/sdb is mouted on /var. In this case dump target will - become /dev/sdb and path will become "/crash" and dump will be saved - on "sdb:/crash/" directory. - -- path /var/crash/ (NFS mounted on /var) - - Say foo.com:/export/tmp is mounted on /var. In this case dump target is - nfs server and path will be adjusted to "/crash" and dump will be saved to - foo.com:/export/tmp/crash/ directory. +Advanced Setups +===============
Kdump boot directory --------------------
Some spelling issue, please see comment in line
On 05/12/2020 11:54 PM, Kairui Song wrote:
Signed-off-by: Kairui Song kasong@redhat.com
kexec-kdump-howto.txt | 195 +++++++++++++++++++++++++++--------------- 1 file changed, 127 insertions(+), 68 deletions(-)
diff --git a/kexec-kdump-howto.txt b/kexec-kdump-howto.txt index 0d2b6cb..06dfbcf 100644 --- a/kexec-kdump-howto.txt +++ b/kexec-kdump-howto.txt @@ -283,11 +283,8 @@ to initate the dump and then click "Restart blade with NMI". This issues a system reset and invokes xmon debugger.
-Advanced Setups
Dump targets
+============
In addition to being able to capture a vmcore to your system's local file system, kdump can be configured to capture a vmcore to a number of other @@ -295,7 +292,8 @@ locations, including a raw disk partition, a dedicated file system, an NFS mounted file system, or a remote system via ssh/scp. Additional options exist for specifying the relative path under which the dump is captured, what to do if the capture fails, and for compressing and filtering the dump -(so as to produce smaller, more manageable, vmcore files). +(so as to produce smaller, more manageable, vmcore files, see "Advanced Setups" +for more detail on these options).
In theory, dumping to a location other than the local file system should be safer than kdump's default setup, as its possible the default setup will try @@ -308,31 +306,131 @@ as allowing for the centralization of vmcore files, should you have several systems from which you'd like to obtain vmcore files. Of course, note that these configurations could present problems if your network is unreliable.
-Advanced setups are configured via modifications to /etc/kdump.conf, -which out of the box, is fairly well documented itself. Any alterations to -/etc/kdump.conf should be followed by a restart of the kdump service, so -the changes can be incorporated in the kdump initrd. Restarting the kdump -service is as simple as '/sbin/systemctl restart kdump.service'.
-Note that kdump.conf is used as a configuration mechanism for capturing dump -files from the initramfs (in the interests of safety), the root file system is -mounted, and the init process is started, only as a last resort if the -initramfs fails to capture the vmcore. As such, configuration made in -/etc/kdump.conf is only applicable to capture recorded in the initramfs. If -for any reason the init process is started on the root file system, only a -simple copying of the vmcore from /proc/vmcore to /var/crash/$DATE/vmcore will -be preformed.
-For both local filesystem and nfs dump the dump target must be mounted before -building kdump initramfs. That means one needs to put an entry for the dump -file system in /etc/fstab so that after reboot when kdump service starts, -it can find the dump target and build initramfs instead of failing. +Kdump target and advanced setups are configured via modifications to +/etc/kdump.conf, which out of the box, is fairly well documented itself. +Any alterations to /etc/kdump.conf should be followed by a restart of the +kdump service, so the changes can be incorporated in the kdump initrd. +Restarting the kdump service is as simple as '/sbin/systemctl restart kdump.service'.
+There are two ways to config the dump target, config dump target only +using "path", and config dump target explicitely. Interpretation of "path" +also differs in two config styples.
^ styles
+Config dump target only using "path" +------------------------------------
+You can change the dump target by setting "path" to a mount point where +dump target is mounted. When there is no explicitly configured dump target, +"path" in kdump.conf represents the current file system path in which vmcore +will be saved. Kdump will automatically detect the underlying device of +"path" and use that as the dump target.
+In fact, upon dump, kdump creates a directory $hostip-$date with-in "path" +and saves vmcore there. So practically dump is saved in $path/$hostip-$date/.
+Kdump will only check current mount status for mount entry corresponding to +"path". So please ensure the dump target is mounted on "path" before kdump +service starts.
+NOTES:
+- It's strongly recommanded to put an mount entry for "path" in /etc/fstab
- and have it auto mounted on boot. This make sure the dump target is
- reachable from the machine and kdump's configuration is stable.
+EXAMPLES:
+- path /var/crash/
- This is the default configuration. Assuming there is no disk mounted
- on /var/ or on /var/crash, dump will be saved on disk backing rootfs
- in directory /var/crash.
+- path /var/crash/ (A separate disk mounted on /var/crash)
- Say a disk /dev/sdb is mounted on /var. In this case dump target will
- become /dev/sdb and path will become "/" and dump will be saved
- on "sdb:/var/crash/" directory.
+- path /var/crash/ (NFS mounted on /var)
- Say foo.com:/export/tmp is mounted on /var. In this case dump target is
- nfs server and path will be adjusted to "/crash" and dump will be saved to
- foo.com:/export/tmp/crash/ directory.
+Config dump target explicitely
^explicitly
+------------------------------
+You can set the dump target explicitly in kdump.conf, and "path" will be +the relative path in the specified dump target. For example, if dump +target is "ext4 /dev/sda", then dump will be saved in "path" directory +on /dev/sda.
+Same is the case for nfs dump. If user specified "nfs foo.com:/export/tmp/" +as dump target, then dump will effectively be saved in +"foo.com:/export/tmp/var/crash/" directory.
+If the dump target is "raw", then "path" is ignored.
+If it's a filesystemd target, kdump will need to know the right mount option.
^ filesystem?
+Kdump will check current mount status, and then /etc/fstab for mount options +corresponding to the specified dump target and use it. If there are +special mount option required for the dump target, it could be set by put +an entry in fstab.
+If there are no related mount entry, mount option is set to "defaults".
+NOTES:
+- It's recommended to put an entry for the dump target in /etc/fstab
- and have it auto mounted on boot. This make sure the dump target is
- reachable from the machine and kdump won't fail.
+- Kdump ignores some mount options, including "noauto", "ro". This
- make it possible to keep the dump target unmounted or read-only
- when not used.
+EXAMPLES:
+- ext4 /dev/sda (mounted)
- path /var/crash/
- In this case dump target is set to /dev/sdb, path is the absolute path
- "/var/crash" in /dev/sda, vmcore path will saved on
- "sda:/var/crash" directory.
+- nfs foo.com:/export/tmp (mounted)
- path /var/crash/
- In this case dump target is nfs server, path is the absolute path
- "/var/crash", vmcore path will saved on "foo.com:/export/tmp/crash/" directory.
+- nfs foo.com:/export/tmp (not mounted)
- path /var/crash/
- Same with above case, kdump will use "defaults" as the mount option
- for the dump target.
+- nfs foo.com:/export/tmp (not mounted, entry with option "noauto,nolock" exists in /etc/fstab)
- path /var/crash/
- In this case dump target is nfs server, vmcore path will saved on
- "foo.com:/export/tmp/crash/" directory, and kdump will inherit "nolock" option.
+Dump target and mkdumprd +------------------------
+MKdumprd is the tool used to create kdump initramfs, and it may change +the mount status of the dump target in some condition.
Usually the dump target should be used only for kdump. If you worry about someone uses the filesystem for something else other than dumping vmcore -you can mount it as read-only. Mkdumprd will still remount it as read-write -for creating dump directory and will move it back to read-only afterwards. +you can mount it as read-only or make it a noauto mount. Mkdumprd will +mount/remount it as read-write for creating dump directory and will +move it back to it's original state afterwards.
-Followings are the available config options for dump targets. +Supported dump target types and requirements +--------------------------------------------
- Raw partition
@@ -423,47 +521,8 @@ you've connected to it, and then input the target system user's password to send over the necessary ssh key file. Restart the kdump service via '/sbin/systemctl restart kdump.service' to commit this change to your kdump initrd.
-Path
-"path" represents the file system path in which vmcore will be saved. In -fact kdump creates a directory $hostip-$date with-in "path" and saves -vmcore there. So practically dump is saved in $path/$hostip-$date/. To -simplify discussion further, if we say dump will be saved in $path, it -is implied that kdump will create another directory inside path and -save vmcore there.
-If a dump target is specified in kdump.conf, then "path" is relative to the -specified dump target. For example, if dump target is "ext4 /dev/sda", then -dump will be saved in "$path" directory on /dev/sda.
-Same is the case for nfs dump. If user specified "nfs foo.com:/export/tmp/" -as dump target, then dump will effectively be saved in -"foo.com:/export/tmp/var/crash/" directory.
-Interpretation of path changes a bit if user has not specified a dump -target explicitly in kdump.conf. In this case, "path" represents the -absolute path from root. And dump target and adjusted path are arrived -at automatically depending on what's mounted in the current system.
-Following are few examples.
-- path /var/crash/
- Assuming there is no disk mounted on /var/ or on /var/crash, dump will
- be saved on disk backing rootfs in directory /var/crash.
-- path /var/crash/ (A separate disk mounted on /var)
- Say a disk /dev/sdb is mouted on /var. In this case dump target will
- become /dev/sdb and path will become "/crash" and dump will be saved
- on "sdb:/crash/" directory.
-- path /var/crash/ (NFS mounted on /var)
- Say foo.com:/export/tmp is mounted on /var. In this case dump target is
- nfs server and path will be adjusted to "/crash" and dump will be saved to
- foo.com:/export/tmp/crash/ directory.
+Advanced Setups +===============
Kdump boot directory
Hi Kairui,
On 05/12/20 at 11:54pm, Kairui Song wrote:
Signed-off-by: Kairui Song kasong@redhat.com
kexec-kdump-howto.txt | 195 +++++++++++++++++++++++++++--------------- 1 file changed, 127 insertions(+), 68 deletions(-)
diff --git a/kexec-kdump-howto.txt b/kexec-kdump-howto.txt index 0d2b6cb..06dfbcf 100644 --- a/kexec-kdump-howto.txt +++ b/kexec-kdump-howto.txt @@ -283,11 +283,8 @@ to initate the dump and then click "Restart blade with NMI". This issues a system reset and invokes xmon debugger.
-Advanced Setups
Dump targets
+============
In addition to being able to capture a vmcore to your system's local file system, kdump can be configured to capture a vmcore to a number of other @@ -295,7 +292,8 @@ locations, including a raw disk partition, a dedicated file system, an NFS mounted file system, or a remote system via ssh/scp. Additional options exist for specifying the relative path under which the dump is captured, what to do if the capture fails, and for compressing and filtering the dump -(so as to produce smaller, more manageable, vmcore files). +(so as to produce smaller, more manageable, vmcore files, see "Advanced Setups" +for more detail on these options).
In theory, dumping to a location other than the local file system should be safer than kdump's default setup, as its possible the default setup will try @@ -308,31 +306,131 @@ as allowing for the centralization of vmcore files, should you have several systems from which you'd like to obtain vmcore files. Of course, note that these configurations could present problems if your network is unreliable.
-Advanced setups are configured via modifications to /etc/kdump.conf, -which out of the box, is fairly well documented itself. Any alterations to -/etc/kdump.conf should be followed by a restart of the kdump service, so -the changes can be incorporated in the kdump initrd. Restarting the kdump -service is as simple as '/sbin/systemctl restart kdump.service'.
-Note that kdump.conf is used as a configuration mechanism for capturing dump -files from the initramfs (in the interests of safety), the root file system is -mounted, and the init process is started, only as a last resort if the -initramfs fails to capture the vmcore. As such, configuration made in -/etc/kdump.conf is only applicable to capture recorded in the initramfs. If -for any reason the init process is started on the root file system, only a -simple copying of the vmcore from /proc/vmcore to /var/crash/$DATE/vmcore will -be preformed.
-For both local filesystem and nfs dump the dump target must be mounted before -building kdump initramfs. That means one needs to put an entry for the dump -file system in /etc/fstab so that after reboot when kdump service starts, -it can find the dump target and build initramfs instead of failing. +Kdump target and advanced setups are configured via modifications to +/etc/kdump.conf, which out of the box, is fairly well documented itself. +Any alterations to /etc/kdump.conf should be followed by a restart of the +kdump service, so the changes can be incorporated in the kdump initrd. +Restarting the kdump service is as simple as '/sbin/systemctl restart kdump.service'.
+There are two ways to config the dump target, config dump target only +using "path", and config dump target explicitely. Interpretation of "path" +also differs in two config styples.
+Config dump target only using "path" +------------------------------------
+You can change the dump target by setting "path" to a mount point where +dump target is mounted. When there is no explicitly configured dump target, +"path" in kdump.conf represents the current file system path in which vmcore +will be saved. Kdump will automatically detect the underlying device of +"path" and use that as the dump target.
+In fact, upon dump, kdump creates a directory $hostip-$date with-in "path" +and saves vmcore there. So practically dump is saved in $path/$hostip-$date/.
+Kdump will only check current mount status for mount entry corresponding to +"path". So please ensure the dump target is mounted on "path" before kdump +service starts.
+NOTES:
+- It's strongly recommanded to put an mount entry for "path" in /etc/fstab
- and have it auto mounted on boot. This make sure the dump target is
- reachable from the machine and kdump's configuration is stable.
+EXAMPLES:
+- path /var/crash/
- This is the default configuration. Assuming there is no disk mounted
- on /var/ or on /var/crash, dump will be saved on disk backing rootfs
- in directory /var/crash.
+- path /var/crash/ (A separate disk mounted on /var/crash)
- Say a disk /dev/sdb is mounted on /var. In this case dump target will
- become /dev/sdb and path will become "/" and dump will be saved
- on "sdb:/var/crash/" directory.
+- path /var/crash/ (NFS mounted on /var)
- Say foo.com:/export/tmp is mounted on /var. In this case dump target is
- nfs server and path will be adjusted to "/crash" and dump will be saved to
- foo.com:/export/tmp/crash/ directory.
+Config dump target explicitely +------------------------------
+You can set the dump target explicitly in kdump.conf, and "path" will be +the relative path in the specified dump target. For example, if dump +target is "ext4 /dev/sda", then dump will be saved in "path" directory +on /dev/sda.
+Same is the case for nfs dump. If user specified "nfs foo.com:/export/tmp/" +as dump target, then dump will effectively be saved in +"foo.com:/export/tmp/var/crash/" directory.
+If the dump target is "raw", then "path" is ignored.
+If it's a filesystemd target, kdump will need to know the right mount option. +Kdump will check current mount status, and then /etc/fstab for mount options +corresponding to the specified dump target and use it. If there are +special mount option required for the dump target, it could be set by put +an entry in fstab.
+If there are no related mount entry, mount option is set to "defaults".
+NOTES:
+- It's recommended to put an entry for the dump target in /etc/fstab
- and have it auto mounted on boot. This make sure the dump target is
- reachable from the machine and kdump won't fail.
+- Kdump ignores some mount options, including "noauto", "ro". This
- make it possible to keep the dump target unmounted or read-only
- when not used.
+EXAMPLES:
+- ext4 /dev/sda (mounted)
- path /var/crash/
- In this case dump target is set to /dev/sdb, path is the absolute path
- "/var/crash" in /dev/sda, vmcore path will saved on
- "sda:/var/crash" directory.
+- nfs foo.com:/export/tmp (mounted)
- path /var/crash/
- In this case dump target is nfs server, path is the absolute path
- "/var/crash", vmcore path will saved on "foo.com:/export/tmp/crash/" directory.
+- nfs foo.com:/export/tmp (not mounted)
- path /var/crash/
- Same with above case, kdump will use "defaults" as the mount option
- for the dump target.
+- nfs foo.com:/export/tmp (not mounted, entry with option "noauto,nolock" exists in /etc/fstab)
- path /var/crash/
- In this case dump target is nfs server, vmcore path will saved on
- "foo.com:/export/tmp/crash/" directory, and kdump will inherit "nolock" option.
+Dump target and mkdumprd +------------------------
+MKdumprd is the tool used to create kdump initramfs, and it may change +the mount status of the dump target in some condition.
Usually the dump target should be used only for kdump. If you worry about someone uses the filesystem for something else other than dumping vmcore -you can mount it as read-only. Mkdumprd will still remount it as read-write -for creating dump directory and will move it back to read-only afterwards. +you can mount it as read-only or make it a noauto mount. Mkdumprd will +mount/remount it as read-write for creating dump directory and will +move it back to it's original state afterwards.
-Followings are the available config options for dump targets. +Supported dump target types and requirements +--------------------------------------------
- Raw partition
@@ -423,47 +521,8 @@ you've connected to it, and then input the target system user's password to send over the necessary ssh key file. Restart the kdump service via '/sbin/systemctl restart kdump.service' to commit this change to your kdump initrd.
-Path
-"path" represents the file system path in which vmcore will be saved. In -fact kdump creates a directory $hostip-$date with-in "path" and saves -vmcore there. So practically dump is saved in $path/$hostip-$date/. To -simplify discussion further, if we say dump will be saved in $path, it -is implied that kdump will create another directory inside path and -save vmcore there.
-If a dump target is specified in kdump.conf, then "path" is relative to the -specified dump target. For example, if dump target is "ext4 /dev/sda", then -dump will be saved in "$path" directory on /dev/sda.
-Same is the case for nfs dump. If user specified "nfs foo.com:/export/tmp/" -as dump target, then dump will effectively be saved in -"foo.com:/export/tmp/var/crash/" directory.
-Interpretation of path changes a bit if user has not specified a dump -target explicitly in kdump.conf. In this case, "path" represents the -absolute path from root. And dump target and adjusted path are arrived -at automatically depending on what's mounted in the current system.
-Following are few examples.
-- path /var/crash/
- Assuming there is no disk mounted on /var/ or on /var/crash, dump will
- be saved on disk backing rootfs in directory /var/crash.
-- path /var/crash/ (A separate disk mounted on /var)
- Say a disk /dev/sdb is mouted on /var. In this case dump target will
- become /dev/sdb and path will become "/crash" and dump will be saved
- on "sdb:/crash/" directory.
-- path /var/crash/ (NFS mounted on /var)
- Say foo.com:/export/tmp is mounted on /var. In this case dump target is
- nfs server and path will be adjusted to "/crash" and dump will be saved to
- foo.com:/export/tmp/crash/ directory.
+Advanced Setups +===============
Kdump boot directory
Late reply :) Can you also update below part, maybe in another appending patch.
One use case of "--mount" in "dracut_args" is you do not want to mount dump target before kdump service startup, for example, to reduce the burden of the shared nfs server. Such as the example below: dracut_args --mount "192.168.1.1:/share /mnt/test nfs4 defaults"
Since we have the new changes, it is not necessary to use above, at least the words can be tuned a bit.
Thanks Dave
On 05/12/2020 11:54 PM, Kairui Song wrote:
Currently kexec-tools always depend on dump target to be mounted, which caused some inconvenience for setup.
So for user configured target, allow kdump to start and build initramfs even if target is not mounted.
Behavior is not changed for "path" based dump target config.
When a unmounted dump target is explcitely configured, mkdumprd will look for corresponding mount info in fstab for the mount options, and failback to use "defaults" as mount option.
Then mkdumprd will try to mount it and do basic checks on the device, then umount it on exit.
NOTE: If there is a fstab entry but "noauto" option is not used, then there must be some reason that the target device is not mounted, mkdumprd will error out and ask user to double check.
Update from v1: Rebase and add some document.
Kairui Song (7): Add a is_mounted helper Allow calling mkdumprd from kdumpctl even if targat not mounted kdump-lib.sh: add fstab failback helper for getting mount info User get_mount_info to replace findmnt calls mkdumprd: generate usable kdump initramfs even target is not mounted kexec-kdump-howto.txt: Add some format to the document Update docs for the new noauto dump target support
kdump-lib-initramfs.sh | 23 ++- kdump-lib.sh | 58 ++++--- kdumpctl | 9 +- kexec-kdump-howto.txt | 363 ++++++++++++++++++++++++++--------------- mkdumprd | 130 +++++++++++---- 5 files changed, 371 insertions(+), 212 deletions(-)
I think you can correct 7/7 in local repo, then for this series, Acked-by: Pingfan Liu piliu@redhat.com
Yes, I agree with the fix on 7/7, thanks.
On Thu, May 21, 2020 at 7:39 PM piliu piliu@redhat.com wrote:
On 05/12/2020 11:54 PM, Kairui Song wrote:
Currently kexec-tools always depend on dump target to be mounted, which caused some inconvenience for setup.
So for user configured target, allow kdump to start and build initramfs even if target is not mounted.
Behavior is not changed for "path" based dump target config.
When a unmounted dump target is explcitely configured, mkdumprd will look for corresponding mount info in fstab for the mount options, and failback to use "defaults" as mount option.
Then mkdumprd will try to mount it and do basic checks on the device, then umount it on exit.
NOTE: If there is a fstab entry but "noauto" option is not used, then there must be some reason that the target device is not mounted, mkdumprd will error out and ask user to double check.
Update from v1: Rebase and add some document.
Kairui Song (7): Add a is_mounted helper Allow calling mkdumprd from kdumpctl even if targat not mounted kdump-lib.sh: add fstab failback helper for getting mount info User get_mount_info to replace findmnt calls mkdumprd: generate usable kdump initramfs even target is not mounted kexec-kdump-howto.txt: Add some format to the document Update docs for the new noauto dump target support
kdump-lib-initramfs.sh | 23 ++- kdump-lib.sh | 58 ++++--- kdumpctl | 9 +- kexec-kdump-howto.txt | 363 ++++++++++++++++++++++++++--------------- mkdumprd | 130 +++++++++++---- 5 files changed, 371 insertions(+), 212 deletions(-)
I think you can correct 7/7 in local repo, then for this series, Acked-by: Pingfan Liu piliu@redhat.com
-
Dave Young -
Kairui Song -
piliu