This patchset adds an aarch64 specific kdump.sysconfig and tries to make it as similar to x86_64 as possible.
Note that not all x86_64 kdump config options make sense for aarch64 architecture, so we will not include those (see PATCH 2/2 for details).
Bhupesh Sharma (2): Add aarch64 specific kdump.sysconfig and use 'nr_cpus' instead of 'maxcpus' aarch64/kdump.sysconfig: Make config options similar to x86_64
kdump.sysconfig.aarch64 | 37 +++++++++++++++++++++++++++++++++++++ kexec-tools.spec | 1 + 2 files changed, 38 insertions(+) create mode 100644 kdump.sysconfig.aarch64
'maxcpus' setting normally don't work on several kdump enabled systems due to a known udev issue.
Currently the fedora kdump configuration is set as the following on the aarch64 systems:
# cat /etc/sysconfig/kdump <..snip..> # This variable lets us append arguments to the current kdump # commandline after processed by KDUMP_COMMANDLINE_REMOVE # KDUMP_COMMANDLINE_APPEND="irqpoll maxcpus=1 reset_devices" <..snip..>
Since the 'maxcpus' setting doesn't limit the number of SMP CPUs, so the kdump kernel still boots with all CPUs available on the system. For e.g on the qualcomm amberwing its 46 CPUs:
# lscpu Architecture: aarch64 Byte Order: Little Endian CPU(s): 46 On-line CPU(s) list: 0-45 Thread(s) per core: 1 Core(s) per socket: 46 Socket(s): 1 NUMA node(s): 1 Vendor ID: Qualcomm Model: 1 Model name: Falkor Stepping: 0x0 CPU max MHz: 2600.0000 CPU min MHz: 600.0000 BogoMIPS: 40.00 L1d cache: 32K L1i cache: 64K L2 cache: 512K L3 cache: 58880K NUMA node0 CPU(s): 0-45 Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 cpuid asimdrdm
This causes the memory consumption in the kdump kernel to swell up and we can end up having OOM issues in the kdump kernel boot.
Whereas if we use 'nr_cpus=1' in the bootargs, the number of SMP CPUs in the kdump kernel get limited to 1.
The 'swiotlb=noforce' setting in bootargs provide us extra guarding, to ensure the crash kernel size requirements do not swell on systems which support swiotlb.
With the above settings, crashkernel boots properly (without OOM) on all the aarch64 boards I could test on - qualcomm amberwings, hp-moonshots and hpe-apache (thunderx2) for crash dump saving on local disk.
Signed-off-by: Bhupesh Sharma bhsharma@redhat.com --- kdump.sysconfig.aarch64 | 37 +++++++++++++++++++++++++++++++++++++ kexec-tools.spec | 1 + 2 files changed, 38 insertions(+) create mode 100644 kdump.sysconfig.aarch64
diff --git a/kdump.sysconfig.aarch64 b/kdump.sysconfig.aarch64 new file mode 100644 index 000000000000..0a6b14c8bd37 --- /dev/null +++ b/kdump.sysconfig.aarch64 @@ -0,0 +1,37 @@ +# Kernel Version string for the -kdump kernel, such as 2.6.13-1544.FC5kdump +# If no version is specified, then the init script will try to find a +# kdump kernel with the same version number as the running kernel. +KDUMP_KERNELVER="" + +# The kdump commandline is the command line that needs to be passed off to +# the kdump kernel. This will likely match the contents of the grub kernel +# line. For example: +# KDUMP_COMMANDLINE="ro root=LABEL=/" +# Dracut depends on proper root= options, so please make sure that appropriate +# root= options are copied from /proc/cmdline. In general it is best to append +# command line options using "KDUMP_COMMANDLINE_APPEND=". +# If a command line is not specified, the default will be taken from +# /proc/cmdline +KDUMP_COMMANDLINE="" + +# This variable lets us remove arguments from the current kdump commandline +# as taken from either KDUMP_COMMANDLINE above, or from /proc/cmdline +# NOTE: some arguments such as crashkernel will always be removed +KDUMP_COMMANDLINE_REMOVE="hugepages hugepagesz slub_debug quiet" + +# This variable lets us append arguments to the current kdump commandline +# after processed by KDUMP_COMMANDLINE_REMOVE +KDUMP_COMMANDLINE_APPEND="irqpoll nr_cpus=1 swiotlb=noforce reset_devices" + +# Any additional kexec arguments required. In most situations, this should +# be left empty +# +# Example: +# KEXEC_ARGS="--elf32-core-headers" +KEXEC_ARGS="" + +#Where to find the boot image +#KDUMP_BOOTDIR="/boot" + +#What is the image type used for kdump +KDUMP_IMG="vmlinuz" diff --git a/kexec-tools.spec b/kexec-tools.spec index 67450f87ad14..b19b5342df24 100644 --- a/kexec-tools.spec +++ b/kexec-tools.spec @@ -33,6 +33,7 @@ Source25: kdumpctl.8 Source26: live-image-kdump-howto.txt Source27: early-kdump-howto.txt Source28: kdump-udev-throttler +Source29: kdump.sysconfig.aarch64
####################################### # These are sources for mkdumpramfs
Looking at the difference of the x86_64 and aarch64 kdump.sysconfig options for Fedora, one can see the following options which are different:
Present in kdump.sysconfig.x86_64 but not in kdump.sysconfig.aarch64: --------------------------------------------------------------------- cgroup_disable=memory mce=off numa=off udev.children-max=2 panic=10 acpi_no_memhotplug transparent_hugepage=never nokaslr
Present in kdump.sysconfig.aarch64 but not in kdump.sysconfig.x86_64: --------------------------------------------------------------------- swiotlb=noforce
After going through all the options, it makes sense to add the following options added to kdump.sysconfig.aarch64:
KDUMP_COMMANDLINE_APPEND="cgroup_disable=memory udev.children-max=2 panic=10 irqpoll nr_cpus=1 swiotlb=noforce reset_devices"
This has helped reduce the memory footprint of crashkernel on several aarch64 machines available in the beaker lab. For e.g. I was seeing OOM issues on large aws ec2 instances with the default crashkernel size of 512M, and I had to use an increased crashkernel size of 786M on the same to boot the crash dump kernel.
Signed-off-by: Bhupesh Sharma bhsharma@redhat.com --- kdump.sysconfig.aarch64 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/kdump.sysconfig.aarch64 b/kdump.sysconfig.aarch64 index 0a6b14c8bd37..c72a0406fa01 100644 --- a/kdump.sysconfig.aarch64 +++ b/kdump.sysconfig.aarch64 @@ -21,7 +21,7 @@ KDUMP_COMMANDLINE_REMOVE="hugepages hugepagesz slub_debug quiet"
# This variable lets us append arguments to the current kdump commandline # after processed by KDUMP_COMMANDLINE_REMOVE -KDUMP_COMMANDLINE_APPEND="irqpoll nr_cpus=1 swiotlb=noforce reset_devices" +KDUMP_COMMANDLINE_APPEND="irqpoll nr_cpus=1 reset_devices cgroup_disable=memory udev.children-max=2 panic=10 swiotlb=noforce"
# Any additional kexec arguments required. In most situations, this should # be left empty
On 05/27/2019 07:29 PM, Bhupesh Sharma wrote:
This patchset adds an aarch64 specific kdump.sysconfig and tries to make it as similar to x86_64 as possible.
Note that not all x86_64 kdump config options make sense for aarch64 architecture, so we will not include those (see PATCH 2/2 for details).
Bhupesh Sharma (2): Add aarch64 specific kdump.sysconfig and use 'nr_cpus' instead of 'maxcpus' aarch64/kdump.sysconfig: Make config options similar to x86_64
kdump.sysconfig.aarch64 | 37 +++++++++++++++++++++++++++++++++++++ kexec-tools.spec | 1 + 2 files changed, 38 insertions(+) create mode 100644 kdump.sysconfig.aarch64
For the series, Acked-by: Pingfan Liu piliu@redhat.com