On Sun, May 3, 2015 at 5:25 AM, Tomáš Trnka tomastrnka@gmx.com wrote:

Hello,

I have a PCI-based SATA controller (Sil3124) that chokes on PCI DAC addressing, so I have to force 32-bit addressing through an IOMMU. Sadly, the stock Fedora kernel falls back to SWIOTLB on this machine, which is both slower (though I haven't benchmarked it) and unreliable (several times I've seen the SWIOTLB overflow and go boom).

Apparently, all I need is the AMD GART-based IOMMU implementation (CONFIG_GART_IOMMU) that is currently disabled by Fedora. This means I currently have to rebuild the kernel every time to re-enable it.

GART_IOMMU is the really old IOMMU found in older Athlon64/Opteron CPUs. Most modern AMD CPUS use AMD_IOMMU, which is enabled. Do you have one of these older CPUs?

Is there any particular reason why this is disabled in Fedora (upstream Kconfig default is enabled)? Given that there is probably a non-negligible fraction of machines that could benefit from the GART IOMMU and the kernel size impact is minimal, would it be possible to restore the upstream default?

Upstream doesn't include it in the x86_64_defconfig and it isn't default y. The Kconfig help text says to enable if unsure, but if you look at the commit log where this option was last changed, it's pretty discouraging.

josh

commit 38901f1c1cae241fd4e3c5e63fcc690b921734b1 Author: Andi Kleen ak@linux.intel.com Date: Fri Oct 4 14:37:56 2013 -0700

x86/iommu: Don't make AMD_GART depend on EXPERT and default y

The AMD_GART driver was made EXPERT/EMBEDDED a long time ago to avoid unbootable 64bit systems with 32bit only devices.

This was before swiotlb was there, which does the job of this fallback today. SWIOTLB is always on, so systems should always boot.

The drawback is that every system has to compile that driver in (it cannot be a module).

Also: - Newer AMD CPUs (the APUs) don't seem to have AMD_GART support at all anymore.

- Newer AMD platforms have a much better real IOMMU

- The AMD GART driver was never very good (lots of overhead, e.g. in flushing due to some workarounds) and it's doubtful it's really better than SWIOTLB.

- On older K8 systems it didn't even work with all chipsets.

- The 32bit device bounce buffer case should be rare/ non performance critical these days anyways.

- On non AMD systems it is not needed at all.

So drop the EXPERT dependency on AMD_GART and remove the default y. The driver can be still compiled in, just it's an explicit decision now, and people who don't want it can unselect it.

I also clarified the description a bit.

This allows to save ~8K text on most modern x86-64 systems.

Signed-off-by: Andi Kleen ak@linux.intel.com Acked-by: Borislav Petkov bp@suse.de Link: http://lkml.kernel.org/r/1380922676-23007-1-git-send-email-andi@firstfloor.o... Signed-off-by: Ingo Molnar mingo@kernel.org