It seems like more and more problems with PCI MSI are turning up in the 2.6.20 kernel. Discussion upstream concluded that maybe it should have been off by default in 2.6.20, so maybe we should just do that in Fedora and make people who want it use "pci=msi" to enable it? It's probably not going to be really stable until 2.6.22...
Chuck Ebbert wrote:
It seems like more and more problems with PCI MSI are turning up in the 2.6.20 kernel. Discussion upstream concluded that maybe it should have been off by default in 2.6.20, so maybe we should just do that in Fedora and make people who want it use "pci=msi" to enable it? It's probably not going to be really stable until 2.6.22...
Was "upstream discussion" Willy? Interesting.
Jon.
Chuck Ebbert wrote:
It seems like more and more problems with PCI MSI are turning up in the 2.6.20 kernel. Discussion upstream concluded that maybe it should have been off by default in 2.6.20, so maybe we should just do that in Fedora and make people who want it use "pci=msi" to enable it? It's probably not going to be really stable until 2.6.22...
I vote yes: turn it off by default. We just ran into an MSI issue on a particular Via motherboard (Asus M2V) that runs the network driver (atl1) I help maintain. The kernel blows up when we start the network unless we turn off MSI.
Jay Cliburn wrote:
Chuck Ebbert wrote:
It seems like more and more problems with PCI MSI are turning up in the 2.6.20 kernel. Discussion upstream concluded that maybe it should have been off by default in 2.6.20, so maybe we should just do that in Fedora and make people who want it use "pci=msi" to enable it? It's probably not going to be really stable until 2.6.22...
I vote yes: turn it off by default. We just ran into an MSI issue on a particular Via motherboard (Asus M2V) that runs the network driver (atl1) I help maintain. The kernel blows up when we start the network unless we turn off MSI.
One of our bug reporters points to a unbuntu thread where they mention they have just turned of msi by default.
https://launchpad.net/ubuntu/+source/linux-source-2.6.20/+bug/74830
On Fri, Mar 30, 2007 at 01:15:28PM -0400, Chuck Ebbert wrote:
Jay Cliburn wrote:
Chuck Ebbert wrote:
It seems like more and more problems with PCI MSI are turning up in the 2.6.20 kernel. Discussion upstream concluded that maybe it should have been off by default in 2.6.20, so maybe we should just do that in Fedora and make people who want it use "pci=msi" to enable it? It's probably not going to be really stable until 2.6.22...
I vote yes: turn it off by default. We just ran into an MSI issue on a particular Via motherboard (Asus M2V) that runs the network driver (atl1) I help maintain. The kernel blows up when we start the network unless we turn off MSI.
One of our bug reporters points to a unbuntu thread where they mention they have just turned of msi by default.
Yeah, we've also turned off MMCONFIG by default. We've seen way too many bugs that go away when they're disabled.
Cheers, Kyle
On Fri, Mar 30, 2007 at 03:04:39PM -0400, Kyle McMartin wrote:
On Fri, Mar 30, 2007 at 01:15:28PM -0400, Chuck Ebbert wrote:
Jay Cliburn wrote:
Chuck Ebbert wrote:
It seems like more and more problems with PCI MSI are turning up in the 2.6.20 kernel. Discussion upstream concluded that maybe it should have been off by default in 2.6.20, so maybe we should just do that in Fedora and make people who want it use "pci=msi" to enable it? It's probably not going to be really stable until 2.6.22...
I vote yes: turn it off by default. We just ran into an MSI issue on a particular Via motherboard (Asus M2V) that runs the network driver (atl1) I help maintain. The kernel blows up when we start the network unless we turn off MSI.
One of our bug reporters points to a unbuntu thread where they mention they have just turned of msi by default.
Yeah, we've also turned off MMCONFIG by default. We've seen way too many bugs that go away when they're disabled.
Isn't that needed though to access higher config space on PCIE ? Or do we not have any drivers that use that yet, making it a nonissue?
Dave
On Fri, Mar 30, 2007 at 03:09:08PM -0400, Dave Jones wrote:
Isn't that needed though to access higher config space on PCIE ? Or do we not have any drivers that use that yet, making it a nonissue?
Yeah, you need it to access Extended config space on PCI-Express, but the only people who need it right now are Infinibarf, and I just provided pci=mmconf to turn it back on.
http://git.kernel.org/?p=linux/kernel/git/bcollins/ubuntu-2.6.git;a=commitdi...
On Fri, 30 Mar 2007 11:00:12 -0400, Chuck Ebbert cebbert@redhat.com wrote:
It seems like more and more problems with PCI MSI are turning up in the 2.6.20 kernel. Discussion upstream concluded that maybe it should have been off by default in 2.6.20, so maybe we should just do that in Fedora and make people who want it use "pci=msi" to enable it? It's probably not going to be really stable until 2.6.22...
I have a laptop (Dell 1501) where hard drive does not work unless MSI is off. It's not hard to add pci=nomsi, and Grubby copies it to new kernel's command line, but the downside is, it took me a day to figure out what was wrong. It completely looked like a garden variety SATA failure. Maybe we want to keep it on in Rawhide and publicise "look at /proc/interrupts, check if count is zero"?
-- Pete
Pete Zaitcev wrote:
On Fri, 30 Mar 2007 11:00:12 -0400, Chuck Ebbert cebbert@redhat.com wrote:
It seems like more and more problems with PCI MSI are turning up in the 2.6.20 kernel. Discussion upstream concluded that maybe it should have been off by default in 2.6.20, so maybe we should just do that in Fedora and make people who want it use "pci=msi" to enable it? It's probably not going to be really stable until 2.6.22...
I have a laptop (Dell 1501) where hard drive does not work unless MSI is off. It's not hard to add pci=nomsi, and Grubby copies it to new kernel's command line, but the downside is, it took me a day to figure out what was wrong. It completely looked like a garden variety SATA failure. Maybe we want to keep it on in Rawhide and publicise "look at /proc/interrupts, check if count is zero"?
I was thinking more about FC6 where people are upgrading to 2.6.20 and finding systems don't work that used to work. For FC7 it's debatable since MSI is mostly fixed in 2.6.21, but I think I will switch it off in FC6.
On Fri, Mar 30, 2007 at 02:03:23PM -0400, Chuck Ebbert wrote:
Pete Zaitcev wrote:
On Fri, 30 Mar 2007 11:00:12 -0400, Chuck Ebbert cebbert@redhat.com wrote:
It seems like more and more problems with PCI MSI are turning up in the 2.6.20 kernel. Discussion upstream concluded that maybe it should have been off by default in 2.6.20, so maybe we should just do that in Fedora and make people who want it use "pci=msi" to enable it? It's probably not going to be really stable until 2.6.22...
I have a laptop (Dell 1501) where hard drive does not work unless MSI is off. It's not hard to add pci=nomsi, and Grubby copies it to new kernel's command line, but the downside is, it took me a day to figure out what was wrong. It completely looked like a garden variety SATA failure. Maybe we want to keep it on in Rawhide and publicise "look at /proc/interrupts, check if count is zero"?
I was thinking more about FC6 where people are upgrading to 2.6.20 and finding systems don't work that used to work. For FC7 it's debatable since MSI is mostly fixed in 2.6.21, but I think I will switch it off in FC6.
Gets my vote too. I've turned off CONFIG_PCI_MSI and turned it back on about 2-3 times now for FC5/FC6, because each time it starts to look more promising, it seems to find new ways to regress.
I might do a build next week in rawhide with it off again too, to see if any oddball bugs fall out.
Dave
Dave Jones wrote:
Gets my vote too. I've turned off CONFIG_PCI_MSI and turned it back on about 2-3 times now for FC5/FC6, because each time it starts to look more promising, it seems to find new ways to regress.
I might do a build next week in rawhide with it off again too, to see if any oddball bugs fall out.
I was thinking of leaving MSI enabled and applying this patch.
Disable MSI by default.
--- Documentation/kernel-parameters.txt | 4 ++-- drivers/pci/msi.c | 6 +++--- drivers/pci/pci.c | 4 ++-- 3 files changed, 7 insertions(+), 7 deletions(-)
--- linux-2.6.20.noarch.orig/drivers/pci/msi.c +++ linux-2.6.20.noarch/drivers/pci/msi.c @@ -28,7 +28,7 @@ static DEFINE_SPINLOCK(msi_lock); static struct msi_desc* msi_desc[NR_IRQS] = { [0 ... NR_IRQS-1] = NULL }; static struct kmem_cache* msi_cachep;
-static int pci_msi_enable = 1; +static int pci_msi_enable = 0;
static int msi_cache_init(void) { @@ -973,9 +973,9 @@ void msi_remove_pci_irq_vectors(struct p } }
-void pci_no_msi(void) +void pci_yes_msi(void) { - pci_msi_enable = 0; + pci_msi_enable = 1; }
EXPORT_SYMBOL(pci_enable_msi); --- linux-2.6.20.noarch.orig/drivers/pci/pci.c +++ linux-2.6.20.noarch/drivers/pci/pci.c @@ -1166,8 +1166,8 @@ static int __devinit pci_setup(char *str if (k) *k++ = 0; if (*str && (str = pcibios_setup(str)) && *str) { - if (!strcmp(str, "nomsi")) { - pci_no_msi(); + if (!strcmp(str, "msi")) { + pci_yes_msi(); } else { printk(KERN_ERR "PCI: Unknown option `%s'\n", str); --- linux-2.6.20.noarch.orig/Documentation/kernel-parameters.txt +++ linux-2.6.20.noarch/Documentation/kernel-parameters.txt @@ -1199,9 +1199,9 @@ and is between 256 and 4096 characters. Mechanism 2. nommconf [IA-32,X86_64] Disable use of MMCONFIG for PCI Configuration - nomsi [MSI] If the PCI_MSI kernel config parameter is + msi [MSI] If the PCI_MSI kernel config parameter is enabled, this kernel boot option can be used to - disable the use of MSI interrupts system-wide. + enable the use of MSI interrupts system-wide. nosort [IA-32] Don't sort PCI devices according to order given by the PCI BIOS. This sorting is done to get a device order compatible with
On Fri, Mar 30, 2007 at 02:47:28PM -0400, Chuck Ebbert wrote:
Dave Jones wrote:
Gets my vote too. I've turned off CONFIG_PCI_MSI and turned it back on about 2-3 times now for FC5/FC6, because each time it starts to look more promising, it seems to find new ways to regress.
I might do a build next week in rawhide with it off again too, to see if any oddball bugs fall out.
I was thinking of leaving MSI enabled and applying this patch.
looks good to me
Dave
Chuck Ebbert wrote:
Dave Jones wrote:
Gets my vote too. I've turned off CONFIG_PCI_MSI and turned it back on about 2-3 times now for FC5/FC6, because each time it starts to look more promising, it seems to find new ways to regress.
I might do a build next week in rawhide with it off again too, to see if any oddball bugs fall out.
I was thinking of leaving MSI enabled and applying this patch.
Ended up applying the attached patch.
Disable PCI MSI and MMCONFIG by default, add kernel parameters to enable them.
Original mmconfig patch by Kyle McMartin kyle@ubuntu.com
Signed-off-by: Chuck Ebbert cebbert@redhat.com
--- Documentation/kernel-parameters.txt | 5 +++++ arch/i386/pci/common.c | 6 +++++- drivers/pci/msi.c | 6 +++++- drivers/pci/pci.c | 2 ++ drivers/pci/pci.h | 2 ++ 5 files changed, 19 insertions(+), 2 deletions(-)
--- linux-2.6.20.noarch.orig/drivers/pci/msi.c +++ linux-2.6.20.noarch/drivers/pci/msi.c @@ -28,7 +28,7 @@ static DEFINE_SPINLOCK(msi_lock); static struct msi_desc* msi_desc[NR_IRQS] = { [0 ... NR_IRQS-1] = NULL }; static struct kmem_cache* msi_cachep;
-static int pci_msi_enable = 1; +static int pci_msi_enable = 0;
static int msi_cache_init(void) { @@ -977,6 +977,10 @@ void pci_no_msi(void) { pci_msi_enable = 0; } +void pci_yes_msi(void) +{ + pci_msi_enable = 1; +}
EXPORT_SYMBOL(pci_enable_msi); EXPORT_SYMBOL(pci_disable_msi); --- linux-2.6.20.noarch.orig/drivers/pci/pci.c +++ linux-2.6.20.noarch/drivers/pci/pci.c @@ -1168,6 +1168,8 @@ static int __devinit pci_setup(char *str if (*str && (str = pcibios_setup(str)) && *str) { if (!strcmp(str, "nomsi")) { pci_no_msi(); + } else if (!strcmp(str, "msi")) { + pci_yes_msi(); } else { printk(KERN_ERR "PCI: Unknown option `%s'\n", str); --- linux-2.6.20.noarch.orig/Documentation/kernel-parameters.txt +++ linux-2.6.20.noarch/Documentation/kernel-parameters.txt @@ -1197,8 +1197,13 @@ and is between 256 and 4096 characters. Mechanism 1. conf2 [IA-32] Force use of PCI Configuration Mechanism 2. + mmconf [IA-32,X86_64] Enable use of MMCONFIG for PCI + Configuration nommconf [IA-32,X86_64] Disable use of MMCONFIG for PCI Configuration + msi [MSI] If the PCI_MSI kernel config parameter is + enabled, this kernel boot option can be used to + enable the use of MSI interrupts system-wide. nomsi [MSI] If the PCI_MSI kernel config parameter is enabled, this kernel boot option can be used to disable the use of MSI interrupts system-wide. --- linux-2.6.20.noarch.orig/arch/i386/pci/common.c +++ linux-2.6.20.noarch/arch/i386/pci/common.c @@ -18,7 +18,7 @@ #include "pci.h"
unsigned int pci_probe = PCI_PROBE_BIOS | PCI_PROBE_CONF1 | PCI_PROBE_CONF2 | - PCI_PROBE_MMCONF; + 0; /* PCI_PROBE_MMCONF */
static int pci_bf_sort; int pci_routeirq; @@ -292,6 +292,10 @@ char * __devinit pcibios_setup(char *st pci_probe &= ~PCI_PROBE_MMCONF; return NULL; } + else if (!strcmp(str, "mmconf")) { + pci_probe |= PCI_PROBE_MMCONF; + return NULL; + } #endif else if (!strcmp(str, "noacpi")) { acpi_noirq_set(); --- linux-2.6.20.noarch.orig/drivers/pci/pci.h +++ linux-2.6.20.noarch/drivers/pci/pci.h @@ -52,9 +52,11 @@ extern unsigned int pci_pm_d3_delay; #ifdef CONFIG_PCI_MSI void disable_msi_mode(struct pci_dev *dev, int pos, int type); void pci_no_msi(void); +void pci_yes_msi(void); #else static inline void disable_msi_mode(struct pci_dev *dev, int pos, int type) { } static inline void pci_no_msi(void) { } +static inline void pci_yes_msi(void) { } #endif #if defined(CONFIG_PCI_MSI) && defined(CONFIG_PM) int pci_save_msi_state(struct pci_dev *dev);
kernel@lists.fedoraproject.org
-
Chuck Ebbert -
Dave Jones -
J. K. Cliburn -
Jon Masters -
Kyle McMartin -
Pete Zaitcev