adding desktop@ since they are also looking at file system options
On Feb 25, 2014, at 1:42 PM, Stephen Gallagher sgallagh@redhat.com wrote:
=== File system ===
The default file system type for workstation installs should be btrfs.
The default file system is definitely up for some debate, but I'd make an argument for using XFS atop LVM[1] for the default filesystem in the Fedora Server, at least in part because Red Hat's storage experts have done the research for us already and determined that XFS is the recommended fit for Red Hat Enterprise Linux 7.
XFS is a really good idea for Server.
Follow-up questions:
- Can Server and Workstation WG's choose different defaults for their product's installers?
- Other than lack of shrink support in XFS, I'd say XFS is suitable for Workstation as well. Would the Workstation WG have concerns about the lack of fs shrink support in the default file system? [1]
Btrfs still makes me somewhat nervous, given that its upstream doesn't consider it stable[3].
That wiki entry appears old. The stable aspect was about disk format, which is now stable. And also the experimental description was removed in kernel 3.13. [2]
My main two concerns with Btrfs: 1. With even minor problems users sometimes go straight to the big hammer approach with "btrfsck/btrfs check --repair" rather than the recommended approaches. Remounting with -o recovery, and even using a newer kernel are recommended on linux-btrfs@ significantly more often than btrfs check --repair.
It's a fair question how fail safe the offline repair utility currently is, and should be. In my monitoring of linux-btrfs@, even if btrfs check --repair made things worse, the offline btrfs restore utility enabled files to be extracted.
2. Supporting multiple device volumes in Anaconda. Although multiple device Btrfs volumes work very well when devices are working, there's no device failure notification yet. When a device fails, the volume becomes read-only; and the volume isn't mountable (or bootable) without the use of "degraded" mount option. While this can be done as a boot parameter, it's sort of a non-obvious thing for the typical user. I think it's valid for either WG to consider reducing exposure by dropping Anaconda support for it, or placarding the feature somehow.
[1] LVM Thin Provisioning, if ready for production prime time as a default, is a work around. It's actually better than fs resize anyway.
[2] http://www.mail-archive.com/linux-btrfs%40vger.kernel.org/msg29945.html
Chris Murphy
On Tue, 2014-02-25 at 15:53 -0700, Chris Murphy wrote:
adding desktop@ since they are also looking at file system options
On Feb 25, 2014, at 1:42 PM, Stephen Gallagher sgallagh@redhat.com wrote:
=== File system ===
The default file system type for workstation installs should be btrfs.
The default file system is definitely up for some debate, but I'd make an argument for using XFS atop LVM[1] for the default filesystem in the Fedora Server, at least in part because Red Hat's storage experts have done the research for us already and determined that XFS is the recommended fit for Red Hat Enterprise Linux 7.
XFS is a really good idea for Server.
Follow-up questions:
- Can Server and Workstation WG's choose different defaults for their
product's installers?
Given my understanding of anaconda's architecture I don't believe this would *technically* present a significant problem. anaconda already has the concept of being used to install different products, and using different defaults for various things depending on what product it's being used to install: this is how RHEL can have different defaults from Fedora.
It would be best to ask the anaconda devs, though. Maybe they think it's a horrible hack and don't want to extend it any further than their paychecks require. CCing bcl and dcantrell.
In terms of *policy*, it'd be up to FESCo, I guess. It seems like a perfectly reasonable point of variance between products to me.
- Other than lack of shrink support in XFS, I'd say XFS is suitable
for Workstation as well. Would the Workstation WG have concerns about the lack of fs shrink support in the default file system? [1]
Btrfs still makes me somewhat nervous, given that its upstream doesn't consider it stable[3].
That wiki entry appears old. The stable aspect was about disk format, which is now stable. And also the experimental description was removed in kernel 3.13. [2]
<snip>
In addition to Chris' points, we discussed btrfs at this week's QA meeting, and agreed that even though it's really not QA's 'job', it seems sensible to just check if Desktop WG has talked to the devs who have, up until now, been taking the job of deciding when btrfs is 'ready for primetime' and developed a plan. Is the btrfs-by-default part of the current tech spec more of a long term aspiration, or is it on the table for F21? Have the concerns about its readiness been evaluated and checked with the domain experts?
Thanks!
On Feb 25, 2014 6:04 PM, "Adam Williamson" awilliam@redhat.com wrote:
On Tue, 2014-02-25 at 15:53 -0700, Chris Murphy wrote:
adding desktop@ since they are also looking at file system options
On Feb 25, 2014, at 1:42 PM, Stephen Gallagher sgallagh@redhat.com
wrote:
=== File system ===
The default file system type for workstation installs should be btrfs.
The default file system is definitely up for some debate, but I'd make an argument for using XFS atop LVM[1] for the default filesystem in the Fedora Server, at least in part because Red Hat's storage experts have done the research for us already and determined that XFS is the recommended fit for Red Hat Enterprise Linux 7.
XFS is a really good idea for Server.
Follow-up questions:
- Can Server and Workstation WG's choose different defaults for their
product's installers?
Given my understanding of anaconda's architecture I don't believe this would *technically* present a significant problem. anaconda already has the concept of being used to install different products, and using different defaults for various things depending on what product it's being used to install: this is how RHEL can have different defaults from Fedora.
It would be best to ask the anaconda devs, though. Maybe they think it's a horrible hack and don't want to extend it any further than their paychecks require. CCing bcl and dcantrell.
In terms of *policy*, it'd be up to FESCo, I guess. It seems like a perfectly reasonable point of variance between products to me.
- Other than lack of shrink support in XFS, I'd say XFS is suitable
for Workstation as well. Would the Workstation WG have concerns about the lack of fs shrink support in the default file system? [1]
Btrfs still makes me somewhat nervous, given that its upstream doesn't consider it stable[3].
That wiki entry appears old. The stable aspect was about disk format, which is now stable. And also the experimental description was removed in kernel 3.13. [2]
<snip>
In addition to Chris' points, we discussed btrfs at this week's QA meeting, and agreed that even though it's really not QA's 'job', it seems sensible to just check if Desktop WG has talked to the devs who have, up until now, been taking the job of deciding when btrfs is 'ready for primetime' and developed a plan. Is the btrfs-by-default part of the current tech spec more of a long term aspiration, or is it on the table for F21? Have the concerns about its readiness been evaluated and checked with the domain experts?
Thanks!
Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net http://www.happyassassin.net
-- desktop mailing list desktop@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/desktop
Just wanted to mention that btrfs still seems to have some pretty horrific issues with some db/logging situations. A couple of days ago I came across a problem (with a bug already reported) with journald. The problem seems to be a combination of journald either needing much lower defaults for max size (mine was around 4G), or to not have to read the entire freaking log before it's able to a reverse sort, and btrfs. I'm honestly not sure those small write problems are solvable unless they occur only infrequently, and I care less about performance than data integrity (with send/receive being great bonuses).
On Tue, Feb 25, 2014 at 5:53 PM, Chris Murphy lists@colorremedies.com wrote:
adding desktop@ since they are also looking at file system options
On Feb 25, 2014, at 1:42 PM, Stephen Gallagher sgallagh@redhat.com wrote:
=== File system ===
The default file system type for workstation installs should be btrfs.
The default file system is definitely up for some debate, but I'd make an argument for using XFS atop LVM[1] for the default filesystem in the Fedora Server, at least in part because Red Hat's storage experts have done the research for us already and determined that XFS is the recommended fit for Red Hat Enterprise Linux 7.
XFS is a really good idea for Server.
I've yet to actually advocate against this majorly, but I'm pretty against using btrfs as the default for any product. At least in the F21 timeframe. It's simply not ready.
Follow-up questions:
Can Server and Workstation WG's choose different defaults for their product's installers?
Other than lack of shrink support in XFS, I'd say XFS is suitable for Workstation as well. Would the Workstation WG have concerns about the lack of fs shrink support in the default file system? [1]
I don't think shrink support is a factor at all for Workstation.
Btrfs still makes me somewhat nervous, given that its upstream doesn't consider it stable[3].
That wiki entry appears old. The stable aspect was about disk format, which is now stable. And also the experimental description was removed in kernel 3.13. [2]
My main two concerns with Btrfs:
- With even minor problems users sometimes go straight to the big hammer approach with "btrfsck/btrfs check --repair" rather than the recommended approaches. Remounting with -o recovery, and even using a newer kernel are recommended on linux-btrfs@ significantly more often than btrfs check --repair.
That's a pretty poor user experience either way. The filesystem should be the last thing the user has to worry about, and forcing them to upgrade to get their FS fixed is indicative of btrfs not being ready.
- Supporting multiple device volumes in Anaconda. Although multiple device Btrfs volumes work very well when devices are working, there's no device failure notification yet. When a device fails, the volume becomes read-only; and the volume isn't mountable (or bootable) without the use of "degraded" mount option. While this can be done as a boot parameter, it's sort of a non-obvious thing for the typical user. I think it's valid for either WG to consider reducing exposure by dropping Anaconda support for it, or placarding the feature somehow.
If, and it's a very big if, Workstation were to go with btrfs, I would really push for a reduced functionality mode similar to what OpenSUSE is doing. No RAID, no multi-device, etc.
josh
Josh Boyer wrote:
I've yet to actually advocate against this majorly, but I'm pretty against using btrfs as the default for any product. At least in the F21 timeframe. It's simply not ready.
What about when it is ready?
Ext4 has its btrfs conversion tool. Changing from ext4 to XFS, for arguably negligible benefits for Workstations, will make it more difficult for Fedora users to transition to btrfs.
On Wed, Feb 26, 2014 at 10:13 AM, Michael Cronenworth mike@cchtml.com wrote:
Josh Boyer wrote:
I've yet to actually advocate against this majorly, but I'm pretty against using btrfs as the default for any product. At least in the F21 timeframe. It's simply not ready.
What about when it is ready?
Then we can use it as default? I don't understand the question there really.
Ext4 has its btrfs conversion tool. Changing from ext4 to XFS, for arguably negligible benefits for Workstations, will make it more difficult for Fedora users to transition to btrfs.
I agree switching from ext4 to XFS is likely not worthwhile. I wouldn't in a million years advocate for people to convert an ext4 fs to btrfs though. Back the data up, do a fresh install.
josh
Josh Boyer wrote:
Then we can use it as default? I don't understand the question there really.
My question was prefacing my suggestion of the conversion tool.
I agree switching from ext4 to XFS is likely not worthwhile. I wouldn't in a million years advocate for people to convert an ext4 fs to btrfs though. Back the data up, do a fresh install.
Is there a technical downside to using the conversion? It has been about a year since I last used it but I cannot recall any conversion issues that myself or others have had.
(I am not advocating the use of btrfs or btrfs-as-default here.)
On Wed, Feb 26, 2014 at 10:30 AM, Michael Cronenworth mike@cchtml.com wrote:
Josh Boyer wrote:
Then we can use it as default? I don't understand the question there really.
My question was prefacing my suggestion of the conversion tool.
I agree switching from ext4 to XFS is likely not worthwhile. I wouldn't in a million years advocate for people to convert an ext4 fs to btrfs though. Back the data up, do a fresh install.
Is there a technical downside to using the conversion? It has been about a year since I last used it but I cannot recall any conversion issues that myself or others have had.
I have no idea, but from a user and upgrade perspective, it's not something I would advocate for at all. Upgrades are already kind of dicey and adding "filesystem conversion" to the tools and QA matrix is just one more risk we don't really need to take. If a user wants to do it manually before they upgrade, then they're certainly able to.
josh
Josh Boyer wrote:
I have no idea, but from a user and upgrade perspective, it's not something I would advocate for at all. Upgrades are already kind of dicey and adding "filesystem conversion" to the tools and QA matrix is just one more risk we don't really need to take. If a user wants to do it manually before they upgrade, then they're certainly able to.
Correct. I didn't mean to suggest using the tool in anaconda.
It would be a free (albeit unsupported) plus to users of Fedora.
On Feb 26, 2014, at 8:13 AM, Michael Cronenworth mike@cchtml.com wrote:
Ext4 has its btrfs conversion tool. Changing from ext4 to XFS, for arguably negligible benefits for Workstations, will make it more difficult for Fedora users to transition to btrfs.
It's an unlikely path because a.) by default we put ext4 on LVM; b.) the convert tool uses ext4 block size to set btrfs leaf size; c.) the convert tool doesn't set extref, although it easily could. The last two are a cake walk to change compared to the first.
On Feb 26, 2014, at 8:20 AM, Josh Boyer jwboyer@fedoraproject.org wrote:
I agree switching from ext4 to XFS is likely not worthwhile.
Whether Server WG goes with ext4 or XFS on LVM, it's worthwhile for Workstation WG to mimic it merely due to simplicity because then we don't need separate installers or composes.
On Feb 26, 2014, at 8:24 AM, David Cantrell dcantrell@redhat.com wrote:
I think filesystem variance across different Fedoras really impacts QA more than us. We already support a lot of filesystems, but the real hit is the QA test matrix.
QA already tests the file system layouts being discussed. Perhaps the least tested is XFS on LVM only because the XFS test case doesn't specify LVM, so testers probably split and do some plain partition and some on LVM.
If Server WG decides on XFS, it effectively increases the Automatic/Guided/easy/default installer path's "Partition Scheme" pop-up from four to five options, and that is a problem. Adamw and I are working on a proposal to reduce these options to one or two: i.e. a WG chosen product specific default, and maybe "one other" which is decided by Base WG or FESCo.
Chris Murphy
On Wed, 2014-02-26 at 12:18 -0700, Chris Murphy wrote:
I agree switching from ext4 to XFS is likely not worthwhile.
Whether Server WG goes with ext4 or XFS on LVM, it's worthwhile for Workstation WG to mimic it merely due to simplicity because then we don't need separate installers or composes.
I'm broadly in agreement with Chris here. I don't see that any 'plain partition' filesystem has such a huge difference to the other that it makes much sense for us to have two products using 'plain partition' filesystems, by default, but *different* ones.
Choosing btrfs by default is a controversial option, but it's at least clearly one with very different results from picking a 'plain partition' filesystem (whether backed by LVM or not). I don't really see the point in having ext4 for one and xfs for the other. If the only argument for desktop to keep ext4 if server goes xfs is 'btrfs conversion!', I'm with cmurf that that's not a compelling argument at all.
The elephant in the room here seems to be LVM backing, I don't see anyone discussing that. Do desktop and server want to keep LVM backing by default if they don't go with btrfs? Do desktop and server have *differing* perspectives there? (Do we want to re-run the Fedora 18 tape where we switch to no LVM backing by default and then have to go back to LVM by default for some reason I've forgotten?)
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 02/26/2014 02:42 PM, Adam Williamson wrote:
On Wed, 2014-02-26 at 12:18 -0700, Chris Murphy wrote:
I agree switching from ext4 to XFS is likely not worthwhile.
Whether Server WG goes with ext4 or XFS on LVM, it's worthwhile for Workstation WG to mimic it merely due to simplicity because then we don't need separate installers or composes.
I'm broadly in agreement with Chris here. I don't see that any 'plain partition' filesystem has such a huge difference to the other that it makes much sense for us to have two products using 'plain partition' filesystems, by default, but *different* ones.
Choosing btrfs by default is a controversial option, but it's at least clearly one with very different results from picking a 'plain partition' filesystem (whether backed by LVM or not). I don't really see the point in having ext4 for one and xfs for the other. If the only argument for desktop to keep ext4 if server goes xfs is 'btrfs conversion!', I'm with cmurf that that's not a compelling argument at all.
The elephant in the room here seems to be LVM backing, I don't see anyone discussing that. Do desktop and server want to keep LVM backing by default if they don't go with btrfs? Do desktop and server have *differing* perspectives there? (Do we want to re-run the Fedora 18 tape where we switch to no LVM backing by default and then have to go back to LVM by default for some reason I've forgotten?)
I can only speak for myself, but regardless of whether Server picks ext4 or XFS, I think we definitely want it to be on LVM. (LVM thin-provisioning is another can of worms, but let's talk about that separately).
On Wed, Feb 26, 2014 at 2:42 PM, Adam Williamson awilliam@redhat.com wrote:
On Wed, 2014-02-26 at 12:18 -0700, Chris Murphy wrote:
I agree switching from ext4 to XFS is likely not worthwhile.
Whether Server WG goes with ext4 or XFS on LVM, it's worthwhile for Workstation WG to mimic it merely due to simplicity because then we don't need separate installers or composes.
I'm broadly in agreement with Chris here. I don't see that any 'plain partition' filesystem has such a huge difference to the other that it makes much sense for us to have two products using 'plain partition' filesystems, by default, but *different* ones.
So my answer was primarily under the premise of Workstation alone. If Server switches to XFS, then yeah maybe using XFS on Workstation makes more sense. If Server doesn't, then there's really no benefit to Workstation doing that. I think we're in violent agreement on this, so we can stop emailing about it now.
The elephant in the room here seems to be LVM backing, I don't see anyone discussing that. Do desktop and server want to keep LVM backing by default if they don't go with btrfs? Do desktop and server have *differing* perspectives there? (Do we want to re-run the Fedora 18 tape where we switch to no LVM backing by default and then have to go back to LVM by default for some reason I've forgotten?)
I'm not sure on the Workstation front. The dm-thinp stuff might be a solution to some of the snapshotting features btrfs would provide, but I think that doesn't necessitate LVM thinp. (IIRC, Alexander Larsson found raw dm-thinp to be more usable and performant for his Docker stuff too.) It's something we'll have to take up within the WG.
josh
On Feb 26, 2014 7:24 AM, "Josh Boyer" jwboyer@fedoraproject.org wrote:
- Other than lack of shrink support in XFS, I'd say XFS is suitable for
Workstation as well. Would the Workstation WG have concerns about the lack of fs shrink support in the default file system? [1]
I don't think shrink support is a factor at all for Workstation.
...
josh
Filesystem shrinking should be a *requirement* for Workstation. I'd love to see Fedora as everyone's primary and only OS, but that's a long term goal. As the flagship desktop offering, Fedora Workstation will be the first choice for inexperienced or casual users, the class of users that boot multiple OSen. The class of users that would switch primary distributions in frustration on discovering that they are forced to reinstall to gain a bit of unallocated space.
Such users may not be the target audience for Workstation, but they still represent mindshare and a pool of potential contributors. Consider carefully on decisions that would alienate this group. "We warned you in the Release Notes, you should have changed the default filesystem " gains no favor.
--Pete
On Wed, 2014-02-26 at 12:43 -0700, Pete Travis wrote:
On Feb 26, 2014 7:24 AM, "Josh Boyer" jwboyer@fedoraproject.org wrote:
- Other than lack of shrink support in XFS, I'd say XFS is suitable for
Workstation as well. Would the Workstation WG have concerns about the lack of fs shrink support in the default file system? [1]
I don't think shrink support is a factor at all for Workstation.
...
josh
Filesystem shrinking should be a *requirement* for Workstation. I'd love to see Fedora as everyone's primary and only OS, but that's a long term goal. As the flagship desktop offering, Fedora Workstation will be the first choice for inexperienced or casual users, the class of users that boot multiple OSen. The class of users that would switch primary distributions in frustration on discovering that they are forced to reinstall to gain a bit of unallocated space.
Well, you're never 'forced' to do that. You can always just do the shrink in pre-flight. It's convenient to have it available from the installer, but it's not *necessary*.
It is kind of a significant convenience, though, and I agree Josh kinda underplayed it. Losing the ability to install alongside full-disk Windows installations without asking the user to do some pre-flight work themselves would be a significant loss.
On Wed, 2014-02-26 at 11:47 -0800, Adam Williamson wrote:
On Wed, 2014-02-26 at 12:43 -0700, Pete Travis wrote:
On Feb 26, 2014 7:24 AM, "Josh Boyer" jwboyer@fedoraproject.org wrote:
- Other than lack of shrink support in XFS, I'd say XFS is suitable for
Workstation as well. Would the Workstation WG have concerns about the lack of fs shrink support in the default file system? [1]
I don't think shrink support is a factor at all for Workstation.
...
josh
Filesystem shrinking should be a *requirement* for Workstation. I'd love to see Fedora as everyone's primary and only OS, but that's a long term goal. As the flagship desktop offering, Fedora Workstation will be the first choice for inexperienced or casual users, the class of users that boot multiple OSen. The class of users that would switch primary distributions in frustration on discovering that they are forced to reinstall to gain a bit of unallocated space.
Well, you're never 'forced' to do that. You can always just do the shrink in pre-flight. It's convenient to have it available from the installer, but it's not *necessary*.
It is kind of a significant convenience, though, and I agree Josh kinda underplayed it. Losing the ability to install alongside full-disk Windows installations without asking the user to do some pre-flight work themselves would be a significant loss.
Although of course I've lost the context there, and we're talking about the *Linux* filesystem. The impact of picking xfs would be that something else couldn't shrink the Fedora install and install alongside it, I guess. So we're being kinda 'rude' to others, but hey, it doesn't hurt us. ;)
On Wed, Feb 26, 2014 at 2:47 PM, Adam Williamson awilliam@redhat.com wrote:
On Wed, 2014-02-26 at 12:43 -0700, Pete Travis wrote:
On Feb 26, 2014 7:24 AM, "Josh Boyer" jwboyer@fedoraproject.org wrote:
- Other than lack of shrink support in XFS, I'd say XFS is suitable for
Workstation as well. Would the Workstation WG have concerns about the lack of fs shrink support in the default file system? [1]
I don't think shrink support is a factor at all for Workstation.
...
josh
Filesystem shrinking should be a *requirement* for Workstation. I'd love to see Fedora as everyone's primary and only OS, but that's a long term goal. As the flagship desktop offering, Fedora Workstation will be the first choice for inexperienced or casual users, the class of users that boot multiple OSen. The class of users that would switch primary distributions in frustration on discovering that they are forced to reinstall to gain a bit of unallocated space.
Well, you're never 'forced' to do that. You can always just do the shrink in pre-flight. It's convenient to have it available from the installer, but it's not *necessary*.
It is kind of a significant convenience, though, and I agree Josh kinda underplayed it. Losing the ability to install alongside full-disk Windows installations without asking the user to do some pre-flight work themselves would be a significant loss.
The question was about XFS's lack of ability to shrink. Not Windows. In that context, and in the context Pete is talking about, XFS being able to shrink really isn't a factor. Windows is already installed, and you'll be shrinking that filesystem to make room for a Fedora install, not the other way around.
If we live in a world where Fedora is the _first_ and _only_ OS a computer has on it when a user gets the machine, and a user needs to shrink the Fedora install to try another OS, then I will gladly agree shrink support is a factor. The last time I checked though, we don't live in that world...
In the case where a user blows away things and installs Fedora as the only OS, then gets pissed off with Fedora and wants to switch... well we've really failed already there. It's arguable that since they did that, they kind of knew what they were doing so they can fix things as needed.
For the users that want to try ALL the OSes because it's fun, I'm not sure that's something Workstation is really going to strive hard at achieving. Also, that is exactly why live images exist.
All of this really only comes into play if XFS is used to begin with, and as I said elsewhere the only reason I would see Workstation switching to that is to have commonality with Server.
josh
Another aspect of xfs we may want to investigate and get feedback from filesystem folks is how well xfs works on 32bit these days.
RHEL7 doesn't have a 32bit version in their beta, so they only need to support 64bit xfs. Does the fact that we expect to have 32bit workstation and/or server weigh into this decision any?
kevin
Kevin Fenzi (kevin@scrye.com) said:
Another aspect of xfs we may want to investigate and get feedback from filesystem folks is how well xfs works on 32bit these days.
RHEL7 doesn't have a 32bit version in their beta, so they only need to support 64bit xfs. Does the fact that we expect to have 32bit workstation and/or server weigh into this decision any?
We expect to have a 32-bit workstation or server?
Not trying to troll, but I don't know that any of these were specifically discussed or specified in the products - are there any arches where Fedora currently exists that we don't necessarily care about having a particular product on? (For example, if you expand to secondary arches, I'd question the idea of s390 Workstation.)
Bill, who does have a 32-bit x86 server under his home desk...
On Wed, 26 Feb 2014 16:03:47 -0500 Bill Nottingham notting@redhat.com wrote:
Kevin Fenzi (kevin@scrye.com) said:
Another aspect of xfs we may want to investigate and get feedback from filesystem folks is how well xfs works on 32bit these days.
RHEL7 doesn't have a 32bit version in their beta, so they only need to support 64bit xfs. Does the fact that we expect to have 32bit workstation and/or server weigh into this decision any?
We expect to have a 32-bit workstation or server?
Not trying to troll, but I don't know that any of these were specifically discussed or specified in the products - are there any arches where Fedora currently exists that we don't necessarily care about having a particular product on? (For example, if you expand to secondary arches, I'd question the idea of s390 Workstation.)
Bill, who does have a 32-bit x86 server under his home desk...
Yeah, I don't know. That would be a good thing to decide for:
a) each of the products.
b) fedora in general
c) spins
kevin
On Wed, Feb 26, 2014 at 02:08:19PM -0700, Kevin Fenzi wrote:
We expect to have a 32-bit workstation or server? Not trying to troll, but I don't know that any of these were specifically discussed or specified in the products - are there any
Yeah, I don't know. That would be a good thing to decide for: a) each of the products. b) fedora in general c) spins
For cloud, I don't think we had a formal vote, but as I recall the main consensus, decided that we're keeping x86 64 and 32-bit for at least F21, and possibly adding ARM if it looks like there are any meaningful installations of ARM IaaS by then. And we'll revisit for F22. I think adding ARM and dropping 32-bit x86 is going to happen... it's just a question of when. (And I assume that the ARM clouds will be AArch64, but my crystal ball is hazy.)
[Aside: this thread demonstrates why we need to go to just one devel mailing list with tags. Cross posting, thread drift, what a mess. :) ]
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 02/26/2014 04:03 PM, Bill Nottingham wrote:
Kevin Fenzi (kevin@scrye.com) said:
Another aspect of xfs we may want to investigate and get feedback from filesystem folks is how well xfs works on 32bit these days.
RHEL7 doesn't have a 32bit version in their beta, so they only need to support 64bit xfs. Does the fact that we expect to have 32bit workstation and/or server weigh into this decision any?
We expect to have a 32-bit workstation or server?
Not trying to troll, but I don't know that any of these were specifically discussed or specified in the products - are there any arches where Fedora currently exists that we don't necessarily care about having a particular product on? (For example, if you expand to secondary arches, I'd question the idea of s390 Workstation.)
Bill, who does have a 32-bit x86 server under his home desk...
That's one of the topics scheduled to be discussed in the Server Technical Specification meeting tomorrow. My personal opinion is that the Server should support any arch defined by FESCo as a "primary arch", which today would mean 32-bit and 64-bit x86 as well as armv7hl.
On Feb 26, 2014, at 1:54 PM, Kevin Fenzi kevin@scrye.com wrote:
Another aspect of xfs we may want to investigate and get feedback from filesystem folks is how well xfs works on 32bit these days.
RHEL7 doesn't have a 32bit version in their beta, so they only need to support 64bit xfs. Does the fact that we expect to have 32bit workstation and/or server weigh into this decision any?
I think the only limit is 16TB max file system, so off hand I'd say no. But this also applies to ext4.
I just tested with 3.13.4-200.fc20.i686+PAE.
XFS (sdc): file system too large to be mounted on this system. EXT4-fs (sdc): filesystem too large to mount safely on this system.
The same 32-bit kernel mounts a 20TB Btrfs volume with no complaints, so I don't know its limit.
http://paste.fedoraproject.org/80784/93470280/
XFS mounts with inode64 by default, same as x86_64. So that's good.
Chris Murphy
On Wed, 26 Feb 2014 21:13:58 -0700 Chris Murphy lists@colorremedies.com wrote:
On Feb 26, 2014, at 1:54 PM, Kevin Fenzi kevin@scrye.com wrote:
Another aspect of xfs we may want to investigate and get feedback from filesystem folks is how well xfs works on 32bit these days.
RHEL7 doesn't have a 32bit version in their beta, so they only need to support 64bit xfs. Does the fact that we expect to have 32bit workstation and/or server weigh into this decision any?
I think the only limit is 16TB max file system, so off hand I'd say no. But this also applies to ext4.
I just tested with 3.13.4-200.fc20.i686+PAE.
XFS (sdc): file system too large to be mounted on this system. EXT4-fs (sdc): filesystem too large to mount safely on this system.
The same 32-bit kernel mounts a 20TB Btrfs volume with no complaints, so I don't know its limit.
http://paste.fedoraproject.org/80784/93470280/
XFS mounts with inode64 by default, same as x86_64. So that's good.
I wasn't referring to filesystem size limits... there was some issue with Xfs and linux kernel stacks on 32bit linux causing crashes and data loss.
I don't know if this has been fixed, no longer applies or just has been ignored by only using it on 64bit machines, but before we go making it a default and shipping it on 32bit, we should ask around about it. ;)
kevin
On Feb 26, 2014, at 9:41 PM, Kevin Fenzi kevin@scrye.com wrote:
On Wed, 26 Feb 2014 21:13:58 -0700 Chris Murphy lists@colorremedies.com wrote:
On Feb 26, 2014, at 1:54 PM, Kevin Fenzi kevin@scrye.com wrote:
Another aspect of xfs we may want to investigate and get feedback from filesystem folks is how well xfs works on 32bit these days.
RHEL7 doesn't have a 32bit version in their beta, so they only need to support 64bit xfs. Does the fact that we expect to have 32bit workstation and/or server weigh into this decision any?
I think the only limit is 16TB max file system, so off hand I'd say no. But this also applies to ext4.
I just tested with 3.13.4-200.fc20.i686+PAE.
XFS (sdc): file system too large to be mounted on this system. EXT4-fs (sdc): filesystem too large to mount safely on this system.
The same 32-bit kernel mounts a 20TB Btrfs volume with no complaints, so I don't know its limit.
http://paste.fedoraproject.org/80784/93470280/
XFS mounts with inode64 by default, same as x86_64. So that's good.
I wasn't referring to filesystem size limits... there was some issue with Xfs and linux kernel stacks on 32bit linux causing crashes and data loss.
Yes I know. Even 32-bit mkfs will make a much bigger file system, it just won't mount it. I'm only finding some things 2+ years old suggesting it was once possible to mount XFS bigger than 16TB on 32bit OS, but after filling it (mounted rw) to just beyond 16TB then the kernel would panic and that was the end of accessing the volume from a 32bit kernel.
inode64 is important even for 32bit because otherwise all the inodes end up in the first 1TB and the whole file system is wonky and slow if it's sufficiently large.
I don't know if this has been fixed, no longer applies or just has been ignored by only using it on 64bit machines, but before we go making it a default and shipping it on 32bit, we should ask around about it. ;)
Sure, I agree. I will ask on the XFS list and also include armv7hl in the inquiry even though I'm not certain they are interested in XFS (?).
Chris Murphy
On Feb 26, 2014, at 10:27 PM, Chris Murphy lists@colorremedies.com wrote:
On Feb 26, 2014, at 9:41 PM, Kevin Fenzi kevin@scrye.com wrote:
I don't know if this has been fixed, no longer applies or just has been ignored by only using it on 64bit machines, but before we go making it a default and shipping it on 32bit, we should ask around about it. ;)
Sure, I agree. I will ask on the XFS list and also include armv7hl in the inquiry even though I'm not certain they are interested in XFS (?).
http://oss.sgi.com/pipermail/xfs/2014-February/034588.html
i686 is regularly tested upstream, and there are no concerns about it being used by default there. I'm a bit less certain about the arm response, so maybe solicit feedback from them in case they have concerns about it?
Chris Murphy
On Wed, Feb 26, 2014 at 09:41:33PM -0700, Kevin Fenzi wrote:
I wasn't referring to filesystem size limits... there was some issue with Xfs and linux kernel stacks on 32bit linux causing crashes and data loss.
Possibly this: https://www.redhat.com/archives/nahant-list/2005-June/msg00304.html
But I too have no idea what's happened on this front in the past nine years. I suspect quite a lot.
On Wed, 2014-02-26 at 15:32 -0500, Josh Boyer wrote:
It is kind of a significant convenience, though, and I agree Josh kinda underplayed it. Losing the ability to install alongside full-disk Windows installations without asking the user to do some pre-flight work themselves would be a significant loss.
The question was about XFS's lack of ability to shrink. Not Windows. In that context, and in the context Pete is talking about, XFS being able to shrink really isn't a factor. Windows is already installed, and you'll be shrinking that filesystem to make room for a Fedora install, not the other way around.
yeah, I noticed the context switch in my follow-up email, sorry. I do think the 'user wants to install something else alongside Fedora' case is worth caring about at least a little bit, but having custom part available is probably good enough.
On Feb 26, 2014 2:11 PM, "Adam Williamson" awilliam@redhat.com wrote:
On Wed, 2014-02-26 at 15:32 -0500, Josh Boyer wrote:
It is kind of a significant convenience, though, and I agree Josh
kinda
underplayed it. Losing the ability to install alongside full-disk Windows installations without asking the user to do some pre-flight
work
themselves would be a significant loss.
The question was about XFS's lack of ability to shrink. Not Windows. In that context, and in the context Pete is talking about, XFS being able to shrink really isn't a factor. Windows is already installed, and you'll be shrinking that filesystem to make room for a Fedora install, not the other way around.
yeah, I noticed the context switch in my follow-up email, sorry. I do think the 'user wants to install something else alongside Fedora' case is worth caring about at least a little bit, but having custom part available is probably good enough. -- Adam Williamson
Right, my concern is users shrinking Fedora to make room for something else. Admittedly not a common circumstance, but one guaranteed to cause an uproar in the enthusiast community, ie potential contributor pool. Not coders on the whole, but there's plenty of room for such in the Fedora community. It is a preventable issue, and I'd like the community repercussions to be weighed against the technical merits of an unshrinkable default filesystem.
--Pete
On Feb 26, 2014, at 2:11 PM, Adam Williamson awilliam@redhat.com wrote:
On Wed, 2014-02-26 at 15:32 -0500, Josh Boyer wrote:
It is kind of a significant convenience, though, and I agree Josh kinda underplayed it. Losing the ability to install alongside full-disk Windows installations without asking the user to do some pre-flight work themselves would be a significant loss.
The question was about XFS's lack of ability to shrink. Not Windows. In that context, and in the context Pete is talking about, XFS being able to shrink really isn't a factor. Windows is already installed, and you'll be shrinking that filesystem to make room for a Fedora install, not the other way around.
yeah, I noticed the context switch in my follow-up email, sorry. I do think the 'user wants to install something else alongside Fedora' case is worth caring about at least a little bit, but having custom part available is probably good enough.
Most users don't know XFS isn't shrinkable. Most of the target market for the easy path is likely used to ext3/4, NTFS, and HFS+ all of which are shrinkable. And yet a minority would choose a shrinkable file system if they knew the default was not shrinkable.
So I opine it's not a factor in whether XFS should be the default file system. But it is a factor in helping the user make the best choice for them, in advance, rather than them having an oh crap moment later.
Docs should consider adding it as a virtual tooltip in the quick install guide. And maybe it's possible to have a real tooltip in the installer.
Chris Murphy
On Feb 26, 2014, at 7:24 AM, Josh Boyer jwboyer@fedoraproject.org wrote:
On Tue, Feb 25, 2014 at 5:53 PM, Chris Murphy lists@colorremedies.com wrote:
XFS is a really good idea for Server.
I've yet to actually advocate against this majorly, but I'm pretty against using btrfs as the default for any product. At least in the F21 timeframe. It's simply not ready.
Jolla/Sailfish OS use it on mobile phones; openSUSE also considers it ready for their next release, in approximately the same time frame as Fedora 21. Btrfs has been offered as an guided partition path option since Fedora 18. It's been visible in the UI since Fedora 14 or 15. It was first proposed as a default for Fedora 16.
I think the WG's need to have some metric by which to make a largely objective decision, and should get their questions/concerns addressed directly from Btrfs developers if they're considering it as a default.
A certain subjectivity is reasonable too, for example whether Fedora Workstation and Server should be biased more toward production/stability, or development/testing than prior Fedoras. I think answering the bias question makes the file system decision more easily fall into place.
Cloud, I think they probably want to stick it out with plain partition ext4 due to booting simplicity.
My main two concerns with Btrfs:
- With even minor problems users sometimes go straight to the big hammer approach with "btrfsck/btrfs check --repair" rather than the recommended approaches. Remounting with -o recovery, and even using a newer kernel are recommended on linux-btrfs@ significantly more often than btrfs check --repair.
That's a pretty poor user experience either way. The filesystem should be the last thing the user has to worry about, and forcing them to upgrade to get their FS fixed is indicative of btrfs not being ready.
The same recommendation happens on the XFS list too when people having file system problems have old kernels and repair tools. Btrfs is young, and an "old" kernel is maybe only 6-9 months old. Fedora kernels are kept exceptionally current, as are the btrfs user space tools which is likely why fewer Fedora users have Btrfs problems compared to distributions that use much older kernels.
Somewhat less often than users immediately trying btrfsck --repair, are users on the XFS list who report having used xfs_repair -L right after a crash instead of first mounting the file system. That's rather damaging too. The offline repair check/repair utility as the first step after a crash is obsolete 10 years ago, yet people still do such things.
The reality is that the repair tool fixes edge cases, because the file system is designed to not really need one. The common problems either don't happen in the first place, are fixed on a normal mount, or are fixed with the recovery mount option.
My suggestion for the Workstation WG is find out if btrfsck --repair is too often causing worse problem. I don't know the answer to that, but I think it needs to be put directly to Btrfs developers. Any other source is just an anecdote.
- Supporting multiple device volumes in Anaconda. Although multiple device Btrfs volumes work very well when devices are working, there's no device failure notification yet. When a device fails, the volume becomes read-only; and the volume isn't mountable (or bootable) without the use of "degraded" mount option. While this can be done as a boot parameter, it's sort of a non-obvious thing for the typical user. I think it's valid for either WG to consider reducing exposure by dropping Anaconda support for it, or placarding the feature somehow.
If, and it's a very big if, Workstation were to go with btrfs, I would really push for a reduced functionality mode similar to what OpenSUSE is doing. No RAID, no multi-device, etc.
I agree, but to some degree this is up to the WG's and anaconda folks to work out. Today if a user chooses 2+ disks, and the default/Automatic/guided installation path with Partition Scheme set to Btrfs, those drives have data profile raid0, and metadata profile raid1. It's been this way for a while. So what you're suggesting is a change at least to the automatic/easy path for Btrfs, and possibly also a change for Manual/custom, and reads like a distinct shift in bias of Fedora.next to something more production/stability oriented than past Fedoras.
Chris Murphy
On Wed, Feb 26, 2014 at 3:32 PM, Chris Murphy lists@colorremedies.com wrote:
On Feb 26, 2014, at 7:24 AM, Josh Boyer jwboyer@fedoraproject.org wrote:
On Tue, Feb 25, 2014 at 5:53 PM, Chris Murphy lists@colorremedies.com wrote:
XFS is a really good idea for Server.
I've yet to actually advocate against this majorly, but I'm pretty against using btrfs as the default for any product. At least in the F21 timeframe. It's simply not ready.
Jolla/Sailfish OS use it on mobile phones; openSUSE also considers it ready for their next release, in approximately the same time frame as Fedora 21. Btrfs has been offered as an guided partition path option since Fedora 18. It's been visible in the UI since Fedora 14 or 15. It was first proposed as a default for Fedora 16.
All true. (Though for clarity, OpenSUSE is offering a reduced feature mode by default.)
I think the WG's need to have some metric by which to make a largely objective decision, and should get their questions/concerns addressed directly from Btrfs developers if they're considering it as a default.
I've discussed some of this with the FS team within Red Hat. I'm hoping to draw them out of hiding, and I hope they come with data.
A certain subjectivity is reasonable too, for example whether Fedora Workstation and Server should be biased more toward production/stability, or development/testing than prior Fedoras. I think answering the bias question makes the file system decision more easily fall into place.
Yes, seems fair.
Cloud, I think they probably want to stick it out with plain partition ext4 due to booting simplicity.
My main two concerns with Btrfs:
- With even minor problems users sometimes go straight to the big hammer approach with "btrfsck/btrfs check --repair" rather than the recommended approaches. Remounting with -o recovery, and even using a newer kernel are recommended on linux-btrfs@ significantly more often than btrfs check --repair.
That's a pretty poor user experience either way. The filesystem should be the last thing the user has to worry about, and forcing them to upgrade to get their FS fixed is indicative of btrfs not being ready.
The same recommendation happens on the XFS list too when people having file system problems have old kernels and repair tools. Btrfs is young, and an "old" kernel is maybe only 6-9 months old.
"Young." 7 years is not young when it comes to having repair and recovery tools, or surviving things like ENOSPC. I realize _those_ things in btrfs-land actually are young, but that is kind of providing some of the hesitation on my side. It took them this long to have those basic features available? How long will it take them to have it so that the corner cases don't break?
Fedora kernels are kept exceptionally current, as are the btrfs user space tools which is likely why fewer Fedora users have Btrfs problems compared to distributions that use much older kernels.
mmm... I have no way to compare against other distributions. I'm glad Fedora is perceived as having better Btrfs support, but I can assure you it isn't because of any concerted effort on our part. However, when it comes to bug reports for filesystems in Fedora kernels, btrfs is by far the leading FS over ext4 or XFS. Some of this is due to age, sure. Most of it is due to sustained effort by a large community of people for upstream development, and a reduced feature set compared to btrfs. In sort, btrfs has a lot of catching up to do, and it's trying to do that while also leapfrogging everything else.
Somewhat less often than users immediately trying btrfsck --repair, are users on the XFS list who report having used xfs_repair -L right after a crash instead of first mounting the file system. That's rather damaging too. The offline repair check/repair utility as the first step after a crash is obsolete 10 years ago, yet people still do such things.
People do stupid things after an error. Yes. That isn't what I'm worried about. I'm worried about hitting those errors to begin with.
The reality is that the repair tool fixes edge cases, because the file system is designed to not really need one. The common problems either don't happen in the first place, are fixed on a normal mount, or are fixed with the recovery mount option.
My suggestion for the Workstation WG is find out if btrfsck --repair is too often causing worse problem. I don't know the answer to that, but I think it needs to be put directly to Btrfs developers. Any other source is just an anecdote.
I think I'd like to see if anything gets discussed at LFS in a month or so. One of the Fedora kernel team members is going to be around, so hopefully we can get some more information from multiple sources involved there.
Also, if btrfsck --repair is a major source of problems, and you have to ask a developer if you'll lose data if you run it, then is the tool _really_ ready? I disagree with the assertion that it isn't needed. The filesystem might be designed to not need one, but either the design or implementation is lacking enough that it is needed and now we're left in a fairly _unstable_ state. That is not what I want as a default FS.
Please don't mistake my hesitation on using it as the default as being against Btrfs. I'm not. I would love to have it work, work well, and used as the default. It allows much more ease of use and interesting features than anything else. I simply do not think it is in a state where that is a safe choice to make. I do not, as a member of a WG or a kernel maintainer, want to continually apologize for people losing data or having unbootable machines because we choose poorly. If we get data and comparisons that show otherwise, as you suggested, then I will be very interested in looking at it as default.
- Supporting multiple device volumes in Anaconda. Although multiple device Btrfs volumes work very well when devices are working, there's no device failure notification yet. When a device fails, the volume becomes read-only; and the volume isn't mountable (or bootable) without the use of "degraded" mount option. While this can be done as a boot parameter, it's sort of a non-obvious thing for the typical user. I think it's valid for either WG to consider reducing exposure by dropping Anaconda support for it, or placarding the feature somehow.
If, and it's a very big if, Workstation were to go with btrfs, I would really push for a reduced functionality mode similar to what OpenSUSE is doing. No RAID, no multi-device, etc.
I agree, but to some degree this is up to the WG's and anaconda folks to work out. Today if a user chooses 2+ disks, and the default/Automatic/guided installation path with Partition Scheme set to Btrfs, those drives have data profile raid0, and metadata profile raid1. It's been this way for a while. So what you're suggesting is a change at least to the automatic/easy path for Btrfs, and possibly also a change for Manual/custom, and reads like a distinct shift in bias of Fedora.next to something more production/stability oriented than past Fedoras.
I agree with what you said here entirely.
I also happen to think that one of the major focus of Workstation is to produce a _product_ that is more stable than past Fedoras. So looking at it from that perspective I don't see a reduced feature set as being that out of line.
josh
desktop@lists.fedoraproject.org