btrfs as default filesystem for F22?

Post by Andre Robatino
openSUSE 13.2, scheduled for release in November, will have btrfs as the
default filesystem. What are the chances that F22 will follow suit, assuming
openSUSE has no major problems with it?
https://news.opensuse.org/2014/09/22/

I've been using btrfs for a while now, and while the kernels 3.15.x and
3.16.{0,1} have been problematic, the latest one is working smooth
again.

Anyway, I recommend using only the core features (snapshots, raid1,
scrubs, balances, cp --reflink, etc...), because others have many
quirks, like send/receive which get corrupted from time to time, raid
5/6 which is work in progress, or problems related to low free space.

To implement btrfs as the default, grubby must support to install /boot
on btrfs (bug #1094489). I have to run grub2-mkconfig with every kernel
update to circumvent this problem.

It is also worth considering adding some scheduled tasks for
maintenance, like rebalances, or scrubs.

--
Juan Orti
https://miceliux.com

drago01

2014-10-03 08:39:19 UTC

On Fri, Oct 3, 2014 at 8:42 AM, Juan Orti Alcaine

I've been using btrfs for a while now, and while the kernels 3.15.x and
3.16.{0,1} have been problematic, the latest one is working smooth again.
Anyway, I recommend using only the core features (snapshots, raid1, scrubs,
balances, cp --reflink, etc...), because others have many quirks, like
send/receive which get corrupted from time to time, raid 5/6 which is work
in progress, or problems related to low free space.
To implement btrfs as the default, grubby must support to install /boot on
btrfs (bug #1094489). I have to run grub2-mkconfig with every kernel update
to circumvent this problem.
It is also worth considering adding some scheduled tasks for maintenance,
like rebalances, or scrubs.

If it needs active "maintenance" like that its worse than what we already have.

Steven Whitehouse

2014-10-03 09:38:20 UTC

Hi,

I've been using btrfs for a while now, and while the kernels 3.15.x
and 3.16.{0,1} have been problematic, the latest one is working smooth
again.
Anyway, I recommend using only the core features (snapshots, raid1,
scrubs, balances, cp --reflink, etc...), because others have many
quirks, like send/receive which get corrupted from time to time, raid
5/6 which is work in progress, or problems related to low free space.
To implement btrfs as the default, grubby must support to install
/boot on btrfs (bug #1094489). I have to run grub2-mkconfig with every
kernel update to circumvent this problem.
It is also worth considering adding some scheduled tasks for
maintenance, like rebalances, or scrubs.

I think "problems related to low free space" is a big issue for a
default file system. If users have a bad experience due to a problem on
the default file system, then that will very likely reflect on their
feelings about Fedora as a whole, so it is vitally important that
whatever fs is the default is as stable as possible.

It is possible to already do all of the things (and more) which you've
listed under "core features" using LVM/dm/md too, with the exception of
cp --reflink, so that it wouldn't result in a big difference in
functionality unless and until the more experimental (for want of a
better term) btrfs features mature. There is also currently a much
greater developer community around LVM/dm/md than there is around the
same (volume level) features in btrfs, and LVM/dm/md supports a wider
range of functionality.

I should also add (just in case anybody gets the wrong idea!) that I
think it should definitely be made as easy as possible for anybody who
wants to evaluate running btrfs on Fedora, but it is far too early to
make it the default yet,

Steve.

Juan Orti Alcaine

2014-10-03 11:18:02 UTC

Post by Steven Whitehouse
Hi,
I should also add (just in case anybody gets the wrong idea!) that I
think it should definitely be made as easy as possible for anybody who
wants to evaluate running btrfs on Fedora, but it is far too early to
make it the default yet,

I agree with your opinion, it's a bit too early. An experienced user can
deal with the idiosyncrasies of btrfs and it's great when you learn it,
but pushing it as the default seems too adventurous.

I want to add to the list of problems the performance degradation over
time in database-like files (journal, vm images, firefox and other
sqlite db). I have experienced minutes of delay consulting the journal
in heavily fragmented journal files.

--
Juan Orti
https://miceliux.com

Ian Kent

2014-10-04 01:43:46 UTC

I agree with your opinion, it's a bit too early. An experienced user can
deal with the idiosyncrasies of btrfs and it's great when you learn it,
but pushing it as the default seems too adventurous.
I want to add to the list of problems the performance degradation over
time in database-like files (journal, vm images, firefox and other
sqlite db). I have experienced minutes of delay consulting the journal
in heavily fragmented journal files.

I still have to agree with not making btrfs the default.

Sadly, I see BUG(), ENOSPC error reports (seem to have returned), and
btrfs-progs SEGV reports on the btrfs mailing quite regularly and while
much of the functionality may now be stable all the features of the
default file system will get stressed more than the other file systems,
including the perhaps not so stable ones.

Post by Juan Orti Alcaine
--
Juan Orti
https://miceliux.com

M. Edward (Ed) Borasky

2014-10-05 05:49:38 UTC

I'm happy with ext4. Just out of curiosity, though, how is XFS working out
on RHEL 7?

Post by Ian Kent

I agree with your opinion, it's a bit too early. An experienced user can
deal with the idiosyncrasies of btrfs and it's great when you learn it,
but pushing it as the default seems too adventurous.
I want to add to the list of problems the performance degradation over
time in database-like files (journal, vm images, firefox and other
sqlite db). I have experienced minutes of delay consulting the journal
in heavily fragmented journal files.

I still have to agree with not making btrfs the default.
Sadly, I see BUG(), ENOSPC error reports (seem to have returned), and
btrfs-progs SEGV reports on the btrfs mailing quite regularly and while
much of the functionality may now be stable all the features of the
default file system will get stressed more than the other file systems,
including the perhaps not so stable ones.

Post by Juan Orti Alcaine
--
Juan Orti
https://miceliux.com

--
devel mailing list
devel at lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

--
Twitter: http://twitter.com/znmeb; Computational Journalism on a Stick
http://j.mp/CompJournoStickOverview

Remember, if you're traveling to Bactria, Hump Day is Tuesday and Thursday.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.fedoraproject.org/pipermail/devel/attachments/20141004/a7a037f5/attachment.html>

Michael Catanzaro

2014-10-05 15:20:18 UTC

Post by Juan Orti Alcaine
Anyway, I recommend using only the core features (snapshots, raid1,
scrubs, balances, cp --reflink, etc...), because others have many
quirks, like send/receive which get corrupted from time to time, raid
5/6 which is work in progress, or problems related to low free space.

FWIW: openSUSE has disabled very many btrfs features because they aren't
stable enough. [1]

FWIW: SUSE Linux Enterprise 12 will likely also be shipping with btrfs
as default later this year, unless they've changed their plans.

FWIW: openSUSE is actually using xfs for /home and btrfs only for
everything else.

Happy Sunday,

Michael

[1] http://lists.opensuse.org/opensuse-factory/2013-09/msg00029.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 473 bytes
Desc: This is a digitally signed message part
URL: <http://lists.fedoraproject.org/pipermail/devel/attachments/20141005/b5a9bb28/attachment.sig>

Josef Bacik

2014-10-05 13:50:01 UTC

On Oct 2, 2014 11:32 PM, "Andre Robatino" <robatino at fedoraproject.org>

Rahul Sundaram

2014-10-05 14:11:39 UTC

Hi

On Sun, Oct 5, 2014 at 9:50 AM, Josef Bacik wrote:

My plan is to push for F23, I'm still wrapping up some balance bugs and

Post by Josef Bacik
some other issues we've found at work and then this will be my next
priority.

Thanks for the update. Is there a list of target bugs?

Rahul
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.fedoraproject.org/pipermail/devel/attachments/20141005/b5c952c5/attachment.html>

Gene Czarcinski

2014-10-05 19:03:20 UTC

Post by Josef Bacik
On Oct 2, 2014 11:32 PM, "Andre Robatino" <robatino at fedoraproject.org

Post by Andre Robatino
openSUSE 13.2, scheduled for release in November, will have btrfs as the
default filesystem. What are the chances that F22 will follow suit,

assuming

Post by Andre Robatino
openSUSE has no major problems with it?
https://news.opensuse.org/2014/09/22/

F23 sounds about right to me. I am very much a fan of BTRFS and
currently use it on all of my systems with few problems but I think that
F22 is a bit early to make it the default.

However, I do believe that there a couple of things that need to happen
to make thjings easier/better:

1. The Fedora developers/maintainers need to take BTRFS more seriously
and address btrfs related problems seriously and quickly. Yes, I know
that BTRFS swap draining is a little bit difficult to deal with when you
are up to you rear end in alligators but, a little more attension please.

2. Currently anaconda supports supports installation of /boot on BTRFS
either as a simple directory on a BTRFS volume (yes, I don't understand
why someone would do this but ...), as a simple directory on a rootfs
("/") subvolume, or as a spepate subvolume. Grub2.02 also supports this
and grubby will support if "real soon now" when pjones can get enough
time to examine and integrate my grubby patches adding BTRFS support.

3. About the only thing that I would like to see changed is to move the
drop-down menu which selects the filesystem type from the "custom" to
the previous (disks?) panel. This would allow a user to select BTRFS
and then let anaconda do its automagic allocation. Yes, I now it is not
that hard to select custom, then select BTRFS and then click on the auto
allocation.

Anyway, we are getting close.

Gene

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.fedoraproject.org/pipermail/devel/attachments/20141005/f2d60293/attachment.html>

Josh Boyer

2014-10-06 00:25:23 UTC

Post by Josef Bacik
On Oct 2, 2014 11:32 PM, "Andre Robatino" <robatino at fedoraproject.org>

My plan is to push for F23, I'm still wrapping up some balance bugs and some
other issues we've found at work and then this will be my next priority.
Suse benefits from having a narrow "supported" criteria, like only use it
with lots of space and don't use any of the RAID stuff, plus they have two
kernel guys on it and Dave Sterba who is now in charge of btrfs-progs.
Fedora unfortunately has me who has Facebook work to do and Eric who is a
professional fs juggler. We will get there, and when we do it will be less
painful than its going to be for Suse since they will have fixed it all for
us ;). Thanks,
F23 sounds about right to me. I am very much a fan of BTRFS and currently
use it on all of my systems with few problems but I think that F22 is a bit
early to make it the default.
However, I do believe that there a couple of things that need to happen to
1. The Fedora developers/maintainers need to take BTRFS more seriously and
address btrfs related problems seriously and quickly. Yes, I know that
BTRFS swap draining is a little bit difficult to deal with when you are up
to you rear end in alligators but, a little more attension please.

Nope. See, we deal with what we think we can deal with and what is
impacting the most people. There are two of us. A non-default
filesystem with a lot of known issues isn't high on the priority list.
If you would like to see more attention on btrfs, find some people
that share your interest to triage and work on the bugs (which are all
upstream bugs) or chip in yourself.

Post by Josef Bacik
2. Currently anaconda supports supports installation of /boot on BTRFS
either as a simple directory on a BTRFS volume (yes, I don't understand why
someone would do this but ...), as a simple directory on a rootfs ("/")
subvolume, or as a spepate subvolume. Grub2.02 also supports this and
grubby will support if "real soon now" when pjones can get enough time to
examine and integrate my grubby patches adding BTRFS support.

This isn't a requirement for btrfs by default. It's a nice to have.

Post by Josef Bacik
Anyway, we are getting close.

If it's getting close, it's entirely because of Josef's and the SuSE
team's efforts. They should be applauded.

josh

Gene Czarcinski

2014-10-06 12:29:01 UTC

Post by Josef Bacik
On Oct 2, 2014 11:32 PM, "Andre Robatino" <robatino at fedoraproject.org>

Nope, I understand. However, what I said is with the understanding to
change to using BTRFS as the default ... whenever it takes place and I
stand by my statement of needing more attention as a requirement.

This isn't a requirement for btrfs by default. It's a nice to have.

Using btrfs by default needs more users to want to use it. Making it
easier for more users to install into btrfs will see increased use. It
will also demonstrate that Fedora development/maintenance is paying
attention to BTRFS.

Now, there is another question which has not been voiced: what is the
"plan" for filessystems in Fedora (and by implication RHEL)? Is it
BTRFS? Or, perhaps is it LVM with XFS? IIRC, some time ago it was
stated that the plan was to move to BTRFS. It is not clear to me that
everyone is onboard with that decision. Or, perhaps that decision is
being reconsidered.

Post by Josef Bacik
Anyway, we are getting close.

If it's getting close, it's entirely because of Josef's and the SuSE
team's efforts. They should be applauded.

Yes, they should! But Fedora moving to increased use of BTRFS seems to
be receding at the speed of time: it is always a couple of releases from
now; go away I am busy with other stuff. Sorry is this bothers some
people but that is the impression I get.

Gene

Ralf Corsepius

2014-10-06 12:45:15 UTC

Post by Gene Czarcinski
Now, there is another question which has not been voiced: what is the
"plan" for filessystems in Fedora (and by implication RHEL)? Is it
BTRFS? Or, perhaps is it LVM with XFS? IIRC, some time ago it was
stated that the plan was to move to BTRFS. It is not clear to me that
everyone is onboard with that decision. Or, perhaps that decision is
being reconsidered.

Let me answer from the position of a mere user. It's not clear to me why
and when users should switch to BTRFS or xfs or else, nor am I not
interested in using anything which would potentially endanger existing
installations (So far, reports I am reading from openSUSE users don't
necessarily sound convincing).

In other words, you'd have to do a lot of marketing and convincing work
to persuade users to use BTRFS/xfs etc.

Ralf

Eric Sandeen

2014-10-06 14:30:39 UTC

Post by Ralf Corsepius

Post by Gene Czarcinski
Now, there is another question which has not been voiced: what is
the "plan" for filessystems in Fedora (and by implication RHEL)?
Is it BTRFS? Or, perhaps is it LVM with XFS? IIRC, some time ago
it was stated that the plan was to move to BTRFS. It is not clear
to me that everyone is onboard with that decision. Or, perhaps
that decision is being reconsidered.

Let me answer from the position of a mere user. It's not clear to me
why and when users should switch to BTRFS or xfs or else, nor am I
not interested in using anything which would potentially endanger
existing installations (So far, reports I am reading from openSUSE
users don't necessarily sound convincing).
In other words, you'd have to do a lot of marketing and convincing
work to persuade users to use BTRFS/xfs etc.
Ralf

I think this is an important point. To make a fundamental change like
this, the rationale and benefits need to be very clearly spelled out,
and not just chase the new hotness (although 6-7 years in, I'm not sure
btrfs can be called new? XFS certainly can't!) ;)

IOWs, I'd like to see much more than "because it can do snapshots and
checksums" as the rationale; there are most definitely interesting things
that btrfs can do (or is working on doing), but as btrfs has evolved, so has
the rest of the Linux storage ecosystem: DM thin provisioning, xfs and ext4
metadata checksums, System Storage Manager (SSM) aiming for administration
ease, etc.

It's up to those proposing a new default to clearly spell out the compelling
advantage to the change.

-Eric

Matěj Cepl

2014-10-06 23:29:21 UTC

Post by Eric Sandeen
IOWs, I'd like to see much more than "because it can do snapshots and
checksums" as the rationale; there are most definitely interesting things
that btrfs can do (or is working on doing), but as btrfs has evolved, so has
the rest of the Linux storage ecosystem: DM thin provisioning, xfs and ext4
metadata checksums, System Storage Manager (SSM) aiming for administration
ease, etc.

So, would you say that the critical path for achieving Time
Slider is on the GUI side now? I mean what is shown on

(I don't even know
what language it is), and

(which is Spanish,
I guess). Unfortunately, I cannot find any screencast of this in
English or any other language I could actually comprehend.

MatÄj

Sergio Pascual

2014-10-07 08:29:58 UTC

Post by MatÄj Cepl
I mean what is shown on
http://youtu.be/-pNh1n3seTg (I don't even know
what language it is), and
http://youtu.be/kdjulcvhhuU (which is Spanish,
I guess).

Both are Spanish
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.fedoraproject.org/pipermail/devel/attachments/20141007/42438e26/attachment.html>

Ric Wheeler

2014-10-07 01:28:07 UTC

This post might be inappropriate. Click to display it.

Gene Czarcinski

2014-10-06 19:37:06 UTC

Post by Ralf Corsepius
Let me answer from the position of a mere user. It's not clear to me
why and when users should switch to BTRFS or xfs or else, nor am I not
interested in using anything which would potentially endanger existing
installations (So far, reports I am reading from openSUSE users don't
necessarily sound convincing).
In other words, you'd have to do a lot of marketing and convincing
work to persuade users to use BTRFS/xfs etc.

+1. BTRFS as the default is not ready for Fedora 23.

However, there are some things that could be done to make it easier for
those of us who want to make it easier to install Fedora onto a btrfs
filesystem.

My point was that unless and until there is more support for BTRFS by
members of the Fedora Project, btrfs cannot be made the default. That
is, *IF* it makes sense to have it be the default.

Gene
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.fedoraproject.org/pipermail/devel/attachments/20141006/c32da7de/attachment.html>

Gerald B. Cox

2014-10-07 15:04:54 UTC

On Mon, Oct 6, 2014 at 12:37 PM, Gene Czarcinski <gczarcinski at gmail.com>

Post by Gene Czarcinski
However, there are some things that could be done to make it easier for
those of us who want to make it easier to install Fedora onto a btrfs
filesystem.
My point was that unless and until there is more support for BTRFS by
members of the Fedora Project, btrfs cannot be made the default. That
is, *IF* it makes sense to have it be the default.

That is a good point. I've been using BTRFS Raid 6 across 6 1TB drives for
over a year now with (knock on wood) no issues. I did this because of the
simplicity of setting up RAID under BTRFS and for me at least that is the
biggest attraction. Maybe I'm mistaken, but there almost seemed (and still
is, IMO) an active discouragement for people to use this configuration.
You're not going to attract people to use and test something if you
simultaneously tell them it is toxic. Also, you can't jump from don't use
it, it isn't safe to ok, let's make it the system default.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.fedoraproject.org/pipermail/devel/attachments/20141007/fe4cfb69/attachment.html>

James Hogarth

2014-10-07 16:57:56 UTC

Post by Gerald B. Cox
On Mon, Oct 6, 2014 at 12:37 PM, Gene Czarcinski <gczarcinski at gmail.com>

Post by Gene Czarcinski
However, there are some things that could be done to make it easier for

those of us who want to make it easier to install Fedora onto a btrfs
filesystem.

Post by Gerald B. Cox

Post by Gene Czarcinski
My point was that unless and until there is more support for BTRFS by

members of the Fedora Project, btrfs cannot be made the default. That is,
IF it makes sense to have it be the default.

Post by Gerald B. Cox
That is a good point. I've been using BTRFS Raid 6 across 6 1TB drives

for over a year now with (knock on wood) no issues. I did this because of
the simplicity of setting up RAID under BTRFS and for me at least that is
the biggest attraction. Maybe I'm mistaken, but there almost seemed (and
still is, IMO) an active discouragement for people to use this
configuration. You're not going to attract people to use and test
something if you simultaneously tell them it is toxic. Also, you can't
jump from don't use it, it isn't safe to ok, let's make it the system
default.
In case you are unaware this is a terrible idea for any data you care about
at this point as scrubs don't work for raid5/6 right now and replacing a
lost drive as a result is ... Finicky at best.

Essentially for all intents raid 5/6 is as safe as raid0 and super
experimental without the benefit of a working scrub on raid0 at present ...

Any data you care about on btrfs must be raid1 or raid10 depending on the
number of drives (with metadata and system set to raid1).
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.fedoraproject.org/pipermail/devel/attachments/20141007/d85ff39e/attachment.html>

Gerald B. Cox

2014-10-07 17:19:03 UTC

Thanks James... I am aware of all the warnings. They might as well put up
a skull & crossbones. I have all my data backed up twice. But this is my
point... you don't say toxic and then simultaneously talk about proposing
it as the default file system on Fedora.

On Tue, Oct 7, 2014 at 9:57 AM, James Hogarth <james.hogarth at gmail.com>

Post by Gerald B. Cox
On Mon, Oct 6, 2014 at 12:37 PM, Gene Czarcinski <gczarcinski at gmail.com>

Post by Gene Czarcinski
However, there are some things that could be done to make it easier for

those of us who want to make it easier to install Fedora onto a btrfs
filesystem.

Post by Gerald B. Cox

Post by Gene Czarcinski
My point was that unless and until there is more support for BTRFS by

members of the Fedora Project, btrfs cannot be made the default. That is,
IF it makes sense to have it be the default.

Post by Gerald B. Cox
That is a good point. I've been using BTRFS Raid 6 across 6 1TB drives

for over a year now with (knock on wood) no issues. I did this because of
the simplicity of setting up RAID under BTRFS and for me at least that is
the biggest attraction. Maybe I'm mistaken, but there almost seemed (and
still is, IMO) an active discouragement for people to use this
configuration. You're not going to attract people to use and test
something if you simultaneously tell them it is toxic. Also, you can't
jump from don't use it, it isn't safe to ok, let's make it the system
default.
In case you are unaware this is a terrible idea for any data you care
about at this point as scrubs don't work for raid5/6 right now and
replacing a lost drive as a result is ... Finicky at best.
Essentially for all intents raid 5/6 is as safe as raid0 and super
experimental without the benefit of a working scrub on raid0 at present ...
Any data you care about on btrfs must be raid1 or raid10 depending on the
number of drives (with metadata and system set to raid1).
--
devel mailing list
devel at lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.fedoraproject.org/pipermail/devel/attachments/20141007/a45877f2/attachment.html>

Josh Boyer

2014-10-07 17:24:59 UTC

Thanks James... I am aware of all the warnings. They might as well put up a
skull & crossbones. I have all my data backed up twice. But this is my
point... you don't say toxic and then simultaneously talk about proposing it
as the default file system on Fedora.

Right... no single person is saying both things. We don't have
split-personality disorder here. Someone started discussing it as
default, and a bunch of other people chimed in that it wasn't ready.
Until those concerns are dealt with, it's not really even a candidate
for default consideration.

josh

Gerald B. Cox

2014-10-07 17:35:37 UTC

On Tue, Oct 7, 2014 at 10:24 AM, Josh Boyer <jwboyer at fedoraproject.org>

Post by Josh Boyer
Right... no single person is saying both things. We don't have
split-personality disorder here.

ROFL... Thanks for the clarification. Don't get me wrong though...I've
very excited about BTRFS; and looking forward to it's stable status.
Hopefully they can get past all the "you're nuts if you use this" warnings
in short order so more people will feel comfortable with trying it out.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.fedoraproject.org/pipermail/devel/attachments/20141007/c4c45453/attachment-0001.html>

Matthias Clasen

2014-10-07 18:05:48 UTC

Thanks James... I am aware of all the warnings. They might as well put up a
skull & crossbones. I have all my data backed up twice. But this is my
point... you don't say toxic and then simultaneously talk about proposing it
as the default file system on Fedora.

I think the point is somewhat valid though. To just keep repeating the
mantra 'its not ready' is not going to make it any more ready. If suse
can identify a stable subset of btrfs features and use it as their
default file system with those restrictions, why can't we do the same ?
The approach makes sense to me, at least...

Are the suse and fedora kernels that different that there is no synergy
to be had here ?

Reindl Harald

2014-10-07 18:15:34 UTC

Post by Matthias Clasen

Thanks James... I am aware of all the warnings. They might as well put up a
skull & crossbones. I have all my data backed up twice. But this is my
point... you don't say toxic and then simultaneously talk about proposing it
as the default file system on Fedora.

I think the point is somewhat valid though. To just keep repeating the
mantra 'its not ready' is not going to make it any more ready. If suse
can identify a stable subset of btrfs features and use it as their
default file system with those restrictions, why can't we do the same ?
The approach makes sense to me, at least...
Are the suse and fedora kernels that different that there is no synergy
to be had here?

it's not a difference of the kernel

it's the question if it *really* achieves anything for users to siwitch
a default with restrictions because *known* rough edges - nobody
supports the users which end in troubles and can throw away all the ext4
Google hits in case of trouble

for radical changes you should not ask "is it possible" - better ask
"has it really benefits beating the drawbacks and so is worth now"

if it ain't broken don't fix it!
is ext4 broken? no!

so why would you prefer to replace it *as default*?

corrupted data is the worst which can happen on a computer and god
beware you realize breakage too late and your backups contain damaged
data - do you really want that possible burden as default for people
know clue how to deal with it? people who know can use BTRFS anyways!

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: OpenPGP digital signature
URL: <http://lists.fedoraproject.org/pipermail/devel/attachments/20141007/a019d4de/attachment.sig>

Gerald B. Cox

2014-10-07 18:19:57 UTC

The developers say that it isn't ready. In my opinion default status
means that all features are at least deemed usable. It would be extremely
confusing to tell the community BTRFS is the default but it is not safe to
use certain features. I want it to be ready too... but the fact remains
it is not.

Post by Gerald B. Cox
Thanks James... I am aware of all the warnings. They might as well

put up a

Post by Gerald B. Cox
skull & crossbones. I have all my data backed up twice. But this is

Post by Gerald B. Cox
point... you don't say toxic and then simultaneously talk about

proposing it

Post by Gerald B. Cox
as the default file system on Fedora.

I think the point is somewhat valid though. To just keep repeating the
mantra 'its not ready' is not going to make it any more ready. If suse
can identify a stable subset of btrfs features and use it as their
default file system with those restrictions, why can't we do the same ?
The approach makes sense to me, at least...
Are the suse and fedora kernels that different that there is no synergy
to be had here ?
--
devel mailing list
devel at lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.fedoraproject.org/pipermail/devel/attachments/20141007/98737614/attachment.html>

Josef Bacik

2014-10-07 18:24:21 UTC

Post by Matthias Clasen

Thanks James... I am aware of all the warnings. They might as well put up a
skull & crossbones. I have all my data backed up twice. But this is my
point... you don't say toxic and then simultaneously talk about proposing it
as the default file system on Fedora.

Because they still have the support staff for when users don't listen,
Fedora doesn't. Thanks,

Josef

Josh Boyer

2014-10-07 19:15:53 UTC

Post by Matthias Clasen

Thanks James... I am aware of all the warnings. They might as well put up a
skull & crossbones. I have all my data backed up twice. But this is my
point... you don't say toxic and then simultaneously talk about proposing it
as the default file system on Fedora.

Because they still have the support staff for when users don't listen,
Fedora doesn't.

Right.

As an aside, I looked at their 3.16.2-1.1.gdcee397 kernel-source SRPM.
I can't find any patches that limit btrfs usage. I could totally be
wrong, but if someone knows of a patch that limits the features please
point me to it.

josh

Thorsten Leemhuis

2014-10-08 05:52:23 UTC

Post by Josh Boyer
[âŠ]

Post by Matthias Clasen
I think the point is somewhat valid though. To just keep repeating the
mantra 'its not ready' is not going to make it any more ready. If suse
can identify a stable subset of btrfs features and use it as their
default file system with those restrictions, why can't we do the same ?
The approach makes sense to me, at least...

Because they still have the support staff for when users don't listen,
Fedora doesn't.

As an aside, I looked at their 3.16.2-1.1.gdcee397 kernel-source SRPM.
I can't find any patches that limit btrfs usage. I could totally be
wrong, but if someone knows of a patch that limits the features please
point me to it.

Due to a coincidence I yesterday took a quick look myself and didn't
spot anything. But in case you haven't looked further: I found one in
the SLE-Kernels:

http://kernel.opensuse.org/cgit/kernel-source/tree/patches.suse/btrfs-8888-add-allow_unsupported-module-parameter.patch?h=SLE11-SP3
http://kernel.opensuse.org/cgit/kernel-source/tree/patches.suse/btrfs-8888-add-allow_unsupported-module-parameter.patch?h=SLE12

HTH

CU
knurd

P.S.: BTW, Jeff Mahoney a year ago posted a small table what btrfs
features they considered supported and unsupported:
http://thread.gmane.org/gmane.linux.suse.opensuse.devel/52669/
Is anyone aware of a more current table like that? Or Josef, do the
Btrfs developers maintain one somewhere? I'd welcome one.

Thorsten Leemhuis

2014-10-08 06:28:06 UTC

Post by Josh Boyer
[âŠ]

Because they still have the support staff for when users don't listen,
Fedora doesn't.

Due to a coincidence I yesterday took a quick look myself and didn't
spot anything. But in case you haven't looked further: I found one in
http://kernel.opensuse.org/cgit/kernel-source/tree/patches.suse/btrfs-8888-add-allow_unsupported-module-parameter.patch?h=SLE11-SP3
http://kernel.opensuse.org/cgit/kernel-source/tree/patches.suse/btrfs-8888-add-allow_unsupported-module-parameter.patch?h=SLE12

BTW & TWIMC, their solution to problems like this
http://aseigo.blogspot.de/2014/09/btrfs-rebalancing.html (ran into this
myself yesterday) seems to be a package called btrfsmaintenance that was
introduced a few days ago. Quoting
http://article.gmane.org/gmane.linux.suse.opensuse.devel/59787/
"""

Post by Thorsten Leemhuis
From: David Sterba <dsterba <at> suse.cz>
Subject: Introduce btrfsmaintenance package to Factory
Newsgroups: gmane.linux.suse.opensuse.devel
Date: 2014-10-06 15:40:37 GMT (1 day, 14 hours and 40 minutes ago)
Hi,
let me introduce a new package that supplements the btrfs filesystem and aims
to automate a few maintenance tasks. This means the scrub, balance, trim or
defragmentation. The package comes from SLE12 where btrfs is going to be the
default filesystem as well.
Each of the tasks can be turned on/off and configured independently. The
default config values were selected to fit the default installation profile.
* scrub - go through all medatada/data and verify the checksums, default period
is one month
* balance - the balance command can do a lot of things, in general moves data
around in big chunks, here we use it to reclaim back the space of the
underused chunks so it can be allocated again according to current needs
The point is to prevent some corner cases where it's not possible to eg.
allocate new metadata chunks because the whole device space is reserved for all
the chunks, although the total space occupied is smaller and the allocation
should succeed.
* trim - run TRIM on the filesystem using the 'fstrim' utility, makes sense for
SSD devices.
* defrag - run defrag on configured directories. This is for convenience and
not necessary.
There's a separate defragmentation task that happens automatically and
defragments only the RPM database files in /var/lib/rpm. This is done via a
zypper plugin and the defrag pass triggers at the end of the installation.
This improves reading the RPM databases later, but the installation process
fragments the files very quickly so it's not likely to bring a significant
speedup here.
Cron takes care of periodic execution of the scripts, but they can be run any
time directly from /usr/share/btrfs/maintenance/, respecting the configured
values in /etc/sysconfig/btrfsmaintenance.
If the period is changed manually, the cron symlinks have to be refreshed, use
"systemctl restart btrfsmaintenance-refresh" (or the
"rcbtrfsmaintenance-refresh" shortcut). Changing the period via yast2 sysconfig
editor triggers the refresh automatically.
The project lives in obs://filesystems/btrfsmaintenance and I'm going to submit
it to Factory.
I'd like to ask volunteers to give it some testing. Feedback is welcome.
Thanks,
David

"""
The package in the opensuse obs:
https://build.opensuse.org/package/show/filesystems/btrfsmaintenance

Seems to mostly consist of shell scripts.

CU
knurd

Gene Czarcinski

2014-10-08 13:12:20 UTC

Post by Josh Boyer
[âŠ]

Because they still have the support staff for when users don't listen,
Fedora doesn't.

"""
https://build.opensuse.org/package/show/filesystems/btrfsmaintenance
Seems to mostly consist of shell scripts.
CU
knurd

The new btrfsmaintenance package looks like something that should be
added to Fedora regardless of btrfs status ... again this is something
that Fedora could do which would make things easy to use and promote
knowledgeable use of BTRFS without making it default. It is the
"support/infrastruction" packages that need more attention and the the
btrfs-kernel or btrfs-progs themselves.

Gene

Josh Boyer

2014-10-08 12:50:13 UTC

Post by Josh Boyer
[...]

Because they still have the support staff for when users don't listen,
Fedora doesn't.

Yes, I found the same yesterday after doing some digging. It's not in
OpenSuSE 13.2 though, so it seems they're just going with whatever
upstream provides there.

I'm not thrilled with adding that patch to Fedora at all. It
accomplishes the goal, but it would break any existing user that is
using one of those btrfs features because the mount would fail.
Fedora would likely just have to taint and then ignore reports with
the taint set, which doesn't make for a great user experience either.
They get away with this in SLE12 because it's roughly the first time
btrfs is available in a supported fashion. I suspect they hit the
same concerns in OpenSuSE that Fedora would though, which is why it
isn't there.

If someone was interested in the history of the patch in OpenSuSE and
wants to dig into it, that would be neat to read.

josh

Thorsten Leemhuis

2014-10-08 13:39:08 UTC

[...]

http://kernel.opensuse.org/cgit/kernel-source/tree/patches.suse/btrfs-8888-add-allow_unsupported-module-parameter.patch?h=SLE11-SP3
http://kernel.opensuse.org/cgit/kernel-source/tree/patches.suse/btrfs-8888-add-allow_unsupported-module-parameter.patch?h=SLE12

[...]
I'm not thrilled with adding that patch to Fedora at all.

Fully agreed.

Post by Josh Boyer
[âŠ] They get away with this in SLE12 because it's roughly the first time
btrfs is available in a supported fashion. [âŠ]

Well, it's supported since SLE11SP2 already, which is more than two
years old, but the point in the end is the same, yes. But FWIW, it seems
that simply how they work afaics, as they do something similar to ext4, too:

http://kernel.opensuse.org/cgit/kernel-source/tree/patches.suse/ext4-unsupported-features.patch?h=SLE12

CU
knurd

Eric Sandeen

2014-10-09 15:27:15 UTC

[...]

[...]
I'm not thrilled with adding that patch to Fedora at all.

Fully agreed.

Post by Josh Boyer
[âŠ] They get away with this in SLE12 because it's roughly the first time
btrfs is available in a supported fashion. [âŠ]

Well, it's supported since SLE11SP2 already, which is more than two
years old, but the point in the end is the same, yes. But FWIW, it seems
http://kernel.opensuse.org/cgit/kernel-source/tree/patches.suse/ext4-unsupported-features.patch?h=SLE12
CU
knurd

Interesting, bigalloc and checksums - yeah, I probably would have chosen those,
too, for now.

I think there's a little difference, though, in that bigalloc in particular,
and checksums to some degree, are really kind of niche / corner case features of
ext4. (Who here even knew ext4 had a "bigalloc" feature, raise your hand!)

The list of unsupported btrfs features seems like a lot of core advertised
functionality - no compression, no raid5, no device replace, no btrfs receive...

-Eric

Gene Czarcinski

2014-10-08 12:39:53 UTC

Post by Gerald B. Cox
On Mon, Oct 6, 2014 at 12:37 PM, Gene Czarcinski
However, there are some things that could be done to make it
easier for those of us who want to make it easier to install
Fedora onto a btrfs filesystem.
My point was that unless and until there is more support for BTRFS
by members of the Fedora Project, btrfs cannot be made the
default. That is, *IF* it makes sense to have it be the default.
That is a good point. I've been using BTRFS Raid 6 across 6 1TB
drives for over a year now with (knock on wood) no issues. I did this
because of the simplicity of setting up RAID under BTRFS and for me at
least that is the biggest attraction. Maybe I'm mistaken, but there
almost seemed (and still is, IMO) an active discouragement for people
to use this configuration. You're not going to attract people to use
and test something if you simultaneously tell them it is toxic. Also,
you can't jump from don't use it, it isn't safe to ok, let's make it
the system default.

I also use raid (d=raid0, m=raid1) and it is nice to have a multidevice
btrfs volume where you can add an additional device to the volume and
also have btrfs shuffle the data to spread it over all the devices.

Also, if there are bugs then they need to be discovered and fixed. They
will not get discovered unless btrfs is used. If the btrfs developers
knew there are a bug, it would have been already fixed.

Gene

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.fedoraproject.org/pipermail/devel/attachments/20141008/67d1876a/attachment.html>

Josh Boyer

2014-10-06 12:49:50 UTC

Post by Josef Bacik
On Oct 2, 2014 11:32 PM, "Andre Robatino" <robatino at fedoraproject.org>

Nope, I understand. However, what I said is with the understanding to
change to using BTRFS as the default ... whenever it takes place and I stand
by my statement of needing more attention as a requirement.

The attention needed is a prerequisite for it to even remotely be
considered for default. I'm explaining the people working on the
kernel aren't staffed to give that attention, so if people would like
btrfs to be the default filesystem in Fedora they need to start
working on it now.

This isn't a requirement for btrfs by default. It's a nice to have.

Um... sure? Except nobody cares if /boot is btrfs or not. It's a
nicety that results in btrfs being the main fs in use for the system,
but it isn't a requirement. You can use it for the rootfs and for
home just fine without /boot being btrfs. (It's also somewhat of a
pipe dream, given that the EFI system partition is never going to be
btrfs and that's another mountpoint under /boot.)

Post by Gene Czarcinski
Now, there is another question which has not been voiced: what is the "plan"
for filessystems in Fedora (and by implication RHEL)? Is it BTRFS? Or,
perhaps is it LVM with XFS? IIRC, some time ago it was stated that the plan
was to move to BTRFS. It is not clear to me that everyone is onboard with
that decision. Or, perhaps that decision is being reconsidered.

Depends. There is no single grand unified plan for Fedora
filesystems. Believe me, if there was that would be amazing.

Workstation would like to use btrfs for a number of nice desktop
technologies (like easy backups, time slider, etc). It's not ready,
so WS stuck with the existing default of ext4. We discussed (briefly)
following SuSE's approach and limiting the features possible with
btrfs, but after discussing with Josef we decided to forego that as
well. There are alternatives that could be used (like dm snapshots)
but the userspace work wasn't going to happen for F21 in any case.

Cloud uses ext4. I don't believe they have any benefit to switching
to any other filesystem so it doesn't matter to them.

Server already switched to using XFS as the default fs, which makes
sense given that RHEL 7 defaults to XFS. I believe btrfs remains a
Technology Preview in RHEL7. My take on that is that it is never
going to be default for RHEL 7 or even move out of Tech Preview, but I
have no real idea. I also have no idea what RHEL 8 will bring as I
don't work on RHEL and even if I did I wouldn't be talking about it at
this point.

As for being onboard with any sort of decision, it basically boils
down to being able to prove that btrfs is ready to be the default FS,
and being able to support it in that manner. Right now neither are
the case.

Post by Josef Bacik
Anyway, we are getting close.

If it's getting close, it's entirely because of Josef's and the SuSE
team's efforts. They should be applauded.

Yes, they should! But Fedora moving to increased use of BTRFS seems to be
receding at the speed of time: it is always a couple of releases from now;
go away I am busy with other stuff. Sorry is this bothers some people but
that is the impression I get.

It's always a couple releases away because btrfs is always a couple of
bugs/issues away from being ready. Your impression doesn't bother
me, but it's slightly wrong. It's not "go away, I'm busy". It's "I'm
busy, if you want to make this happen please find people to put in the
work to make it so." There's a difference there that's worth
noticing.

josh

Josef Bacik

2014-10-06 12:54:49 UTC

Post by Josef Bacik
On Oct 2, 2014 11:32 PM, "Andre Robatino" <robatino at fedoraproject.org>

Nope, I understand. However, what I said is with the understanding to

change to using BTRFS as the default ... whenever it takes place and I
stand by my statement of needing more attention as a requirement.

This isn't a requirement for btrfs by default. It's a nice to have.

Using btrfs by default needs more users to want to use it. Making it

easier for more users to install into btrfs will see increased use. It
will also demonstrate that Fedora development/maintenance is paying
attention to BTRFS.

Post by Gene Czarcinski
Now, there is another question which has not been voiced: what is the

"plan" for filessystems in Fedora (and by implication RHEL)? Is it BTRFS?
Or, perhaps is it LVM with XFS? IIRC, some time ago it was stated that the
plan was to move to BTRFS. It is not clear to me that everyone is onboard
with that decision. Or, perhaps that decision is being reconsidered.

Post by Josef Bacik
Anyway, we are getting close.

If it's getting close, it's entirely because of Josef's and the SuSE
team's efforts. They should be applauded.

Yes, they should! But Fedora moving to increased use of BTRFS seems to

be receding at the speed of time: it is always a couple of releases from
now; go away I am busy with other stuff. Sorry is this bothers some people
but that is the impression I get.

Well that's exactly what it is, go away I'm busy with other stuff :). The
fact is I'm the only one who can drive btrfs as the default filesystem
feature in Fedora, and since I've left Red Hat that has become much less of
an priority for me. But my "other stuff" is still mostly related to btrfs,
so its not like this has just been abandoned, the focus has just shifted
and I no longer feel like we need to be using btrfs as the default fs in
Fedora to have a successful project, so it got moved down the priority
list. It will happen, and when it happens it will be relatively painless
because Suse will have worked out a lot of the distro esque kinks and us at
Facebook will have worked out a lot of the at scale kinks. Thanks,

Josef
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.fedoraproject.org/pipermail/devel/attachments/20141006/3e96257d/attachment.html>

Ric Wheeler

2014-10-06 13:50:19 UTC

Post by Josef Bacik
Well that's exactly what it is, go away I'm busy with other stuff :). The
fact is I'm the only one who can drive btrfs as the default filesystem feature
in Fedora, and since I've left Red Hat that has become much less of an
priority for me. But my "other stuff" is still mostly related to btrfs, so
its not like this has just been abandoned, the focus has just shifted and I no
longer feel like we need to be using btrfs as the default fs in Fedora to have
a successful project, so it got moved down the priority list. It will happen,
and when it happens it will be relatively painless because Suse will have
worked out a lot of the distro esque kinks and us at Facebook will have worked
out a lot of the at scale kinks. Thanks,
Josef

I think that the out of space issues are not that different from any file system
on write enabled snapshots - it certainly can be mysterious and confuse users,
but that is something that we have to deal with in order to get this kind of
sophistication into end users hands (documentation? better tooling like the
snapper tool, etc?).

One of the harder challenges I think for btrfs is still getting the repair tools
rock solid - how is our track record these days with repairing btrfs after bad
things happen :) ?

Regards,

Ric

Eric Sandeen

2014-10-06 14:20:55 UTC

Post by Josef Bacik
Well that's exactly what it is, go away I'm busy with other stuff
:). The fact is I'm the only one who can drive btrfs as the
default filesystem feature in Fedora, and since I've left Red Hat
that has become much less of an priority for me. But my "other
stuff" is still mostly related to btrfs, so its not like this has
just been abandoned, the focus has just shifted and I no longer
feel like we need to be using btrfs as the default fs in Fedora to
have a successful project, so it got moved down the priority list.
It will happen, and when it happens it will be relatively painless
because Suse will have worked out a lot of the distro esque kinks
and us at Facebook will have worked out a lot of the at scale
kinks. Thanks,
Josef

I think that the out of space issues are not that different from any
file system on write enabled snapshots - it certainly can be
mysterious and confuse users, but that is something that we have to
deal with in order to get this kind of sophistication into end users
hands (documentation? better tooling like the snapper tool, etc?).
One of the harder challenges I think for btrfs is still getting the
repair tools rock solid - how is our track record these days with
repairing btrfs after bad things happen :) ?

In my recent (past couple weeks) testing, it's dismal, although a bunch
of fsck patches have shown up on the list which I haven't tried yet.

Doing what I considered to be light fuzzing of a filesystem, and attempting
a btrfsck & mount led to segfaults, aborts, and mount failures the vast
majority of the time - more or less often, depending on the particular metadata
layout. Other filesystems fared much better in this testing.

And the best-practice repair strategy with btrfs is still not clear to me;
there is btrfs check (aka btrfsck) of course, with various suboptions the
admin should know about: init-csum-tree, init-extent-tree, backup, subvol-extents;
several of these are not documented ... and also mount options:
-o degraded, -o recovery, -o skip_balance. And other tools; btrfs
rescue, with various suboptions - super-recovery, chunk-recover.
There's btrfs scrub, which will work online and "Errors are corrected along the
way if possible." and btrfs-rescue, a last-ditch effort to extract files
from the smoking remains of an otherwise unrescuable filesystem.

It may be that I'm just not up to speed on btrfs - I've realized that I can't
be an in-depth expert on 3 different filesystems - but to me, the filesystem
repair and recovery plan for btrfs is not at all reliable or obvious or clear.

(I need to send this same post to the btrfs-list, I guess, but I did want to
point out that it is, to me, a big concern for Fedora decision-making. btrfs
was first proposed as default in F17, 3 years ago, and ISTR reliable fsck being a
gating item back then. And now here we are...)

Thanks,
-Eric

Josef Bacik

2014-10-06 14:26:02 UTC

Post by Josef Bacik
Well that's exactly what it is, go away I'm busy with other stuff :). The
fact is I'm the only one who can drive btrfs as the default filesystem
feature in Fedora, and since I've left Red Hat that has become much less of
an priority for me. But my "other stuff" is still mostly related to btrfs,
so its not like this has just been abandoned, the focus has just shifted and
I no longer feel like we need to be using btrfs as the default fs in Fedora
to have a successful project, so it got moved down the priority list. It
will happen, and when it happens it will be relatively painless because Suse
will have worked out a lot of the distro esque kinks and us at Facebook will
have worked out a lot of the at scale kinks. Thanks,
Josef

I think that the out of space issues are not that different from any file
system on write enabled snapshots - it certainly can be mysterious and
confuse users, but that is something that we have to deal with in order to
get this kind of sophistication into end users hands (documentation? better
tooling like the snapper tool, etc?).
One of the harder challenges I think for btrfs is still getting the repair
tools rock solid - how is our track record these days with repairing btrfs
after bad things happen :) ?

Funny you should ask, I just added a bunch of functionality last week!
So right now our fsck fixes anything wrong in the extent tree, which
is where a majority of the problems happen. The other side of course
is the fs tree which is infinitely easier to deal with, but I usually
wait for a user with a broken fs to fix before adding the ability to
fix it into fsck so I'm sure we have a good testcase to work against.
Every fix I add to fsck gets a test image added to btrfs-progs to make
sure we're never regressing.

Obviously we aren't in xfs/e2fsprogs territory, but it'll fix 90% of
the problems and then the other 10% are just a matter of having an
example to work off of. Thanks,

Josef

Eric Sandeen

2014-10-06 15:52:13 UTC

Post by Josef Bacik
Obviously we aren't in xfs/e2fsprogs territory, but it'll fix 90% of
the problems and then the other 10% are just a matter of having an
example to work off of. Thanks,
Josef

Josef, just as a datapoint: after corrupting 32k random bytes on a 2G
image lightly populated and made with default mkfs options, then running
fsck with all of your recent fixes, I got 9 mount failures out of 20
attempts, 55% success.

Running the same test, but w/ 2 devices, each randomly damaged to
the same extent, and created with -m raid1 -d raid0, I get
10 failures out of 20, 50% success.

(Note that this is just a low-bar "will it mount" test, I'm
not looking at what's in the repaired filesystem at all).

What sort of testing yielded your 90% success rate?

Thanks,
-Eric

Josef Bacik

2014-10-06 15:58:08 UTC

Post by Eric Sandeen

Post by Josef Bacik
Obviously we aren't in xfs/e2fsprogs territory, but it'll fix 90% of
the problems and then the other 10% are just a matter of having an
example to work off of. Thanks,
Josef

Josef, just as a datapoint: after corrupting 32k random bytes on a 2G
image lightly populated and made with default mkfs options, then running
fsck with all of your recent fixes, I got 9 mount failures out of 20
attempts, 55% success.
Running the same test, but w/ 2 devices, each randomly damaged to
the same extent, and created with -m raid1 -d raid0, I get
10 failures out of 20, 50% success.
(Note that this is just a low-bar "will it mount" test, I'm
not looking at what's in the repaired filesystem at all).
What sort of testing yielded your 90% success rate?

The 90% is just off the top of my head, I used to be doing lots of
fsck work for users with corrupted fs, I do that a lot less now, so it
seems like we're making progress. I have never done any fsfuzzer
testing, I'll put that on the list. Thanks,

Josef

Ric Wheeler

2014-10-06 18:19:06 UTC

This post might be inappropriate. Click to display it.

Josef Bacik

2014-10-06 19:12:46 UTC