Re: Strange prformance degradation when COW writes happen at fixed offsets

2012/2/24 Nik Markovic [off-list ref]:

To add... I also tried nodatasum (only) and nodatacow otions. I found
somewhere that nodatacow doesn't really mean tthat COW is disabled.
Test data is still the same - CPU spikes and times are the same.

On Fri, Feb 24, 2012 at 2:38 PM, Nik Markovic <nmarkovi.navteq@gmail.=

com> wrote:

quoted

On Fri, Feb 24, 2012 at 12:38 AM, Duncan [off-list ref] wrot=

e:

quoted

quoted

Nik Markovic posted on Thu, 23 Feb 2012 20:31:02 -0600 as excerpted=

:

quoted

quoted

quoted

I noticed a few errors in the script that I used. I corrected it a=

nd it

quoted

quoted

quoted

seems that degradation is occurring even at fully random writes:

I don't have an ssd, but is it possible that you're simply seeing e=

rase-

quoted

quoted

block related degradation due to multi-write-block sized erase-bloc=

ks?

quoted

quoted

It seems to me that when originally written to the btrfs-on-ssd, th=

e file

quoted

quoted

will likely be written block-sequentially enough that the file as a=

 whole

quoted

quoted

takes up relatively few erase-blocks. =A0As you COW-write individua=

l

quoted

quoted

blocks, they'll be written elsewhere, perhaps all the changed block=

s to a

quoted

quoted

new erase-block, perhaps each to a different erase block.

This is a very interesting insight. I wasn't even aware of the
erase-block issue, so I did some reading up on it...

quoted

As you increase the successive COW generation count, the file's fil=

e-

quoted

quoted

system/write blocks will be spread thru more and more erase-blocks,
basically fragmentation but of the SSD-critical type, into more and=

 more

quoted

quoted

erase blocks, thus affecting modification and removal time but not =

read

quoted

quoted

time.

OK, so time to write would increase due to fragmentation and writing=

,

quoted

it now makes sense (though I don't see why small writes would affect
this, but my concerns are not writes anyway), but why would cp
--reflink time increase so much. Yes, new extents would be created,
but btrfs doesn't write into data blocks, does it? I figured its
metadata would be kept in one place. I figure the only thing BTRFS
would do on cp --reflink=3Dalways:
1. Take a collection of extents owned by source.
2. Make the new copy use the same collection of extents.
3. Write the collection of extents to the "directory".

Now this process seems to be CPU intensive. When I remove or make a
reflink copy, one core pikes up to 100%, which tells me that there's=

 a

quoted

performance issue there, not an ssd issue. Also, only one CPU thread
is being used for this. I figured that I can improve this by some
setting. Maybe thread_pool mount option? Are there any updates in
later kernels that I should possibly pick up?

[...]

Unless I am wrong, this would disable COW completely and reflink cop=

y.

quoted

Reflinks are a crucial component and the sole
reason I picked BTRFS for the system that I am writing for my compan=

y.

quoted

The autodefrag option addresses multiple writes. Writing is not the
problem, but cp --reflink should be near-instant. That was the reaso=

n

quoted

we chose BTRFS over ZFS, which seemed to be the only feasible
alternative. ZFS snapshot complicate the design and deduplication co=

py

quoted

time is the same as (or not much better than) raw copy.

[...]

As I mentioned above, the COW is the crucial component of our system=

,

quoted

XFS won't do. Our system does not do random writes. In fact it is
mainly heavy on read operation. The system does occasional "rotation
of rust" on large files in a way that version control system would
(large files are modified and then used as a new baseline)

The symptoms you are reporting are quite similar to what I'm seeing in
our Ceph cluster:

http://comments.gmane.org/gmane.comp.file-systems.btrfs/15413

AFAIK, Chris and Josef are working on it, but you'll have to wait for
kernel 3.4, until this will be available in mainline. If you are
feeling adventurous, you could try the patches in Josef's git tree,
but I think it's still experimental.

Regards,
Christian
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html