Btree directories (Re: Status of HFS+ support)

3 messages, 3 authors, 2000-08-30 · open the first message on its own page

Btree directories (Re: Status of HFS+ support)

From: Matthew Wilcox <hidden>
Date: 2000-08-29 17:18:51

On Tue, Aug 29, 2000 at 06:40:08PM +0200, Halfmann, Klaus wrote:
On the other hand: The hfs acces is normally not needed for heavy
concurrent acces. (Mhh, there might be AFP Servers publishing HFS
partions) For now it should be enough to have a global update
lock on the root of the node. Im not firm with kernel code yet, dont
ask me about the details :-)
It's not that easy.  Unfortunately, getdents is a nasty interface (I
actually haven't seen a good one -- anyone know of one which doesn't
suffer from this problem?)

You can lock while you're in a call, but eventually, you fill up the
user's buffer and return.  At that point, you have to drop the lock
because they might never call you again.  So you have to consider
the case of a file being added or removed between calls to getdents.
Current HFS doesn't even pretend to try.  It just stores an index (0
.. n-1) and you pick up from there.  So sometimes you get files twice,
sometimes files don't show up at all.

I suspect you need to store the key of the item you just found, and
then continue filling in the buffer from there on subsequent calls.
The trouble is that there frequently isn't enough space available to
do that.  The proposed ext2 btree extensins needed 64 bits of space and
there was only 32 bits available.  What size keys does HFS+ have?

Hmmm.. now the LFS patches have gone in and f_pos is now a 64-bit quantity,
this sounds more plausible.  I'd be curious to hear from the ReiserFS
people how they solved this problem.

There's an additional complication of rewinddir/seekdir/telldir, but
let's get onto that later.
I'd really like to know how the concurrency of the kernel filesystem
should work, can anybody feed me with documentations about that ?
I've cc'd linux-fsdevel, where people can fill you in.

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: Btree directories (Re: Status of HFS+ support)

From: <hidden>
Date: 2000-08-30 02:22:05

Matthew Wilcox wrote:
You can lock while you're in a call, but eventually, you fill up the
user's buffer and return.  At that point, you have to drop the lock
because they might never call you again.  So you have to consider
the case of a file being added or removed between calls to getdents.
Current HFS doesn't even pretend to try.  It just stores an index (0
.. n-1) and you pick up from there.  So sometimes you get files twice,
sometimes files don't show up at all.

I suspect you need to store the key of the item you just found, and
then continue filling in the buffer from there on subsequent calls.
The trouble is that there frequently isn't enough space available to
do that.  The proposed ext2 btree extensins needed 64 bits of space and
there was only 32 bits available.  What size keys does HFS+ have?
Catalog keys in HFS+ can be anywhere from 8 bytes all the way up to
518 bytes. I actually do save the key and do a search in the btree
in my HFS+ module. (It doesn't seem to work however. And yes, I'm
helping out Klaus with his code as well as working on a kernel
module with completely separate code.)
Hmmm.. now the LFS patches have gone in and f_pos is now a 64-bit quantity,
this sounds more plausible.  I'd be curious to hear from the ReiserFS
people how they solved this problem.
The support for 64 bit filesystems is important for HFS+ as well, since
it uses 64 bit file sizes for all files.
quoted
I'd really like to know how the concurrency of the kernel filesystem
should work, can anybody feed me with documentations about that ?
I've cc'd linux-fsdevel, where people can fill you in.
I've been ignoring this issue, but I certainly would love pointers to
more documentation.

	Brad Boyer
	flar@pants.nu


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: Btree directories (Re: Status of HFS+ support)

From: Chris Mason <hidden>
Date: 2000-08-30 14:25:46

On 8/29/00, 1:18:51 PM, Matthew Wilcox [off-list ref] wrote regarding
Btree directories (Re: Status of HFS+ support):

[ 32 bit directory offsets ]
Hmmm.. now the LFS patches have gone in and f_pos is now a 64-bit
quantity,
this sounds more plausible.  I'd be curious to hear from the ReiserFS
people how they solved this problem.
In reiserfs, the offset in the directory is a hash of the file name.  The
directories are sparse, so we could have a directory with 4 items,
offsets 1, 2, 32458, 2million.  The hash of a given item never changes,
new items can be inserted or removed from either side of any existing
item (except . and ..)

-chris


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/
Keyboard shortcuts
hback out one level
jnext message in thread
kprevious message in thread
ldrill in
Escclose help / fold thread tree
?toggle this help