Brian Gerst [off-list ref] writes:
quoted
Is it possible to determine the object locations inside the remote
pack
file? If so, it would be possible to use Range: headers to download
selected objects from a pack.
That's what the .idx file is for, except that after you fetch
the range, you may find you would need something else that the
object is delta against.
On Sun, Oct 16, 2005 at 09:56:49AM -0700, Junio C Hamano wrote:
That's what the .idx file is for, except that after you fetch
the range, you may find you would need something else that the
object is delta against.
Would it make sense to load the pack indexes for each base up front,
and then fetch individual objects from a pack if they exist in one of
a base's pack indexes? In such a case, it may not even make sense to
try fetching the object directly first.
What are the circumstances under which it makes more sense to fetch the
whole pack rather than fetching individual objects from it?
--
For a successful technology, reality must take precedence over public
relations, for nature cannot be fooled.
On Sun, 16 Oct 2005, Nick Hengeveld wrote:
On Sun, Oct 16, 2005 at 09:56:49AM -0700, Junio C Hamano wrote:
quoted
That's what the .idx file is for, except that after you fetch
the range, you may find you would need something else that the
object is delta against.
Would it make sense to load the pack indexes for each base up front,
and then fetch individual objects from a pack if they exist in one of
a base's pack indexes? In such a case, it may not even make sense to
try fetching the object directly first.
At the start, you have the option of either fetching the list of packs or
the object. There are three cases:
1) the object isn't available separately; we need to fetch the list of
packs to find it in a pack.
2) there aren't any new packs; we need to fetch the object individually.
3) the object is present both individually and in a pack.
(2) is more common than (1), because we don't repack every update. (3)
doesn't happen at all, currently, because we prune after packing. So it
makes most sense to try the object at once.
On the other hand, the parallel code should probably do both at the same
time, since it can, and it only causes notable latency, not bandwidth. We
probably also ought to speculatively get any new index files in parallel
with whatever else we're doing, since it is likely that we'll need some
pack at some point, and then we'll need all the index files to decide what
pack to get.
What are the circumstances under which it makes more sense to fetch the
whole pack rather than fetching individual objects from it?
I'm not sure there's a good way of deciding without a plan for what
conditions cause there to be a choice.
-Daniel
*This .sig left intentionally blank*