Thread (25 messages) 25 messages, 4 authors, 1d ago

Re: [PATCH 1/2] http: use unique tempfiles for packfile URI downloads

From: Ted Nyman <hidden>
Date: 2026-07-14 01:58:30

While that does sound like a safe and correct approach, stepping
back briefly, would it not be wasteful for the second process to
download the same packfile that the first has already started
downloading?
Yes. If two fetches overlap, the second download is redundant.
Are there better ways for these processes to coordinate with each
other? Instead of appending to the file, what if the second process
uses a predictable temporary name (which we already use) to open a
new file with O_CREAT | O_EXCL to avoid this redundant work?
Using the existing pack-<hash>.pack.temp name with O_CREAT | O_EXCL
would prevent concurrent writes, but EEXIST alone would not
distinguish an in-progress download from one left by an earlier
failed or interrupted invocation. The existing .pack.temp name is not
covered by the tmp_* pruning path, so simply waiting for it to
disappear could leave a fetch stuck after a crash.

The waiting case would also need a complete handoff. If the first
process finishes, the second would need to notice the installed pack
and account for the expected index-pack result and keep state. If the
first process fails and removes its temporary file, the second would
need to retry as the downloader. That is possible, but introduces
cross-process coordination and a timeout policy in http-fetch.

The unique tempfile preserves the existing "download, index, then
install" behavior for each invocation and fixes both the
concurrent-append and EOF-resume failures. Avoiding the duplicate
transfer would be useful for large packs, but I would prefer to keep
that as a follow-up unless you think it is necessary for this
correctness fix.

Thanks,
Ted
Keyboard shortcuts
hback out one level
jnext message in thread
kprevious message in thread
ldrill in
Escclose help / fold thread tree
?toggle this help