Re: [PATCH] Teach "git add" and friends to be paranoid

On Mon, 22 Feb 2010, Dmitry Potapov wrote:

Currently, we calculate SHA-1, then lookup whether the object with this
SHA-1 exists, and if it does not, then deflate and write it to the
object storage. So, we avoid deflate and write cost if this object
already exists. Moreover, when we deflate data, we create the temporary
file in the same directory where the target object will be stored, thus
avoiding cross-directory rename (which is important for some reason, but
I don't remember why).  So, creating the temporary file requires the
knowledge first two digits of SHA-1, which you cannot know without
calculation SHA-1.

Even that initial SHA1 calculation can be done in chunks.  Worth doing 
for large enough files only though.


Nicolas