Re: Are binary xdeltas only used if you use git-gc?

RESOLVED!!!

Finally...
What happened was actually quite reasonable, in hindsight...
As I said in the original mail, this was what I did:

cp version7.1.tar version7.2.tar
git add version7.2.tar
git commit -m "same data as old, so git will use old blob"
echo MAGICPLACE read below...
cp /path/to/work/realNewVersion7.2.tar version7.2.tar
git add version7.2.tar
git commit -m "and now, commit the really new version, so git can xdelta"
git push --thin

The problem was solved (that is, the "git push" became optimal,
when I added a "git push" right after the MAGICPLACE mark above...
In that way, the remote repo learned about the "dummy" commit that
referenced the old blob... and when I did the subsequent "git push"
at the end, the remote side could see that it already had this "dummy"
commit to "xdelta on", and that it only needed the delta...

Originally, when I used only one "git push --thin" at the end, the remote
side didn't have the "dummy" commit, so it probably said: "I can't
apply a delta, give me the full object".

Phew.

So it seems that if you must introduce a new file that is
very similar to an existing one (in my case, a new version
of software kept in an uncompressed .tar file),
you have to do what I did above to allow for optimal "git push"es:
that is:

1. Create the new filename by just copying the old
 (so the old blob is used)
2. commit
3. PUSH
4. copy the real new file
5. commit
6. PUSH.

If you omit the middle PUSH in step 3, neither "git push", nor "git push --thin"
can realize that this new file can be "incrementally built" on the remote side
(even though git-gc totally squashes it in the pack).

Thanks to all the people who responded, and especially Nicolas...
Merci!


On 11/3/08, Nicolas Pitre [off-list ref] wrote:

On Mon, 3 Nov 2008, Thanassis Tsiodras wrote:

quoted

Despair...

I just tested "git push --thin"...
Doesn't work.

It still sends the complete object, not a tiny pack as it could (should).

But perhaps I now understand why:

I run git-gc on both the remote end and the working end (before
changing anything,
i.e. with both repos being in sync - "git pull" and "git push" report all
OK).
I then noticed that on the remote side, .git/objects/pack had one big pack
file,
but on the local one I have two .pack files...!

I proceeded to try (many combinations of params on) git-repack in a vain
attempt
to make my local repos also have one single .pack file (presumably, it
should be able
to exactly mirror the remote one, since it has the same objects inside
it!). No way...

Please stop thinking that your repository layout has anything to do with
what is actually transferred on a push.  It has not.

Here's a small test that you can do locally:

	mkdir repo_a
	mkdir repo_b
	cd repo_a
	git init
	seq 1000000 > data
	git add data
	git commit -m "initial commit"
	cd ../repo_b
	git init
	cd ../repo_a
	git push ../repo_b master:master

Here you should see a line that says:

	Writing objects: 100% (3/3), 2.01 MiB, done.

Therefore 2.1 MiB were transferred.  Now let's continue:

	echo "foo" >> data
	git add data
	git commit -m "second commit"
	git push ../repo_b master:master

You should get:

	Writing objects: 100% (3/3), 423 bytes, done.

And this means that you even don't need the --thin switch (which is
wrong -- this has been broken before but that's another story) for the
transfer to actually send only the difference and not the whole file
again.  And note that none of those repositoryes actually contain any
pack as everything is still loose objects.

quoted

I'm at a loss as to why the two repos are having different "pack
representation" of the same objects

That's only because those objects entered each repositories in a
different way.

quoted

and why git-gc and git-repack fail
to create a single pack on my working side,

Maybe you have a .keep file in .git/objects/pack/ ?  If so delete it and
run 'git repack -a -d'.

quoted

but I'm guessing that this is why "git push --thin" fails to send
small xdeltas...

Not at all.

Please provide a complete log of your tests and maybe we could find
something.


Nicolas

-- 
What I gave, I have; what I spent, I had; what I kept, I lost. -Old Epitaph