git log -z doesn't separate commits with NULs

Hello all.

I wanted to generate several files with some statistics using "git log 
-z" command.
I did something like this:
git log -z --patch HEAD~10..HEAD -- SomePathHere | xargs -0 
--max-chars=1000000 ~/1.sh

If I put
echo "started"
into the file  ~/1.sh I see that the file is called only once instead of 
multiple times.

I'm newbie to xargs, thus I tested with and that worked as I expected.
find . -type f -print0 | xargs -0  ./1.sh
That produced a lost of "started" lines.

Thus I suspect there is a but in git log -z command and that doesn't 
"Separate the commits with NULs instead of with new newlines." as 
promised in the documents.
Is my understanding correct or I don't understand the documentation or 
somehow pass wrong parameters into git log?

Thank you.
Best regards,
Nikolaj

On 23/02/12 09:14, Nikolaj Shurkaev wrote:

Hello all.

I wanted to generate several files with some statistics using "git log
-z" command.
I did something like this:
git log -z --patch HEAD~10..HEAD -- SomePathHere | xargs -0
--max-chars=1000000 ~/1.sh

If I put
echo "started"
into the file ~/1.sh I see that the file is called only once instead of
multiple times.

I'm newbie to xargs, thus I tested with and that worked as I expected.
find . -type f -print0 | xargs -0 ./1.sh
That produced a lost of "started" lines.

Thus I suspect there is a but in git log -z command and that doesn't
"Separate the commits with NULs instead of with new newlines." as
promised in the documents.
Is my understanding correct or I don't understand the documentation or
somehow pass wrong parameters into git log?

Just a guess, but doesn't the "--patch" option to git log ask it to 
produce a patch output? Surely that will override the -z: patch will not 
be expecting NULs.

Thank you.
Best regards,
Nikolaj
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

Am 23.02.2012 10:14, schrieb Nikolaj Shurkaev:

I wanted to generate several files with some statistics using "git log
-z" command.
I did something like this:
git log -z --patch HEAD~10..HEAD -- SomePathHere | xargs -0
--max-chars=1000000 ~/1.sh

If I put
echo "started"
into the file  ~/1.sh I see that the file is called only once instead of
multiple times.

That is because xargs calls the program with as many arguments as
possible, unless directed otherwise. Put this in the script:

	echo "started $*"

and repeat. Then try this:

 ... | xargs -0 -n 1 ~/1.sh

-- Hannes

On Thu, Feb 23, 2012 at 12:14:23PM +0300, Nikolaj Shurkaev wrote:

I wanted to generate several files with some statistics using "git
log -z" command.
I did something like this:
git log -z --patch HEAD~10..HEAD -- SomePathHere | xargs -0
--max-chars=1000000 ~/1.sh

I'm not sure what "1.sh" is expecting to take as input, but that will
feed entire commits, including their commit message and entire diff, to
the script on its command line.

That seems like an awkward interface, but we don't really know what your
script intends to do. Maybe it is worth sharing the contents of the
script.

If I put echo "started" into the file  ~/1.sh I see that the file is
called only once instead of multiple times.

Yes. The point of xargs is usually to cram as many arguments into each
invocation of "1.sh" as possible, splitting into multiple invocations
only when we hit the argument-list memory limit that the OS imposes.

If you want xargs to give each argument its own invocation of the
script, use "xargs -n1".

I'm newbie to xargs, thus I tested with and that worked as I expected.
find . -type f -print0 | xargs -0  ./1.sh
That produced a lost of "started" lines.

If you instrument your 1.sh more[1], you will find that is not executing
once per file, but rather getting a large chunk of files per invocation.

[1] Try adding: echo "got args: $*"

Thus I suspect there is a but in git log -z command and that doesn't
"Separate the commits with NULs instead of with new newlines." as
promised in the documents.

You could verify that assertion by looking at the output. Try piping
your "git log" command through "cat -A | less". When I try it, I see a
NUL between each commit (cat -A will show it as "^@").

-Peff

On Thu, Feb 23, 2012 at 10:02:31AM +0000, Luke Diamand wrote:

quoted

Thus I suspect there is a but in git log -z command and that doesn't
"Separate the commits with NULs instead of with new newlines." as
promised in the documents.
Is my understanding correct or I don't understand the documentation or
somehow pass wrong parameters into git log?

Just a guess, but doesn't the "--patch" option to git log ask it to
produce a patch output? Surely that will override the -z: patch will
not be expecting NULs.

No. You will get the patch text and the log message together, with
commits separated by NUL. Some diff output formats will also respect
"-z" to produce NULs internally (e.g., "--raw" will use it to separate
filenames), but "--patch" does not.

-Peff

Hello.

You were right. I added parameter --max-args=1 to xargs command and that 
started work as I wanted initially.
Thank you very much.

--
Nikolaj

23.02.2012 13:17, Johannes Sixt пишет:

Am 23.02.2012 10:14, schrieb Nikolaj Shurkaev:

quoted

I wanted to generate several files with some statistics using "git log
-z" command.
I did something like this:
git log -z --patch HEAD~10..HEAD -- SomePathHere | xargs -0
--max-chars=1000000 ~/1.sh

If I put
echo "started"
into the file  ~/1.sh I see that the file is called only once instead of
multiple times.

That is because xargs calls the program with as many arguments as
possible, unless directed otherwise. Put this in the script:

	echo "started $*"

and repeat. Then try this:

  ... | xargs -0 -n 1 ~/1.sh

-- Hannes

Hello.

Thank you very much for your tips. They really helped me. I was trying 
to create patches that would affect only some given files or folders. By 
this moment I have the following:

GeneratePatches.sh
---------------------
#!/bin/bash
#parameter 1 - <since>..<to>
#parameter 2 - path to file
git log -z --reverse --format=email --patch "$1" -- "$2" | xargs --null 
--max-args=1 ./CreatePatchFile.sh
---------------------

and CreatePatchFile.sh
---------------------
#!/bin/bash

myPatchNumber=$(ls ./*-patch.patch 2>/dev/null | wc -l)
let "myPatchNumber += 1"

patchFile="./"$(printf "%04d" $myPatchNumber)"-patch.patch"
echo "$@" > "$patchFile"
---------------------

I call
./GeneratePatches.sh HEAD~3..HEAD SomePath
and that produces something very similar to what I want.

Perhaps there is a better way to do that.

Thank you once again.
---
Best regards,
Nikolaj

23.02.2012 13:24, Jeff King пишет:

On Thu, Feb 23, 2012 at 12:14:23PM +0300, Nikolaj Shurkaev wrote:

quoted

I wanted to generate several files with some statistics using "git
log -z" command.
I did something like this:
git log -z --patch HEAD~10..HEAD -- SomePathHere | xargs -0
--max-chars=1000000 ~/1.sh

I'm not sure what "1.sh" is expecting to take as input, but that will
feed entire commits, including their commit message and entire diff, to
the script on its command line.

That seems like an awkward interface, but we don't really know what your
script intends to do. Maybe it is worth sharing the contents of the
script.

quoted

If I put echo "started" into the file  ~/1.sh I see that the file is
called only once instead of multiple times.

Yes. The point of xargs is usually to cram as many arguments into each
invocation of "1.sh" as possible, splitting into multiple invocations
only when we hit the argument-list memory limit that the OS imposes.

If you want xargs to give each argument its own invocation of the
script, use "xargs -n1".

quoted

I'm newbie to xargs, thus I tested with and that worked as I expected.
find . -type f -print0 | xargs -0  ./1.sh
That produced a lost of "started" lines.

If you instrument your 1.sh more[1], you will find that is not executing
once per file, but rather getting a large chunk of files per invocation.

[1] Try adding: echo "got args: $*"

quoted

Thus I suspect there is a but in git log -z command and that doesn't
"Separate the commits with NULs instead of with new newlines." as
promised in the documents.

You could verify that assertion by looking at the output. Try piping
your "git log" command through "cat -A | less". When I try it, I see a
NUL between each commit (cat -A will show it as "^@").

-Peff

Nikolaj Shurkaev [off-list ref] writes:

Thank you very much for your tips. They really helped me. I was trying
to create patches that would affect only some given files or
folders. By this moment I have the following:

GeneratePatches.sh
---------------------
#!/bin/bash
#parameter 1 - <since>..<to>
#parameter 2 - path to file
git log -z --reverse --format=email --patch "$1" -- "$2" | xargs
--null --max-args=1 ./CreatePatchFile.sh
---------------------

and CreatePatchFile.sh
---------------------
#!/bin/bash

myPatchNumber=$(ls ./*-patch.patch 2>/dev/null | wc -l)
let "myPatchNumber += 1"

patchFile="./"$(printf "%04d" $myPatchNumber)"-patch.patch"
echo "$@" > "$patchFile"
---------------------

I call
./GeneratePatches.sh HEAD~3..HEAD SomePath
and that produces something very similar to what I want.

Perhaps there is a better way to do that.

So what git-format-patch is lacking?

-- 
Jakub Narebski

For example there are commits that affect not only files in folder A but 
files in folder B, C and so on.
If I do format-patch that will give me nice patches, but there are 
modifications of folders B, C and so on there.
I do not know a way to generate patches via format-patch that affect 
only files in folder A.

This is why I wrote those scripts.

23.02.2012 16:15, Jakub Narebski пишет:

Nikolaj Shurkaev[off-list ref]  writes:

quoted

Thank you very much for your tips. They really helped me. I was trying
to create patches that would affect only some given files or
folders. By this moment I have the following:

GeneratePatches.sh
---------------------
#!/bin/bash
#parameter 1 -<since>..<to>
#parameter 2 - path to file
git log -z --reverse --format=email --patch "$1" -- "$2" | xargs
--null --max-args=1 ./CreatePatchFile.sh
---------------------

and CreatePatchFile.sh
---------------------
#!/bin/bash

myPatchNumber=$(ls ./*-patch.patch 2>/dev/null | wc -l)
let "myPatchNumber += 1"

patchFile="./"$(printf "%04d" $myPatchNumber)"-patch.patch"
echo "$@">  "$patchFile"
---------------------

I call
./GeneratePatches.sh HEAD~3..HEAD SomePath
and that produces something very similar to what I want.

Perhaps there is a better way to do that.

So what git-format-patch is lacking?

On Thu, Feb 23, 2012 at 04:48:43PM +0300, Nikolaj Shurkaev wrote:

For example there are commits that affect not only files in folder A
but files in folder B, C and so on.  If I do format-patch that will
give me nice patches, but there are modifications of folders B, C and
so on there.  I do not know a way to generate patches via format-patch
that affect only files in folder A.

Doesn't:

  git format-patch HEAD~3..HEAD SomePath

do what you want? It is certainly designed to, and it seems to work for
me.

-Peff