Re: [PATCHv4] Add Gitweb support for XZ compressed snapshots

Mark A Rada [off-list ref] writes:

Ok, so I got a good nights sleep now, and reviewed the results of my
benchmarks to make sure they were consistent (turns out I had the
archive sizes in the wrong order for the XZ repository tests).

I also reworded a number of things and added a conclusion to the
benchmarks.

Let me know what you think.

Well separated change.  Very detailed commit message; that's good!.

[...]

Linux 2.6 series (f5886c7f96f2542382d3a983c5f13e03d7fc5259) 	349M
gzip 	23.70s user    	0.47s system  	99% cpu  24.227 total 	76M
gunzip 	3.74s user     	0.74s system  	94% cpu  4.741 total
bzip2 	130.96s user   	0.53s system  	99% cpu  2:11.97 total 	59M
bunzip2 31.05s user    	1.02s system  	99% cpu  32.355 total
xz 	448.78s user 	0.91s system  	99% cpu  7:31.28 total 	51M
unxz 	7.67s user     	0.80s system  	98% cpu  8.607 total

Git (0a53e9ddeaddad63ad106860237bbf53411d11a7) 			11M
gzip 	0.77s user 	0.03s system 	99% cpu  0.792 total 	2.5M
gunzip 	0.12s user 	0.02s system 	98% cpu  0.142 total
bzip2 	3.42s user 	0.02s system 	99% cpu  3.454 total 	2.1M
bunzip2 0.95s user 	0.03s system 	99% cpu  0.984 total
xz 	12.88s user 	0.14s system 	98% cpu  13.239 total 	1.9M
unxz 	0.27s user 	0.03s system 	99% cpu  0.298 total

XZ (669413bb2db954bbfde3c4542fddbbab53891eb4) 			1.8M
gzip  	0.12s user 	0.00s system 	95% cpu  0.132 total 	442K
gunzip 	0.02s user 	0.00s system 	97% cpu  0.027 total
bzip2  	1.28s user 	0.01s system 	99% cpu  1.298 total 	363K
bunzip2 0.15s user 	0.01s system 	100% cpu 0.157 total
xz 	1.62s user 	0.03s system 	99% cpu  1.652 total 	347K
unxz   	0.05s user 	0.00s system 	99% cpu  0.058 total

Note that for me the above results are not aligned in table.
This is a cosmetic issue.

Purely from a time and memory perspective, nothing compares to GZip in
each of the three tests. Though, if you have an average upload speed of
20KB/s, it would take ~400 seconds longer to transfer the kernel
snapshot
that was BZip2 compressed than it would the XZ compressed snapshot, the
transfer time difference is even greater when compared to the GZip
compressed snapshot. The wall clock time savings are relatively the same
for all test cases, but less dramatic for the smaller repositories.

The obvious downside for XZ compressed snapshots is the large CPU and
memory load put on the server to actualy generate the snapshot. Though
XZ
will eventually have good threading support, and I suspect then that the
wall clock time for making an XZ compressed snapshot would go down
considerably if the server had a beefy multi-core CPU.

I have not enabled XZ compression by default because the current default
is GZip, and XZ is only really competitive with BZip2. Also, the XZ
format
is still fairly new (the format was declared stable about 6 months ago),
and there have been no "stable" releases of the utils yet.

Those above three paragraphs are strangely wrapped, with single word
for a whole line ('snapshot', 'XZ', 'format').  This is a cosmetic issue.

Signed-off-by: Mark Rada <redacted>

-- 
Jakub Narebski
Git User's Survey 2009: http://tinyurl.com/GitSurvey2009