Thread (18 messages) flat view 18 messages, 5 authors, 2016-06-15

Re: kernel.org and GIT tree rebuilding

From: Jeff Garzik <hidden>
Date: 2016-06-15 22:42:00

David S. Miller wrote:
To get a clean history to push to Linus, I typically blow
away my trees and make fresh ones to stick patches into
which I want to merge.

That mostly works fine here on my local systems, but I know this
brings the master.org mirroring system to it's knees.  So what is the
generally condoned way to do stuff like this in a more friendly way?

Should I:

1) Do a git pull from Linus's tree once he takes my changes, then
   ask GIT to prune the tree?  How do I do that and how does it work?
It takes sometimes an hour for my tree updates on master.kernel.org
to propagate to rsync.kernel.org so I can ask Linus to pull.
That's crazy.
Unfortunately you cannot fix this by changing your actions.  This is the 
cumulative effect of all the git kernel trees on kernel.org.  It now 
takes over an hour for my non-git changes to propagate from master to 
the mirrors, as well.

This is all due to the rsync sweeps, which have to scan metric tons of 
inodes and dentries.  Orders of magnitude over the pre-git days.

ftpadmin@kernel.org folks are supposedly working on an inotify-based 
system, and an improved rsync application.  No ETA or details.

As an aside, cold-cache, git really punishes my disks.  Ted T'so noted 
that it really drains laptop batteries, too.

2) Should I use .git/object/ database symlinking?

   Are there any scripts out there which do this automatically?
   Something as simple to run as "git-pull-script" and it takes
   care of using links when possible on a local filesystem.
On both kernel.org and locally, I use 'cp -al' to duplicate the initial 
.git/objects directory, and then rsync (->kernel.org) or git-pull-script 
(<-kernel.org) to update it after that.

That definitely helps.

Maybe somebody needs to script a relink cron job for kernel.org?

	Jeff

Keyboard shortcuts
hback out one level
jnext message in thread
kprevious message in thread
ldrill in
Escclose help / fold thread tree
?toggle this help