From: Russell King - ARM Linux <hidden> Date: 2016-06-15 22:50:25
On Sun, Jan 16, 2011 at 09:10:17PM +0800, Jello huang wrote:
yes,git doesn't handle that case and i rename the pack name,but there is
also the similar error.Now i just delet the git tree and clone it again
tonight .
_Always_ without fail fetch Linus' tree before pulling my tree.
My tree is a rsync clone of the objects and pack files in Linus' tree,
plus whatever git decided to build on top of that - for local commits
that's individual object files. For remote pulls, that's probably a few
small pack files.
There is *no* repacking of my tree. So the only times it gets 'repacked'
is when Linus repacks his tree.
Let's say you already have a copy of my tree from a month ago, and Linus
has pulled some work from me into his tree, and repacked his tree into one
single pack file. At the moment, the largest pack file from Linus is
400MB plus a 50MB index.
You already have most of the contents of that 400MB pack file, but if
you're missing even _one_ object which is contained within it, git will
have to download the _entire_ 400MB pack file and index file to retrieve
it.
However, if you first fetch Linus' tree via the git protocol, it can just
request the objects it doesn't have from the git server. That will mean
you'll have all the objects in the large pack files before you start trying
to pull my tree, and git won't have to download 400MB for the sake of
retrieving just maybe 10k that you didn't have.
This isn't something special with my tree - it's a side effect of the
http protocol git uses. So, before you fetch _any_ http-based git tree,
first make sure you're up to date with Linus'.
(I update my tree from Linus' in rsync mode to make http-based stuff a
lot more friendly to people using it - some of whom are stuck behind
firewalls which can only do http. Fetching a constantly repacked git
tree via http results in hundreds of megabytes needing to be fetched
every time.)
So please, whenever possible, always fetch Linus' latest tree _first_
and then mine. Same goes for any other http based tree which doesn't
auto-repack.
On 01/16/11 14:42, Russell King - ARM Linux wrote:
Let's say you already have a copy of my tree from a month ago, and Linus
has pulled some work from me into his tree, and repacked his tree into one
single pack file. At the moment, the largest pack file from Linus is
400MB plus a 50MB index.
You already have most of the contents of that 400MB pack file, but if
you're missing even _one_ object which is contained within it, git will
have to download the _entire_ 400MB pack file and index file to retrieve
it.
I thought this has changed with "smart http" in git 1.6.6.
Am I missing something?
Detlef
Hi Detlef,
On Fri, Jan 21, 2011 at 02:38:11PM +0100, Detlef Vollmann wrote:
On 01/16/11 14:42, Russell King - ARM Linux wrote:
quoted
Let's say you already have a copy of my tree from a month ago, and Linus
has pulled some work from me into his tree, and repacked his tree into one
single pack file. At the moment, the largest pack file from Linus is
400MB plus a 50MB index.
You already have most of the contents of that 400MB pack file, but if
you're missing even _one_ object which is contained within it, git will
have to download the _entire_ 400MB pack file and index file to retrieve
it.
I thought this has changed with "smart http" in git 1.6.6.
Am I missing something?
Well, not all http repos offer smart http. E.g. Russell doesn't[1],
probably because the serving machine doesn't have the power to nice
serve a repo via git:// or smart http://.
Best regards
Uwe
[1] I didn't recheck though
--
Pengutronix e.K. | Uwe Kleine-K?nig |
Industrial Linux Solutions | http://www.pengutronix.de/ |
From: Russell King - ARM Linux <hidden> Date: 2016-06-15 22:50:26
On Fri, Jan 21, 2011 at 02:47:28PM +0100, Uwe Kleine-K?nig wrote:
Hi Detlef,
On Fri, Jan 21, 2011 at 02:38:11PM +0100, Detlef Vollmann wrote:
quoted
On 01/16/11 14:42, Russell King - ARM Linux wrote:
quoted
Let's say you already have a copy of my tree from a month ago, and Linus
has pulled some work from me into his tree, and repacked his tree into one
single pack file. At the moment, the largest pack file from Linus is
400MB plus a 50MB index.
You already have most of the contents of that 400MB pack file, but if
you're missing even _one_ object which is contained within it, git will
have to download the _entire_ 400MB pack file and index file to retrieve
it.
I thought this has changed with "smart http" in git 1.6.6.
Am I missing something?
Well, not all http repos offer smart http. E.g. Russell doesn't[1],
probably because the serving machine doesn't have the power to nice
serve a repo via git:// or smart http://.
What is smart http? I don't particularly follow git developments.
On 01/21/11 14:57, Russell King - ARM Linux wrote:
On Fri, Jan 21, 2011 at 02:47:28PM +0100, Uwe Kleine-K?nig wrote:
quoted
Hi Detlef,
On Fri, Jan 21, 2011 at 02:38:11PM +0100, Detlef Vollmann wrote:
quoted
On 01/16/11 14:42, Russell King - ARM Linux wrote:
quoted
Let's say you already have a copy of my tree from a month ago, and Linus
has pulled some work from me into his tree, and repacked his tree into one
single pack file. At the moment, the largest pack file from Linus is
400MB plus a 50MB index.
You already have most of the contents of that 400MB pack file, but if
you're missing even _one_ object which is contained within it, git will
have to download the _entire_ 400MB pack file and index file to retrieve
it.
I thought this has changed with "smart http" in git 1.6.6.
Am I missing something?
Well, not all http repos offer smart http. E.g. Russell doesn't[1],
probably because the serving machine doesn't have the power to nice
serve a repo via git:// or smart http://.
What is smart http? I don't particularly follow git developments.
Setting up Smart HTTP
...
To set it up, it?s best to walk through the instructions on the
`git-http-backend` documentation page. Basically, you have to install Git
v1.6.6 or higher on a server with an Apache 2.x webserver (it has to be
Apache, currently - other CGI servers don?t work, last I checked). Then
you add something similar to this to your http.conf file:
SetEnv GIT_PROJECT_ROOT /var/www/git
SetEnv GIT_HTTP_EXPORT_ALL
ScriptAlias /git/ /usr/libexec/git-core/git-http-backend/
Great. Deciding that it will be http://servername.example.com/git/ is
really damned annoying as that's traditionally where gitweb lives,
which requires a different script alias.
It seems that due to a lack of coordination between different git
developers, people running webservers have a choice between providing
gitweb or this http extension.
I'm really not interested in working out how to bodge this into working
along side the existing gitweb setup by adding lots of rewrite rules, so
as gitweb got there first I think it has priority, that's what we have
and we'll have to live without the smart http extensions.
It's really not that big a deal if you follow the advice I've given.
Setting up Smart HTTP
...
To set it up, it?s best to walk through the instructions on the
`git-http-backend` documentation page. Basically, you have to install Git
v1.6.6 or higher on a server with an Apache 2.x webserver (it has to be
Apache, currently - other CGI servers don?t work, last I checked). Then
you add something similar to this to your http.conf file:
SetEnv GIT_PROJECT_ROOT /var/www/git
SetEnv GIT_HTTP_EXPORT_ALL
ScriptAlias /git/ /usr/libexec/git-core/git-http-backend/
Great. Deciding that it will be http://servername.example.com/git/ is
really damned annoying as that's traditionally where gitweb lives,
which requires a different script alias.
It seems that due to a lack of coordination between different git
developers, people running webservers have a choice between providing
gitweb or this http extension.
Huh?
/git/ is just the example here, you can use any name you want.
E.g. I use /auth/ for authenticated users, and others use /gitmob/
or /gitanon/ for non-authenticated users.
And you can use something like gitsmart, githttp, or whatever...
Detlef
Setting up Smart HTTP
...
To set it up, it?s best to walk through the instructions on the
`git-http-backend` documentation page. Basically, you have to install Git
v1.6.6 or higher on a server with an Apache 2.x webserver (it has to be
Apache, currently - other CGI servers don?t work, last I checked). Then
you add something similar to this to your http.conf file:
SetEnv GIT_PROJECT_ROOT /var/www/git
SetEnv GIT_HTTP_EXPORT_ALL
ScriptAlias /git/ /usr/libexec/git-core/git-http-backend/
Great. Deciding that it will be http://servername.example.com/git/ is
really damned annoying as that's traditionally where gitweb lives,
which requires a different script alias.
It seems that due to a lack of coordination between different git
developers, people running webservers have a choice between providing
gitweb or this http extension.
I'm really not interested in working out how to bodge this into working
along side the existing gitweb setup by adding lots of rewrite rules, so
as gitweb got there first I think it has priority, that's what we have
and we'll have to live without the smart http extensions.
IIRC it's designed to live along side the http:// clone url.
git-http-backend can still serve dumb http clients including a web
browser.
But note that as git-http-backend less info it has to calculate much
more. So the load it introduces should be comparable to running
git-daemon as should be the times to fetch from it. So AFAIK the only
reason to run it is that more corporate users can access port 80.
Best regards
Uwe
--
Pengutronix e.K. | Uwe Kleine-K?nig |
Industrial Linux Solutions | http://www.pengutronix.de/ |
Russell,thanks for your reply so elaborate and i have gotten the idea now.
On 16 January 2011 21:42, Russell King - ARM Linux
[off-list ref]wrote:
On Sun, Jan 16, 2011 at 09:10:17PM +0800, Jello huang wrote:
quoted
yes,git doesn't handle that case and i rename the pack name,but there is
also the similar error.Now i just delet the git tree and clone it again
tonight .
_Always_ without fail fetch Linus' tree before pulling my tree.
My tree is a rsync clone of the objects and pack files in Linus' tree,
plus whatever git decided to build on top of that - for local commits
that's individual object files. For remote pulls, that's probably a few
small pack files.
There is *no* repacking of my tree. So the only times it gets 'repacked'
is when Linus repacks his tree.
Let's say you already have a copy of my tree from a month ago, and Linus
has pulled some work from me into his tree, and repacked his tree into one
single pack file. At the moment, the largest pack file from Linus is
400MB plus a 50MB index.
You already have most of the contents of that 400MB pack file, but if
you're missing even _one_ object which is contained within it, git will
have to download the _entire_ 400MB pack file and index file to retrieve
it.
However, if you first fetch Linus' tree via the git protocol, it can just
request the objects it doesn't have from the git server. That will mean
you'll have all the objects in the large pack files before you start trying
to pull my tree, and git won't have to download 400MB for the sake of
retrieving just maybe 10k that you didn't have.
This isn't something special with my tree - it's a side effect of the
http protocol git uses. So, before you fetch _any_ http-based git tree,
first make sure you're up to date with Linus'.
(I update my tree from Linus' in rsync mode to make http-based stuff a
lot more friendly to people using it - some of whom are stuck behind
firewalls which can only do http. Fetching a constantly repacked git
tree via http results in hundreds of megabytes needing to be fetched
every time.)
So please, whenever possible, always fetch Linus' latest tree _first_
and then mine. Same goes for any other http based tree which doesn't
auto-repack.
not a lucky dog. i used 1.6.0,there was a large pack.
On 21 January 2011 22:28, Detlef Vollmann [off-list ref] wrote:
On 01/21/11 14:57, Russell King - ARM Linux wrote:
quoted
On Fri, Jan 21, 2011 at 02:47:28PM +0100, Uwe Kleine-K?nig wrote:
quoted
Hi Detlef,
On Fri, Jan 21, 2011 at 02:38:11PM +0100, Detlef Vollmann wrote:
quoted
On 01/16/11 14:42, Russell King - ARM Linux wrote:
quoted
Let's say you already have a copy of my tree from a month ago, and
Linus
has pulled some work from me into his tree, and repacked his tree into
one
single pack file. At the moment, the largest pack file from Linus is
400MB plus a 50MB index.
You already have most of the contents of that 400MB pack file, but if
you're missing even _one_ object which is contained within it, git will
have to download the _entire_ 400MB pack file and index file to
retrieve
it.
I thought this has changed with "smart http" in git 1.6.6.
Am I missing something?
Well, not all http repos offer smart http. E.g. Russell doesn't[1],
probably because the serving machine doesn't have the power to nice
serve a repo via git:// or smart http://.
What is smart http? I don't particularly follow git developments.
From: Johannes Sixt <hidden> Date: 2016-06-15 22:50:27
Am 1/21/2011 15:50, schrieb Russell King - ARM Linux:
SetEnv GIT_PROJECT_ROOT /var/www/git
SetEnv GIT_HTTP_EXPORT_ALL
ScriptAlias /git/ /usr/libexec/git-core/git-http-backend/
Great. Deciding that it will be http://servername.example.com/git/ is
really damned annoying as that's traditionally where gitweb lives,
which requires a different script alias.
...
I'm really not interested in working out how to bodge this into working
along side the existing gitweb setup by adding lots of rewrite rules,...
Am 1/21/2011 15:50, schrieb Russell King - ARM Linux:
quoted
SetEnv GIT_PROJECT_ROOT /var/www/git
SetEnv GIT_HTTP_EXPORT_ALL
ScriptAlias /git/ /usr/libexec/git-core/git-http-backend/
Great. Deciding that it will be http://servername.example.com/git/ is
really damned annoying as that's traditionally where gitweb lives,
which requires a different script alias.
...
I'm really not interested in working out how to bodge this into working
along side the existing gitweb setup by adding lots of rewrite rules,...
I was going to say, I have both gitweb and smart http support running in
parallel, it's not terribly complicated. The way I'm doing it is a bit
older (mainly that long ScriptAliasMatch broken out more), but it's
still only 11 lines to the apache config:
# Various environment setup, should be obvious
SetEnv GIT_PROJECT_ROOT /path
SetEnv GIT_HTTP_EXPORT_ALL 1
SetEnv GIT_EXEC_PATH /usr/libexec/git-core/
# Here so that we can do straight wget fetches for the objects files,
# pack files and refs heads.
# This isn't strictly neccisary, but if it's done we can use the inbuilt
# sendfile() of apache for the sending of those files vs. funneling them
# through the CGI
AliasMatch ^/(.*/objects/[0-9a-f]{2}/[0-9a-f]{38})$ /path/$1
AliasMatch ^/(.*/objects/pack/pack-[0-9a-f]{40}.(pack|idx))$ /path/$1
AliasMatch ^/(.*/refs/heads/.*)$ /path/$1
# Meat of how to make this all work
ScriptAliasMatch "^/(.*/HEAD)$"
/usr/libexec/git-core/git-http-backend/path/$1
ScriptAliasMatch "^/(.*/info/refs)$"
/usr/libexec/git-core/git-http-backend/path/$1
ScriptAliasMatch "^/(.*/objects/info/[^/]+)$"
/usr/libexec/git-core/git-http-backend/path/$1
ScriptAliasMatch "^/pub/scm/(.*/git-upload-pack)$"
/usr/libexec/git-core/git-http-backend/path/$1
ScriptAliasMatch "^/pub/scm/(.*/git-receive-pack)$"
/usr/libexec/git-core/git-http-backend/path/$1
Just an example anyway, the paths have been shortened to fit e-mail better.
- John 'Warthog9' Hawley
From: Russell King - ARM Linux <hidden> Date: 2016-06-15 22:50:27
On Mon, Jan 24, 2011 at 08:21:09AM +0100, Johannes Sixt wrote:
Am 1/21/2011 15:50, schrieb Russell King - ARM Linux:
quoted
SetEnv GIT_PROJECT_ROOT /var/www/git
SetEnv GIT_HTTP_EXPORT_ALL
ScriptAlias /git/ /usr/libexec/git-core/git-http-backend/
Great. Deciding that it will be http://servername.example.com/git/ is
really damned annoying as that's traditionally where gitweb lives,
which requires a different script alias.
...
I'm really not interested in working out how to bodge this into working
along side the existing gitweb setup by adding lots of rewrite rules,...
Thanks, added taking two server restarts (one of them took a few minutes
for apache to restart as it was soo busy...)
I assume this also means that I need to update the URLs people are using
(as the tree wasn't exported under http://hostname.example.com/git/)?