[Q] Encrypted GIT?

Hi, list!

I want to create a private GIT repo (without working copy) on a
machine in external data-center. While I do not actually believe that
it is possible that someone who has physical access to a machine would
be interested in peeking into my repo, I'd like to play safe and to
have this issue covered.

Please advise what is the best way to do it. Are there any existing solutions?

Thanks,
Alexander.

On Thu, Mar 13, 2008 at 11:48:53AM +0300, Alexander Gladysh [off-list ref] wrote:

I want to create a private GIT repo (without working copy) on a
machine in external data-center. While I do not actually believe that
it is possible that someone who has physical access to a machine would
be interested in peeking into my repo, I'd like to play safe and to
have this issue covered.

Please advise what is the best way to do it. Are there any existing solutions?

i don't think but you can write a wrapper around git receive/upload-pack
and use (for example) tar+gpg to keep your repo encrypted on the disc.

Hi,

On Thu, 13 Mar 2008, Miklos Vajna wrote:

On Thu, Mar 13, 2008 at 11:48:53AM +0300, Alexander Gladysh [off-list ref] wrote:

quoted

I want to create a private GIT repo (without working copy) on a 
machine in external data-center. While I do not actually believe that 
it is possible that someone who has physical access to a machine would 
be interested in peeking into my repo, I'd like to play safe and to 
have this issue covered.

Please advise what is the best way to do it. Are there any existing 
solutions?

i don't think but you can write a wrapper around git receive/upload-pack 
and use (for example) tar+gpg to keep your repo encrypted on the disc.

The problem is: you cannot decrypt on the remote side, otherwise you will 
lose all the security.

But if you do not decrypt on the remote side, you cannot store deltified 
objects (you lose all the benefits of Git's efficient storage), neither 
can you update incrementally (you lose all the benefits of Git's efficient 
transport).

The latter can be remedied (somewhat) by encrypting each object 
individually.  In that case, .gitattributes can help (you should be able 
to find a mail to that extent, which I sent no more than 2 weeks ago).  
However, you must make sure that the encryption is repeatable, i.e. two 
different encryption runs _must_ result in _identical_ output.

If it is only a single file containing all your secrets, it can also make 
sense to just encrypt it, and track the _encrypted_ file directly 
(without clean/smudge filters).

Hth,
Dscho

On Thu, Mar 13, 2008 at 12:55:11PM +0100, Johannes Schindelin [off-list ref] wrote:

The latter can be remedied (somewhat) by encrypting each object 
individually.  In that case, .gitattributes can help (you should be able 
to find a mail to that extent, which I sent no more than 2 weeks ago).  
However, you must make sure that the encryption is repeatable, i.e. two 
different encryption runs _must_ result in _identical_ output.

afaik, this is not the case for gpg.

On Thu, Mar 13, 2008 at 01:16:44PM +0100, Miklos Vajna wrote:

On Thu, Mar 13, 2008 at 12:55:11PM +0100, Johannes Schindelin [off-list ref] wrote:

quoted

The latter can be remedied (somewhat) by encrypting each object 
individually.  In that case, .gitattributes can help (you should be able 
to find a mail to that extent, which I sent no more than 2 weeks ago).  
However, you must make sure that the encryption is repeatable, i.e. two 
different encryption runs _must_ result in _identical_ output.

afaik, this is not the case for gpg.

No, and you wouldn't want to use gpg because of the overhead it adds
around an encrypted message.  You would need to use a raw encryption
algorithm, or one with very minimal wrapping.  It's normally at this
point that that you'd need to bring in a security expert to ask a
whole lot of questions about your exact use scenario, do a formal
threat analysis, since there are all sorts of unanswered questions
about what kind of key management solution you really need for your
situation.

It's usually not as simple as "just encrypt it".  How many people need
to have access to the to the repository?  Do you need to revoke access
to the repository later?  Who is allowed to give a new person access
to the repository?  etc., etc., etc.

						- Ted

 It's normally at this point that that you'd need to bring in a security expert to ask a
 whole lot of questions about your exact use scenario, do a formal
 threat analysis, since there are all sorts of unanswered questions
 about what kind of key management solution you really need for your
 situation.

Uh. This is for kind of hobbyist noncommercial usage, so there are not
that much resources for bringing in security experts. :-)

Also I do not expect this data to be protected from determined (payed)
professional attack -- a determined professional would probably be
able to find some weaker spot elsewhere. However I do want such attack
to cost enough to ward off idle amateurs and bored professionals. :-)

 It's usually not as simple as "just encrypt it".

How many people need to have access to the to the repository?

Well, 2-5, up to ten, I guess. In immediate future -- two persons only. :-)

 Do you need to revoke access to the repository later?

Probably. But restricting remote access should be enough.

Who is allowed to give a new person access to the repository?

To keep things simple, me myself only.

etc., etc., etc.

Thank you,
Alexander.

Hi,

On Thu, 13 Mar 2008, Theodore Tso wrote:

On Thu, Mar 13, 2008 at 01:16:44PM +0100, Miklos Vajna wrote:

quoted

On Thu, Mar 13, 2008 at 12:55:11PM +0100, Johannes Schindelin [off-list ref] wrote:

quoted

The latter can be remedied (somewhat) by encrypting each object 
individually.  In that case, .gitattributes can help (you should be 
able to find a mail to that extent, which I sent no more than 2 
weeks ago).  However, you must make sure that the encryption is 
repeatable, i.e. two different encryption runs _must_ result in 
_identical_ output.

afaik, this is not the case for gpg.

No, and you wouldn't want to use gpg because of the overhead it adds
around an encrypted message.

To the contrary: if your files are small (which they are most likely), you 
_want_ the overhead, in order to make the encryption harder to crack.

AFAICT gpg is a good all-round encryption tool, and reinventing the wheel 
just for encrypting things in a git repository just does not cut it.

Ciao,
Dscho

On Thu, Mar 13, 2008 at 08:58:54AM -0400, Theodore Tso wrote:

No, and you wouldn't want to use gpg because of the overhead it adds
around an encrypted message.  You would need to use a raw encryption
algorithm, or one with very minimal wrapping.  It's normally at this

Well, "raw encryption algorithm" is a bit vague here. :)

I thought about this a while ago and come to a few conclusions:

  - encrypting before git sees content sucks, because you are either
    sacrificing security (content X always encrypts to Y) or system
    stability (git doesn't know that Y and Y' are really the same thing)

  - encrypting at the object level (when we do zlib) sucks, because we
    still want to name contents by their hash, which means the object
    database index contains information about what's in your content.
    There's also some per-object overhead. Plus any system without the
    key can't do deltas.

  - encrypting whole packfiles sucks for local storage, since you lose
    the random access property (unless you go with something static like
    an ECB mode, but then you are sacrificing security).

  - encrypting whole packfiles is a bit better for transport. The
    key-holding repo does the deltas and just treats the remote repo as
    dumb storage (it can't be smart, since that would involve looking at
    the data). Storage overhead is minimal if packfiles are a reasonable
    size.

So I think the last makes the most sense, where your local repo is
totally unprotected, but you efficiently push git objects to a remote
untrusted repo.

You could probably do something totally external to git using bundles as
the primitive. Store an encrypted index on the remote that says "here
are the packs I have, and the objects they contain." Whenever you push,
pull the index (which is of course more network-intensive than regular
git protocol, but not as bad as pulling all the data) and calculate a
thin-pack bundle yourself. Encrypt the bundle and store remotely.

point that that you'd need to bring in a security expert to ask a
whole lot of questions about your exact use scenario, do a formal
threat analysis, since there are all sorts of unanswered questions
about what kind of key management solution you really need for your
situation.

I don't know if a formal thread analysis is necessary. I think most
people are interested in "if the contents of remote storage X are
known, how much do people know about the _contents_ of my repo stored on
X?" and they don't care about masking the size, time of updates, etc.

That's a fairly straightforward application of cryptography. The tricky
part is doing it in a way that can still leverage some of git's
efficiencies.

It's usually not as simple as "just encrypt it".  How many people need
to have access to the to the repository?  Do you need to revoke access
to the repository later?  Who is allowed to give a new person access
to the repository?  etc., etc., etc.

Sure, those are all interesting questions for a complete system. But I
think it makes sense to incrementally try the basics first.

-Peff

On Thu, Mar 13, 2008 at 04:21:44PM +0100, Johannes Schindelin wrote:

quoted

No, and you wouldn't want to use gpg because of the overhead it adds
around an encrypted message.

To the contrary: if your files are small (which they are most likely), you 
_want_ the overhead, in order to make the encryption harder to crack.

Not necessarily. Using random IVs, random salts, and random padding does
increase security.  Adding headers to every object that tell which
algorithm and parameters were used are nice for interoperability, but
don't help with security. Doing per-object asymmetric encryptions (gpg
--encrypt without --symmetric) is performance insanity.

AFAICT gpg is a good all-round encryption tool, and reinventing the wheel 
just for encrypting things in a git repository just does not cut it.

Keep in mind that in the example you posted before, you were not using
99% of gpg. You were just asking it to do a symmetric CBC cipher using a
passphrase. So it is overkill for that, but at the same time not
actually very flexible for doing those sorts of low-level things.
OpenSSL provides a much better toolkit for that.

-Peff

On Thu, Mar 13, 2008 at 11:53:22AM -0400, Jeff King wrote:

You could probably do something totally external to git using bundles as
the primitive. Store an encrypted index on the remote that says "here
are the packs I have, and the objects they contain." Whenever you push,
pull the index (which is of course more network-intensive than regular
git protocol, but not as bad as pulling all the data) and calculate a
thin-pack bundle yourself. Encrypt the bundle and store remotely.

Oh, and a scheme like this generalizes well from "there is one key" to
"N asymmetric keyholders".

I don't know if a formal thread analysis is necessary. I think most

That should of course be "threa_t_ analysis". I of course want to
formally analyze this thread. ;)

-Peff

On Thu, 13 Mar 2008 11:48:53 +0300
"Alexander Gladysh" [off-list ref] wrote:

Hi, list!

I want to create a private GIT repo (without working copy) on a
machine in external data-center. While I do not actually believe that
it is possible that someone who has physical access to a machine would
be interested in peeking into my repo, I'd like to play safe and to
have this issue covered.

Please advise what is the best way to do it. Are there any existing
solutions?

Potential solution to store arbitrary data in a safe manner:

mkdir remote_git_raw remote_git
sshfs <data-center>:<path @ datacenter> $PWD/remote_git_raw
encfs $PWD/remote_git_raw $PWD/remote_git

This will lock your data in a remote encfs volume.  (Uses FUSE)

Not quite sure about the implications on performance... but this will
certainly keep your data safe on that remote location.

On Thu, Mar 13, 2008 at 11:53:22AM -0400, Jeff King wrote:

  - encrypting whole packfiles is a bit better for transport. The
    key-holding repo does the deltas and just treats the remote repo as
    dumb storage (it can't be smart, since that would involve looking at
    the data). Storage overhead is minimal if packfiles are a reasonable
    size.

So I think the last makes the most sense, where your local repo is
totally unprotected, but you efficiently push git objects to a remote
untrusted repo.

If the main goal is primarily backup of your repository to an
untrusted remote server, yes, that makes perfect sense.  

If you assume multiple trusted developers would actually be
*operating* on an encrypted repo, the life gets much harder, as you've
pointed out.

  - encrypting before git sees content sucks, because you are either
    sacrificing security (content X always encrypts to Y) or system
    stability (git doesn't know that Y and Y' are really the same thing)

It's not clear that "content X always encrypts to Y" is a fatal flaw,
by the way.  Yes, it leaks a bit of information, but in a source code
management situation, it may not matter.  If you do absolutely care,
tough, it might be that the simplest solution is to store the entire
repository and working tree under cryptofs.  After all, what's the
point of encrypting the local repo if the checked-out working tree is
unprotected for all to see?  :-)

						- Ted

On Thu, Mar 13, 2008 at 12:12:01PM -0400, Theodore Tso wrote:

If the main goal is primarily backup of your repository to an
untrusted remote server, yes, that makes perfect sense.  

If you assume multiple trusted developers would actually be
*operating* on an encrypted repo, the life gets much harder, as you've
pointed out.

Well, it depends on the meaning of "operate". :) I think you could still
use it as a rendezvous point as you would any bare repository.  Pushing
and pulling would have a little larger network overhead, and a lot more
CPU overhead.

But yes, that scheme is horrible for a working repo.

quoted

  - encrypting before git sees content sucks, because you are either
    sacrificing security (content X always encrypts to Y) or system
    stability (git doesn't know that Y and Y' are really the same thing)

It's not clear that "content X always encrypts to Y" is a fatal flaw,
by the way.  Yes, it leaks a bit of information, but in a source code

Agreed (I actually recommended in Dscho's original thread "you can do it
by eliminating the salt, if you accept the consequences...").

So after my saying "no formal threat analysis is necessary" you have
clearly called me on making a bunch of usage assumptions. Oops. :)

management situation, it may not matter.  If you do absolutely care,
tough, it might be that the simplest solution is to store the entire
repository and working tree under cryptofs.  After all, what's the
point of encrypting the local repo if the checked-out working tree is
unprotected for all to see?  :-)

Yes. And it doesn't involve any git-specific code at all. :)

-Peff

On Thu, Mar 13, 2008 at 12:55:11PM +0100, Johannes Schindelin wrote:

The latter can be remedied (somewhat) by encrypting each object 
individually.  In that case, .gitattributes can help (you should be able 
to find a mail to that extent, which I sent no more than 2 weeks ago).  
However, you must make sure that the encryption is repeatable, i.e. two 
different encryption runs _must_ result in _identical_ output.

Any decent file encryption program will never have this characteristic.
It's normally a bad idea from a security perspective.

David

On Mar 13, 2008, at 1:48 AM, Alexander Gladysh wrote:

Hi, list!

I want to create a private GIT repo (without working copy) on a
machine in external data-center. While I do not actually believe that
it is possible that someone who has physical access to a machine would
be interested in peeking into my repo, I'd like to play safe and to
have this issue covered.

Please advise what is the best way to do it. Are there any existing  
solutions?

An obvious and easy solution: use an encrypted partition on the  
remote server and ssh as transport. Last time I checked, git on  
encrypted volumes is plenty fast.

__Luke

On Thu, 13 Mar 2008 11:36:46 -0700
Luke Lu [off-list ref] wrote:

An obvious and easy solution: use an encrypted partition on the  
remote server and ssh as transport. Last time I checked, git on  
encrypted volumes is plenty fast.

If its an encrypted partition on the remote server... then its visible
@ that server.. which I don't think is desired in the situation.

An encrypted partition is fairly useless on a remote server unless the
remote server is expected to be physically removed/powered down...
otherwise anything can get into that data while its alive (pending
permissions, lack-of-holes, etc..)

The encfs solution makes sure that nothing is ever revealed
remote-side... all data is prevented from even going over ssh in its
unencrypted form.

On Mar 13, 2008, at 12:15 PM, Thomas Harning wrote:

On Thu, 13 Mar 2008 11:36:46 -0700
Luke Lu [off-list ref] wrote:

quoted

An obvious and easy solution: use an encrypted partition on the
remote server and ssh as transport. Last time I checked, git on
encrypted volumes is plenty fast.

If its an encrypted partition on the remote server... then its visible
@ that server.. which I don't think is desired in the situation.

An encrypted partition is fairly useless on a remote server unless the
remote server is expected to be physically removed/powered down...
otherwise anything can get into that data while its alive (pending
permissions, lack-of-holes, etc..)

The encfs solution makes sure that nothing is ever revealed
remote-side... all data is prevented from even going over ssh in its
unencrypted form.

Yes encfs over an sshfs is probably the safest. But it is intolerably  
slow if you need any kind of random access of data, which git does  
all the time. You can mount the encrypted partition using a key over  
ssh per git push or pull to minimize exposure while get the  
performance you want.

__Luke