From: Georgios Kontaxis via GitGitGadget <hidden> Date: 2021-03-21 00:02:50
From: Georgios Kontaxis <redacted>
Gitweb extracts content from the Git log and makes it accessible
over HTTP. As a result, e-mail addresses found in commits are
exposed to web crawlers. This may result in unsolicited messages.
This is a feature for redacting e-mail addresses from the generated
HTML content.
This feature does not prevent someone from downloading the
unredacted commit log and extracting information from it.
It aims to hinder the low-effort bulk collection of e-mail
addresses by web crawlers.
Signed-off-by: Georgios Kontaxis <redacted>
---
gitweb: Redacted e-mail addresses feature.
Gitweb extracts content from the Git log and makes it accessible over
HTTP. As a result, e-mail addresses found in commits are exposed to web
crawlers. This may result in unsolicited messages. This is a feature for
redacting e-mail addresses from the generated HTML content.
This feature does not prevent someone from downloading the unredacted
commit log and extracting information from it. It aims to hinder the
low-effort bulk collection of e-mail addresses by web crawlers.
Signed-off-by: Georgios Kontaxis geko1702+commits@99rst.org
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-910%2Fkontaxis%2Fkontaxis%2Femail_privacy-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-910/kontaxis/kontaxis/email_privacy-v1
Pull-Request: https://github.com/gitgitgadget/git/pull/910
Documentation/gitweb.conf.txt | 12 ++++++++++++
gitweb/gitweb.perl | 36 ++++++++++++++++++++++++++++++++---
2 files changed, 45 insertions(+), 3 deletions(-)
@@ -896,6 +896,18 @@ same as of the snippet above: It is an error to specify a ref that does not pass "git check-ref-format" scrutiny. Duplicated values are filtered.+email_privacy::+ Redact e-mail addresses from the generated HTML, etc. content.+ This hides e-mail addresses found in the commit log from web crawlers.+ Enabled by default.+++It is highly recommended to keep this feature enabled unless web crawlers+are hindered in some other way. You can disable this feature as shown below:+++---------------------------------------------------------------------------+$feature{'email_privacy'}{'default'} = [0];+---------------------------------------------------------------------------+ EXAMPLES --------
@@ -569,6 +569,15 @@ sub evaluate_uri {'sub'=>\&feature_extra_branch_refs,'override'=>0,'default'=>[]},++# Redact e-mail addresses.++# To disable system wide have in $GITWEB_CONFIG+# $feature{'email_privacy'}{'default'} = [0];+'email_privacy'=>{+'sub'=>sub{feature_bool('email_privacy',@_)},+'override'=>0,+'default'=>[1]},);subgitweb_get_feature{
@@ -3471,6 +3480,10 @@ sub parse_tag {if($tag{'author'}=~m/^([^<]+) <([^>]*)>/){$tag{'author_name'}=$1;$tag{'author_email'}=$2;+if(gitweb_check_feature('email_privacy')){+$tag{'author_email'}="private";+$tag{'author'}=~s/<([^>]+)>/<private>/;+}}else{$tag{'author_name'}=$tag{'author'};}
@@ -3519,6 +3532,10 @@ sub parse_commit_text {if($co{'author'}=~m/^([^<]+) <([^>]*)>/){$co{'author_name'}=$1;$co{'author_email'}=$2;+if(gitweb_check_feature('email_privacy')){+$co{'author_email'}="private";+$co{'author'}=~s/<([^>]+)>/<private>/;+}}else{$co{'author_name'}=$co{'author'};}
@@ -3529,6 +3546,10 @@ sub parse_commit_text {if($co{'committer'}=~m/^([^<]+) <([^>]*)>/){$co{'committer_name'}=$1;$co{'committer_email'}=$2;+if(gitweb_check_feature('email_privacy')){+$co{'committer_email'}="private";+$co{'committer'}=~s/<([^>]+)>/<private>/;+}}else{$co{'committer_name'}=$co{'committer'};}
On Sun, Mar 21 2021, Georgios Kontaxis via GitGitGadget wrote:
From: Georgios Kontaxis <redacted>
Gitweb extracts content from the Git log and makes it accessible
over HTTP. As a result, e-mail addresses found in commits are
exposed to web crawlers. This may result in unsolicited messages.
This is a feature for redacting e-mail addresses from the generated
HTML content.
This feature does not prevent someone from downloading the
unredacted commit log and extracting information from it.
It aims to hinder the low-effort bulk collection of e-mail
addresses by web crawlers.
So web crawlers that aren't going to obey robots.txt?
I'm not opposed to this feature, but a glance at gitweb's documentation
seems to show that we don't discuss how to set robots.txt up for it at
all.
Perhaps having that in the docs or otherwise in the default setup would
get us most of the win of this feature?
Signed-off-by: Georgios Kontaxis <redacted>
---
Odd:
gitweb: Redacted e-mail addresses feature.
Gitweb extracts content from the Git log and makes it accessible over
HTTP. As a result, e-mail addresses found in commits are exposed to web
crawlers. This may result in unsolicited messages. This is a feature for
redacting e-mail addresses from the generated HTML content.
This feature does not prevent someone from downloading the unredacted
commit log and extracting information from it. It aims to hinder the
low-effort bulk collection of e-mail addresses by web crawlers.
Signed-off-by: Georgios Kontaxis geko1702+commits@99rst.org
To have this duplication of the patch here below "---", some GGG feature
gone awry?
@@ -896,6 +896,18 @@ same as of the snippet above: It is an error to specify a ref that does not pass "git check-ref-format" scrutiny. Duplicated values are filtered.+email_privacy::+ Redact e-mail addresses from the generated HTML, etc. content.+ This hides e-mail addresses found in the commit log from web crawlers.+ Enabled by default.+++It is highly recommended to keep this feature enabled unless web crawlers+are hindered in some other way. You can disable this feature as shown below:+++---------------------------------------------------------------------------+$feature{'email_privacy'}{'default'} = [0];+---------------------------------------------------------------------------
I think there's plenty of gitweb users that are going to be relying on
the current behavior, so doesn't it make more sense for this to be
opt-in rather than opt-out?
@@ -569,6 +569,15 @@ sub evaluate_uri {'sub'=>\&feature_extra_branch_refs,'override'=>0,'default'=>[]},++# Redact e-mail addresses.++# To disable system wide have in $GITWEB_CONFIG+# $feature{'email_privacy'}{'default'} = [0];+'email_privacy'=>{+'sub'=>sub{feature_bool('email_privacy',@_)},+'override'=>0,+'default'=>[1]},);
All of these hunks (and the below) should use some new function that
does this feature check + sanitizing instead of copy/pasting mostly the
same code N times. e.g.:
sub maybe_hide_email {
my $email = shift;
return $email unless gitweb_check_feature('email_privacy');
return hide_email($email);
}
then:
$tag{author_email} = maybe_hide_email($2);
Also it looks like this isn't a new issue, but does this need to
implement its own E-Mail parser? We ship with Mail::Address for
git-send-email, can gitweb (and the elided hide_email() function above)
use that too?
quoted hunk
@@ -8060,8 +8085,13 @@ sub git_commitdiff { close $fd or print "Reading git-diff-tree failed\n"; } elsif ($format eq 'patch') {- local $/ = undef;- print <$fd>;+ while (my $line = <$fd>) {+ if (gitweb_check_feature('email_privacy') &&+ $line =~ m/^([^<]+) <([^>]*)>/) {+ $line =~ s/<([^>]+)>/<private>/;+ }+ print $line;+ } close $fd or print "Reading git-format-patch failed\n";
Is that "patch" output meant for "git am"? Won't this severely break
that use-case if so?
From: brian m. carlson <hidden> Date: 2021-03-21 02:12:27
On 2021-03-21 at 00:42:58, Ævar Arnfjörð Bjarmason wrote:
On Sun, Mar 21 2021, Georgios Kontaxis via GitGitGadget wrote:
quoted
From: Georgios Kontaxis <redacted>
Gitweb extracts content from the Git log and makes it accessible
over HTTP. As a result, e-mail addresses found in commits are
exposed to web crawlers. This may result in unsolicited messages.
This is a feature for redacting e-mail addresses from the generated
HTML content.
This feature does not prevent someone from downloading the
unredacted commit log and extracting information from it.
It aims to hinder the low-effort bulk collection of e-mail
addresses by web crawlers.
So web crawlers that aren't going to obey robots.txt?
I'm not opposed to this feature, but a glance at gitweb's documentation
seems to show that we don't discuss how to set robots.txt up for it at
all.
Perhaps having that in the docs or otherwise in the default setup would
get us most of the win of this feature?
I'm going to guess that the two features are orthogonal. robots.txt is
great for communicating to well-meaning actors what you do and don't
want crawled. For example, one might ask a web crawler not to crawl
individual commits because that creates excessive load on the server.
This option is about preventing email harvesting, usually for the
purposes of sending spam. Spam is email abuse and all reasonable people
know it's unacceptable, so by definition the people doing this are bad
actors and are not likely to honor the robots.txt. As someone who runs
his own mail server, that is certainly my experience.
So I am in favor of this feature. I think it mirrors what many other
tools do in this space and having it as an option is valuable.
@@ -896,6 +896,18 @@ same as of the snippet above: It is an error to specify a ref that does not pass "git check-ref-format" scrutiny. Duplicated values are filtered.+email_privacy::+ Redact e-mail addresses from the generated HTML, etc. content.+ This hides e-mail addresses found in the commit log from web crawlers.+ Enabled by default.+++It is highly recommended to keep this feature enabled unless web crawlers+are hindered in some other way. You can disable this feature as shown below:+++---------------------------------------------------------------------------+$feature{'email_privacy'}{'default'} = [0];+---------------------------------------------------------------------------
I think there's plenty of gitweb users that are going to be relying on
the current behavior, so doesn't it make more sense for this to be
opt-in rather than opt-out?
I agree this make sense as an opt-in feature. While many people will
want to enable it, users who are performing an upgrade won't necessarily
want the behavior to change right away.
--
brian m. carlson (he/him or they/them)
Houston, Texas, US
From: Georgios Kontaxis via GitGitGadget <hidden> Date: 2021-03-21 03:33:53
From: Georgios Kontaxis <redacted>
Gitweb extracts content from the Git log and makes it accessible
over HTTP. As a result, e-mail addresses found in commits are
exposed to web crawlers and they may not respect robots.txt.
This may result in unsolicited messages.
This is a feature for redacting e-mail addresses
from the generated HTML, etc. content.
This feature does not prevent someone from downloading the
unredacted commit log, e.g., by cloning the repository, and
extracting information from it.
It aims to hinder the low-effort bulk collection of e-mail
addresses by web crawlers.
Changes since v1:
- Turned off the feature by default.
- Removed duplicate code.
- Added note about Gitweb consumers receiving redacted logs.
Signed-off-by: Georgios Kontaxis <redacted>
---
gitweb: Redacted e-mail addresses feature.
Gitweb extracts content from the Git log and makes it accessible over
HTTP. As a result, e-mail addresses found in commits are exposed to web
crawlers. This may result in unsolicited messages. This is a feature for
redacting e-mail addresses from the generated HTML content.
This feature does not prevent someone from downloading the unredacted
commit log and extracting information from it. It aims to hinder the
low-effort bulk collection of e-mail addresses by web crawlers.
Signed-off-by: Georgios Kontaxis geko1702+commits@99rst.org
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-910%2Fkontaxis%2Fkontaxis%2Femail_privacy-v2
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-910/kontaxis/kontaxis/email_privacy-v2
Pull-Request: https://github.com/gitgitgadget/git/pull/910
Range-diff vs v1:
1: 6fe6ebdb8e59 ! 1: 74af11ca8bf2 gitweb: redacted e-mail addresses feature.
@@ Commit message
Gitweb extracts content from the Git log and makes it accessible
over HTTP. As a result, e-mail addresses found in commits are
- exposed to web crawlers. This may result in unsolicited messages.
- This is a feature for redacting e-mail addresses from the generated
- HTML content.
+ exposed to web crawlers and they may not respect robots.txt.
+ This may result in unsolicited messages.
+ This is a feature for redacting e-mail addresses
+ from the generated HTML, etc. content.
This feature does not prevent someone from downloading the
- unredacted commit log and extracting information from it.
+ unredacted commit log, e.g., by cloning the repository, and
+ extracting information from it.
It aims to hinder the low-effort bulk collection of e-mail
addresses by web crawlers.
+ Changes since v1:
+ - Turned off the feature by default.
+ - Removed duplicate code.
+ - Added note about Gitweb consumers receiving redacted logs.
+
Signed-off-by: Georgios Kontaxis [off-list ref]
## Documentation/gitweb.conf.txt ##
@@ Documentation/gitweb.conf.txt: same as of the snippet above:
+email_privacy::
+ Redact e-mail addresses from the generated HTML, etc. content.
+ This hides e-mail addresses found in the commit log from web crawlers.
-+ Enabled by default.
++ Disabled by default.
++
-+It is highly recommended to keep this feature enabled unless web crawlers
-+are hindered in some other way. You can disable this feature as shown below:
++It is highly recommended to enable this feature unless web crawlers are
++hindered in some other way. Note that crawlers intent on harvesting e-mail
++addresses may disregard robots.txt. You can enable this feature like so:
++
+---------------------------------------------------------------------------
-+$feature{'email_privacy'}{'default'} = [0];
++$feature{'email_privacy'}{'default'} = [1];
+---------------------------------------------------------------------------
+++
++Note that if Gitweb is not the final step in a workflow then subsequent
++steps may misbehave because of the redacted information they receive.
+
EXAMPLES
@@ gitweb/gitweb.perl: sub evaluate_uri {
'override' => 0,
'default' => []},
+
-+ # Redact e-mail addresses.
++ # Redact e-mail addresses.
+
-+ # To disable system wide have in $GITWEB_CONFIG
-+ # $feature{'email_privacy'}{'default'} = [0];
++ # To enable system wide have in $GITWEB_CONFIG
++ # $feature{'email_privacy'}{'default'} = [1];
+ 'email_privacy' => {
+ 'sub' => sub { feature_bool('email_privacy', @_) },
+ 'override' => 0,
-+ 'default' => [1]},
++ 'default' => [0]},
);
sub gitweb_get_feature {
+@@ gitweb/gitweb.perl: sub parse_date {
+ return %date;
+ }
+
++sub hide_mailaddr_if_private {
++ my $line = shift;
++ return $line unless (gitweb_check_feature('email_privacy') &&
++ $line =~ m/^([^<]+) <([^>]*)>/);
++ return hide_mailaddr($line)
++}
++
++sub hide_mailaddr {
++ my $mailaddr = shift;
++ $mailaddr =~ s/<([^>]*)>/<private>/;
++ return $mailaddr;
++}
++
+ sub parse_tag {
+ my $tag_id = shift;
+ my %tag;
@@ gitweb/gitweb.perl: sub parse_tag {
if ($tag{'author'} =~ m/^([^<]+) <([^>]*)>/) {
$tag{'author_name'} = $1;
$tag{'author_email'} = $2;
+ if (gitweb_check_feature('email_privacy')) {
+ $tag{'author_email'} = "private";
-+ $tag{'author'} =~ s/<([^>]+)>/<private>/;
++ $tag{'author'} = hide_mailaddr($tag{'author'});
+ }
} else {
$tag{'author_name'} = $tag{'author'};
@@ gitweb/gitweb.perl: sub parse_commit_text {
$co{'author_email'} = $2;
+ if (gitweb_check_feature('email_privacy')) {
+ $co{'author_email'} = "private";
-+ $co{'author'} =~ s/<([^>]+)>/<private>/;
++ $co{'author'} = hide_mailaddr($co{'author'});
+ }
} else {
$co{'author_name'} = $co{'author'};
@@ gitweb/gitweb.perl: sub parse_commit_text {
$co{'committer_email'} = $2;
+ if (gitweb_check_feature('email_privacy')) {
+ $co{'committer_email'} = "private";
-+ $co{'committer'} =~ s/<([^>]+)>/<private>/;
++ $co{'committer'} = hide_mailaddr($co{'committer'});
+ }
} else {
$co{'committer_name'} = $co{'committer'};
@@ gitweb/gitweb.perl: sub parse_commit_text {
+ # remove added spaces, redact e-mail addresses if applicable.
foreach my $line (@commit_lines) {
$line =~ s/^ //;
-+ if (gitweb_check_feature('email_privacy') &&
-+ $line =~ m/^([^<]+) <([^>]*)>/) {
-+ $line =~ s/<([^>]+)>/<private>/;
-+ }
++ $line = hide_mailaddr_if_private($line);
}
$co{'comment'} = \@commit_lines;
@@ gitweb/gitweb.perl: sub git_commitdiff {
- local $/ = undef;
- print <$fd>;
+ while (my $line = <$fd>) {
-+ if (gitweb_check_feature('email_privacy') &&
-+ $line =~ m/^([^<]+) <([^>]*)>/) {
-+ $line =~ s/<([^>]+)>/<private>/;
-+ }
-+ print $line;
++ print hide_mailaddr_if_private($line);
+ }
close $fd
or print "Reading git-format-patch failed\n";
Documentation/gitweb.conf.txt | 16 +++++++++++++
gitweb/gitweb.perl | 42 ++++++++++++++++++++++++++++++++---
2 files changed, 55 insertions(+), 3 deletions(-)
@@ -896,6 +896,22 @@ same as of the snippet above: It is an error to specify a ref that does not pass "git check-ref-format" scrutiny. Duplicated values are filtered.+email_privacy::+ Redact e-mail addresses from the generated HTML, etc. content.+ This hides e-mail addresses found in the commit log from web crawlers.+ Disabled by default.+++It is highly recommended to enable this feature unless web crawlers are+hindered in some other way. Note that crawlers intent on harvesting e-mail+addresses may disregard robots.txt. You can enable this feature like so:+++---------------------------------------------------------------------------+$feature{'email_privacy'}{'default'} = [1];+---------------------------------------------------------------------------+++Note that if Gitweb is not the final step in a workflow then subsequent+steps may misbehave because of the redacted information they receive.+ EXAMPLES --------
@@ -569,6 +569,15 @@ sub evaluate_uri {'sub'=>\&feature_extra_branch_refs,'override'=>0,'default'=>[]},++# Redact e-mail addresses.++# To enable system wide have in $GITWEB_CONFIG+# $feature{'email_privacy'}{'default'} = [1];+'email_privacy'=>{+'sub'=>sub{feature_bool('email_privacy',@_)},+'override'=>0,+'default'=>[0]},);subgitweb_get_feature{
@@ -3449,6 +3458,19 @@ sub parse_date {return%date;}+subhide_mailaddr_if_private{+my$line=shift;+return$lineunless(gitweb_check_feature('email_privacy')&&+$line=~m/^([^<]+) <([^>]*)>/);+returnhide_mailaddr($line)+}++subhide_mailaddr{+my$mailaddr=shift;+$mailaddr=~s/<([^>]*)>/<private>/;+return$mailaddr;+}+subparse_tag{my$tag_id=shift;my%tag;
@@ -3471,6 +3493,10 @@ sub parse_tag {if($tag{'author'}=~m/^([^<]+) <([^>]*)>/){$tag{'author_name'}=$1;$tag{'author_email'}=$2;+if(gitweb_check_feature('email_privacy')){+$tag{'author_email'}="private";+$tag{'author'}=hide_mailaddr($tag{'author'});+}}else{$tag{'author_name'}=$tag{'author'};}
@@ -3519,6 +3545,10 @@ sub parse_commit_text {if($co{'author'}=~m/^([^<]+) <([^>]*)>/){$co{'author_name'}=$1;$co{'author_email'}=$2;+if(gitweb_check_feature('email_privacy')){+$co{'author_email'}="private";+$co{'author'}=hide_mailaddr($co{'author'});+}}else{$co{'author_name'}=$co{'author'};}
@@ -3529,6 +3559,10 @@ sub parse_commit_text {if($co{'committer'}=~m/^([^<]+) <([^>]*)>/){$co{'committer_name'}=$1;$co{'committer_email'}=$2;+if(gitweb_check_feature('email_privacy')){+$co{'committer_email'}="private";+$co{'committer'}=hide_mailaddr($co{'committer'});+}}else{$co{'committer_name'}=$co{'committer'};}
On Sun, Mar 21 2021, Georgios Kontaxis via GitGitGadget wrote:
quoted
From: Georgios Kontaxis <redacted>
Gitweb extracts content from the Git log and makes it accessible
over HTTP. As a result, e-mail addresses found in commits are
exposed to web crawlers. This may result in unsolicited messages.
This is a feature for redacting e-mail addresses from the generated
HTML content.
This feature does not prevent someone from downloading the
unredacted commit log and extracting information from it.
It aims to hinder the low-effort bulk collection of e-mail
addresses by web crawlers.
So web crawlers that aren't going to obey robots.txt?
I'm not opposed to this feature, but a glance at gitweb's documentation
seems to show that we don't discuss how to set robots.txt up for it at
all.
Perhaps having that in the docs or otherwise in the default setup would
get us most of the win of this feature?
File robots.txt is basically asking nicely and we should work on that.
At the same time crawlers that look for addresses to send SPAM to
will probably ignore it so this change is meant for them.
quoted
Signed-off-by: Georgios Kontaxis <redacted>
---
Odd:
quoted
gitweb: Redacted e-mail addresses feature.
Gitweb extracts content from the Git log and makes it accessible
over
HTTP. As a result, e-mail addresses found in commits are exposed to
web
crawlers. This may result in unsolicited messages. This is a feature
for
redacting e-mail addresses from the generated HTML content.
This feature does not prevent someone from downloading the
unredacted
commit log and extracting information from it. It aims to hinder the
low-effort bulk collection of e-mail addresses by web crawlers.
Signed-off-by: Georgios Kontaxis geko1702+commits@99rst.org
To have this duplication of the patch here below "---", some GGG feature
gone awry?
@@ -896,6 +896,18 @@ same as of the snippet above: It is an error to specify a ref that does not pass "git
check-ref-format"
scrutiny. Duplicated values are filtered.
+email_privacy::
+ Redact e-mail addresses from the generated HTML, etc. content.
+ This hides e-mail addresses found in the commit log from web
crawlers.
+ Enabled by default.
++
+It is highly recommended to keep this feature enabled unless web
crawlers
+are hindered in some other way. You can disable this feature as shown
below:
++
+---------------------------------------------------------------------------
+$feature{'email_privacy'}{'default'} = [0];
+---------------------------------------------------------------------------
I think there's plenty of gitweb users that are going to be relying on
the current behavior, so doesn't it make more sense for this to be
opt-in rather than opt-out?
My concern is that Gitweb operators may not understand the need
for this feature or maybe won't be aware the feature exists.
Nevertheless, I've changed the feature to be off by default.
Perhaps we can revisit this decision in the future? :)
@@ -569,6 +569,15 @@ sub evaluate_uri {'sub'=>\&feature_extra_branch_refs,'override'=>0,'default'=>[]},++# Redact e-mail addresses.++# To disable system wide have in $GITWEB_CONFIG+# $feature{'email_privacy'}{'default'} = [0];+'email_privacy'=>{+'sub'=>sub{feature_bool('email_privacy',@_)},+'override'=>0,+'default'=>[1]},);
All of these hunks (and the below) should use some new function that
does this feature check + sanitizing instead of copy/pasting mostly the
same code N times. e.g.:
sub maybe_hide_email {
my $email = shift;
return $email unless gitweb_check_feature('email_privacy');
return hide_email($email);
}
then:
$tag{author_email} = maybe_hide_email($2);
Also it looks like this isn't a new issue, but does this need to
implement its own E-Mail parser? We ship with Mail::Address for
git-send-email, can gitweb (and the elided hide_email() function above)
use that too?
Thanks for the suggestion.
I've removed the duplicate code.
The places where there are e-mail addresses today are pretty specific.
(Pretty much the author, committer fields and the Signed-off-by lines)
In theory someone could write a comment with a bunch of addresses in it
but that would be unstructured text and I think Mail::Address is not good
with that.
I can definitely keep working on this topic but maybe in subsequent PRs?
Assuming we're not exposing any addresses right now or redacting things
we shouldn't.
quoted
@@ -8060,8 +8085,13 @@ sub git_commitdiff { close $fd or print "Reading git-diff-tree failed\n"; } elsif ($format eq 'patch') {- local $/ = undef;- print <$fd>;+ while (my $line = <$fd>) {+ if (gitweb_check_feature('email_privacy') &&+ $line =~ m/^([^<]+) <([^>]*)>/) {+ $line =~ s/<([^>]+)>/<private>/;+ }+ print $line;+ } close $fd or print "Reading git-format-patch failed\n";
Is that "patch" output meant for "git am"? Won't this severely break
that use-case if so?
Not sure who may be consuming that output.
I've added a note in the documentation for gitweb.conf.
If a web crawler can get to the information by following URLs
then I think we should redact it.
Hopefully by documenting this as a potential issue Gitweb operators
can create a workaround specific to their use case.
Possibly implementing access control and leaving this feature off.
From: Georgios Kontaxis via GitGitGadget <hidden> Date: 2021-03-21 17:29:27
From: Georgios Kontaxis <redacted>
Gitweb extracts content from the Git log and makes it accessible
over HTTP. As a result, e-mail addresses found in commits are
exposed to web crawlers and they may not respect robots.txt.
This may result in unsolicited messages.
This is a feature for redacting e-mail addresses
from the generated HTML, etc. content.
This feature does not prevent someone from downloading the
unredacted commit log, e.g., by cloning the repository, and
extracting information from it.
It aims to hinder the low-effort bulk collection of e-mail
addresses by web crawlers.
Changes since v1:
- Turned off the feature by default.
- Removed duplicate code.
- Added note about Gitweb consumers receiving redacted logs.
Changes since v2:
- The feature can be set on a per-project basis. ('override' => 1)
Signed-off-by: Georgios Kontaxis <redacted>
---
gitweb: Redacted e-mail addresses feature.
Gitweb extracts content from the Git log and makes it accessible over
HTTP. As a result, e-mail addresses found in commits are exposed to web
crawlers. This may result in unsolicited messages. This is a feature for
redacting e-mail addresses from the generated HTML content.
This feature does not prevent someone from downloading the unredacted
commit log and extracting information from it. It aims to hinder the
low-effort bulk collection of e-mail addresses by web crawlers.
Signed-off-by: Georgios Kontaxis geko1702+commits@99rst.org
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-910%2Fkontaxis%2Fkontaxis%2Femail_privacy-v3
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-910/kontaxis/kontaxis/email_privacy-v3
Pull-Request: https://github.com/gitgitgadget/git/pull/910
Range-diff vs v2:
1: 74af11ca8bf2 ! 1: 930cdefe7ee0 gitweb: redacted e-mail addresses feature.
@@ Commit message
- Removed duplicate code.
- Added note about Gitweb consumers receiving redacted logs.
+ Changes since v2:
+ - The feature can be set on a per-project basis. ('override' => 1)
+
Signed-off-by: Georgios Kontaxis [off-list ref]
## Documentation/gitweb.conf.txt ##
@@ gitweb/gitweb.perl: sub evaluate_uri {
+ # $feature{'email_privacy'}{'default'} = [1];
+ 'email_privacy' => {
+ 'sub' => sub { feature_bool('email_privacy', @_) },
-+ 'override' => 0,
++ 'override' => 1,
+ 'default' => [0]},
);
Documentation/gitweb.conf.txt | 16 +++++++++++++
gitweb/gitweb.perl | 42 ++++++++++++++++++++++++++++++++---
2 files changed, 55 insertions(+), 3 deletions(-)
@@ -896,6 +896,22 @@ same as of the snippet above: It is an error to specify a ref that does not pass "git check-ref-format" scrutiny. Duplicated values are filtered.+email_privacy::+ Redact e-mail addresses from the generated HTML, etc. content.+ This hides e-mail addresses found in the commit log from web crawlers.+ Disabled by default.+++It is highly recommended to enable this feature unless web crawlers are+hindered in some other way. Note that crawlers intent on harvesting e-mail+addresses may disregard robots.txt. You can enable this feature like so:+++---------------------------------------------------------------------------+$feature{'email_privacy'}{'default'} = [1];+---------------------------------------------------------------------------+++Note that if Gitweb is not the final step in a workflow then subsequent+steps may misbehave because of the redacted information they receive.+ EXAMPLES --------
@@ -569,6 +569,15 @@ sub evaluate_uri {'sub'=>\&feature_extra_branch_refs,'override'=>0,'default'=>[]},++# Redact e-mail addresses.++# To enable system wide have in $GITWEB_CONFIG+# $feature{'email_privacy'}{'default'} = [1];+'email_privacy'=>{+'sub'=>sub{feature_bool('email_privacy',@_)},+'override'=>1,+'default'=>[0]},);subgitweb_get_feature{
@@ -3449,6 +3458,19 @@ sub parse_date {return%date;}+subhide_mailaddr_if_private{+my$line=shift;+return$lineunless(gitweb_check_feature('email_privacy')&&+$line=~m/^([^<]+) <([^>]*)>/);+returnhide_mailaddr($line)+}++subhide_mailaddr{+my$mailaddr=shift;+$mailaddr=~s/<([^>]*)>/<private>/;+return$mailaddr;+}+subparse_tag{my$tag_id=shift;my%tag;
@@ -3471,6 +3493,10 @@ sub parse_tag {if($tag{'author'}=~m/^([^<]+) <([^>]*)>/){$tag{'author_name'}=$1;$tag{'author_email'}=$2;+if(gitweb_check_feature('email_privacy')){+$tag{'author_email'}="private";+$tag{'author'}=hide_mailaddr($tag{'author'});+}}else{$tag{'author_name'}=$tag{'author'};}
@@ -3519,6 +3545,10 @@ sub parse_commit_text {if($co{'author'}=~m/^([^<]+) <([^>]*)>/){$co{'author_name'}=$1;$co{'author_email'}=$2;+if(gitweb_check_feature('email_privacy')){+$co{'author_email'}="private";+$co{'author'}=hide_mailaddr($co{'author'});+}}else{$co{'author_name'}=$co{'author'};}
@@ -3529,6 +3559,10 @@ sub parse_commit_text {if($co{'committer'}=~m/^([^<]+) <([^>]*)>/){$co{'committer_name'}=$1;$co{'committer_email'}=$2;+if(gitweb_check_feature('email_privacy')){+$co{'committer_email'}="private";+$co{'committer'}=hide_mailaddr($co{'committer'});+}}else{$co{'committer_name'}=$co{'committer'};}
On Sun, Mar 21 2021, Georgios Kontaxis via GitGitGadget wrote:
quoted hunk
From: Georgios Kontaxis <redacted>
Gitweb extracts content from the Git log and makes it accessible
over HTTP. As a result, e-mail addresses found in commits are
exposed to web crawlers and they may not respect robots.txt.
This may result in unsolicited messages.
This is a feature for redacting e-mail addresses
from the generated HTML, etc. content.
This feature does not prevent someone from downloading the
unredacted commit log, e.g., by cloning the repository, and
extracting information from it.
It aims to hinder the low-effort bulk collection of e-mail
addresses by web crawlers.
Changes since v1:
- Turned off the feature by default.
- Removed duplicate code.
- Added note about Gitweb consumers receiving redacted logs.
Changes since v2:
- The feature can be set on a per-project basis. ('override' => 1)
Signed-off-by: Georgios Kontaxis <redacted>
---
gitweb: Redacted e-mail addresses feature.
Gitweb extracts content from the Git log and makes it accessible over
HTTP. As a result, e-mail addresses found in commits are exposed to web
crawlers. This may result in unsolicited messages. This is a feature for
redacting e-mail addresses from the generated HTML content.
This feature does not prevent someone from downloading the unredacted
commit log and extracting information from it. It aims to hinder the
low-effort bulk collection of e-mail addresses by web crawlers.
Signed-off-by: Georgios Kontaxis geko1702+commits@99rst.org
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-910%2Fkontaxis%2Fkontaxis%2Femail_privacy-v3
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-910/kontaxis/kontaxis/email_privacy-v3
Pull-Request: https://github.com/gitgitgadget/git/pull/910
Range-diff vs v2:
1: 74af11ca8bf2 ! 1: 930cdefe7ee0 gitweb: redacted e-mail addresses feature.
@@ Commit message
- Removed duplicate code.
- Added note about Gitweb consumers receiving redacted logs.
+ Changes since v2:
+ - The feature can be set on a per-project basis. ('override' => 1)
+
Signed-off-by: Georgios Kontaxis [off-list ref]
## Documentation/gitweb.conf.txt ##
@@ gitweb/gitweb.perl: sub evaluate_uri {
+ # $feature{'email_privacy'}{'default'} = [1];
+ 'email_privacy' => {
+ 'sub' => sub { feature_bool('email_privacy', @_) },
-+ 'override' => 0,
++ 'override' => 1,
+ 'default' => [0]},
);
Documentation/gitweb.conf.txt | 16 +++++++++++++
gitweb/gitweb.perl | 42 ++++++++++++++++++++++++++++++++---
2 files changed, 55 insertions(+), 3 deletions(-)
@@ -896,6 +896,22 @@ same as of the snippet above: It is an error to specify a ref that does not pass "git check-ref-format" scrutiny. Duplicated values are filtered.+email_privacy::+ Redact e-mail addresses from the generated HTML, etc. content.+ This hides e-mail addresses found in the commit log from web crawlers.+ Disabled by default.+++It is highly recommended to enable this feature unless web crawlers are+hindered in some other way. Note that crawlers intent on harvesting e-mail+addresses may disregard robots.txt. You can enable this feature like so:+++---------------------------------------------------------------------------+$feature{'email_privacy'}{'default'} = [1];+---------------------------------------------------------------------------+++Note that if Gitweb is not the final step in a workflow then subsequent+steps may misbehave because of the redacted information they receive.+ EXAMPLES --------
@@ -569,6 +569,15 @@ sub evaluate_uri {'sub'=>\&feature_extra_branch_refs,'override'=>0,'default'=>[]},++# Redact e-mail addresses.++# To enable system wide have in $GITWEB_CONFIG+# $feature{'email_privacy'}{'default'} = [1];+'email_privacy'=>{+'sub'=>sub{feature_bool('email_privacy',@_)},+'override'=>1,+'default'=>[0]},);subgitweb_get_feature{
@@ -3449,6 +3458,19 @@ sub parse_date {return%date;}
[snip]
So in the v1 feedback I suggested:
BEGIN QUOTE
sub maybe_hide_email {
my $email = shift;
return $email unless gitweb_check_feature('email_privacy');
return hide_email($email);
}
then:
$tag{author_email} = maybe_hide_email($2);
END QUOTE
But:
quoted hunk
sub parse_tag {
my $tag_id = shift;
my %tag;
@@ -3471,6 +3493,10 @@ sub parse_tag { if ($tag{'author'} =~ m/^([^<]+) <([^>]*)>/) { $tag{'author_name'} = $1; $tag{'author_email'} = $2;+ if (gitweb_check_feature('email_privacy')) {+ $tag{'author_email'} = "private";+ $tag{'author'} = hide_mailaddr($tag{'author'});+ }
This code seems quite awkward, we've already done the regex match, but
this code:
Is going to do it again incrementally, and then just act on a
search-replacement if we've got the feature enabled.
It seems much simpler to just turn your:
Into:
$tag{'author'} = maybe_hide_mailaddr($tag{author}, \$tag{author_email});
Using:
sub maybe_hide_email {
my ($email, $ref) = shift;
return $email unless gitweb_check_feature('email_privacy');
$$ref = "private" if $ref;
return hide_email($email);
}
Which also works for the case where you don't have a "private" hash key
to assign to. But maybe it overcomplicates things...
@@ -8060,8 +8095,9 @@ sub git_commitdiff { close $fd or print "Reading git-diff-tree failed\n"; } elsif ($format eq 'patch') {- local $/ = undef;- print <$fd>;+ while (my $line = <$fd>) {+ print hide_mailaddr_if_private($line);+ }
Urm, have you tested this? How does a while loop over a <$fd> make sense
when $/ is undef, the readline() operator will always return just one
record, so having a while loop doesn't make sense.
I'm not sure of the input here, but given that if you're expecting to
replace all e-mail addresses on all lines with this function that's not
how it'll work, the s/// doesn't have a /g, so it'll stop at the first
replacement.
close $fd
or print "Reading git-format-patch failed\n";
}
base-commit: a5828ae6b52137b913b978e16cd2334482eb4c1f
On Sun, Mar 21 2021, Georgios Kontaxis via GitGitGadget wrote:
quoted
From: Georgios Kontaxis <redacted>
Gitweb extracts content from the Git log and makes it accessible
over HTTP. As a result, e-mail addresses found in commits are
exposed to web crawlers and they may not respect robots.txt.
This may result in unsolicited messages.
This is a feature for redacting e-mail addresses
from the generated HTML, etc. content.
This feature does not prevent someone from downloading the
unredacted commit log, e.g., by cloning the repository, and
extracting information from it.
It aims to hinder the low-effort bulk collection of e-mail
addresses by web crawlers.
Changes since v1:
- Turned off the feature by default.
- Removed duplicate code.
- Added note about Gitweb consumers receiving redacted logs.
Changes since v2:
- The feature can be set on a per-project basis. ('override' => 1)
Signed-off-by: Georgios Kontaxis <redacted>
---
gitweb: Redacted e-mail addresses feature.
Gitweb extracts content from the Git log and makes it accessible
over
HTTP. As a result, e-mail addresses found in commits are exposed to
web
crawlers. This may result in unsolicited messages. This is a feature
for
redacting e-mail addresses from the generated HTML content.
This feature does not prevent someone from downloading the
unredacted
commit log and extracting information from it. It aims to hinder the
low-effort bulk collection of e-mail addresses by web crawlers.
Signed-off-by: Georgios Kontaxis geko1702+commits@99rst.org
Published-As:
https://github.com/gitgitgadget/git/releases/tag/pr-910%2Fkontaxis%2Fkontaxis%2Femail_privacy-v3
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git
pr-910/kontaxis/kontaxis/email_privacy-v3
Pull-Request: https://github.com/gitgitgadget/git/pull/910
Range-diff vs v2:
1: 74af11ca8bf2 ! 1: 930cdefe7ee0 gitweb: redacted e-mail addresses
feature.
@@ Commit message
- Removed duplicate code.
- Added note about Gitweb consumers receiving redacted logs.
+ Changes since v2:
+ - The feature can be set on a per-project basis. ('override'
=> 1)
+
Signed-off-by: Georgios Kontaxis [off-list ref]
## Documentation/gitweb.conf.txt ##
@@ gitweb/gitweb.perl: sub evaluate_uri {
+ # $feature{'email_privacy'}{'default'} = [1];
+ 'email_privacy' => {
+ 'sub' => sub { feature_bool('email_privacy', @_) },
-+ 'override' => 0,
++ 'override' => 1,
+ 'default' => [0]},
);
Documentation/gitweb.conf.txt | 16 +++++++++++++
gitweb/gitweb.perl | 42 ++++++++++++++++++++++++++++++++---
2 files changed, 55 insertions(+), 3 deletions(-)
diff --git a/Documentation/gitweb.conf.txt
b/Documentation/gitweb.conf.txt
index 7963a79ba98b..b7af3240177d 100644
@@ -896,6 +896,22 @@ same as of the snippet above: It is an error to specify a ref that does not pass "git
check-ref-format"
scrutiny. Duplicated values are filtered.
+email_privacy::
+ Redact e-mail addresses from the generated HTML, etc. content.
+ This hides e-mail addresses found in the commit log from web
crawlers.
+ Disabled by default.
++
+It is highly recommended to enable this feature unless web crawlers are
+hindered in some other way. Note that crawlers intent on harvesting
e-mail
+addresses may disregard robots.txt. You can enable this feature like
so:
++
+---------------------------------------------------------------------------
+$feature{'email_privacy'}{'default'} = [1];
+---------------------------------------------------------------------------
++
+Note that if Gitweb is not the final step in a workflow then subsequent
+steps may misbehave because of the redacted information they receive.
+
EXAMPLES
--------
@@ -569,6 +569,15 @@ sub evaluate_uri {'sub'=>\&feature_extra_branch_refs,'override'=>0,'default'=>[]},++# Redact e-mail addresses.++# To enable system wide have in $GITWEB_CONFIG+# $feature{'email_privacy'}{'default'} = [1];+'email_privacy'=>{+'sub'=>sub{feature_bool('email_privacy',@_)},+'override'=>1,+'default'=>[0]},);subgitweb_get_feature{
@@ -3449,6 +3458,19 @@ sub parse_date {return%date;}
[snip]
So in the v1 feedback I suggested:
BEGIN QUOTE
sub maybe_hide_email {
my $email = shift;
return $email unless gitweb_check_feature('email_privacy');
return hide_email($email);
}
then:
$tag{author_email} = maybe_hide_email($2);
END QUOTE
But:
quoted
sub parse_tag {
my $tag_id = shift;
my %tag;
@@ -3471,6 +3493,10 @@ sub parse_tag { if ($tag{'author'} =~ m/^([^<]+) <([^>]*)>/) { $tag{'author_name'} = $1; $tag{'author_email'} = $2;+ if (gitweb_check_feature('email_privacy')) {+ $tag{'author_email'} = "private";+ $tag{'author'} = hide_mailaddr($tag{'author'});+ }
This code seems quite awkward, we've already done the regex match, but
this code:
Is going to do it again incrementally, and then just act on a
search-replacement if we've got the feature enabled.
I think hide_mailaddr_if_private (which checks the feature flag
and looks for the pattern) is only called when we haven't done so
already. In every other case we call hide_mailaddr directly.
So there's no duplication of checks.
Into:
$tag{'author'} = maybe_hide_mailaddr($tag{author},
\$tag{author_email});
Using:
sub maybe_hide_email {
my ($email, $ref) = shift;
return $email unless gitweb_check_feature('email_privacy');
$$ref = "private" if $ref;
return hide_email($email);
}
Which also works for the case where you don't have a "private" hash key
to assign to. But maybe it overcomplicates things...
@@ -8060,8 +8095,9 @@ sub git_commitdiff { close $fd or print "Reading git-diff-tree failed\n"; } elsif ($format eq 'patch') {- local $/ = undef;- print <$fd>;+ while (my $line = <$fd>) {+ print hide_mailaddr_if_private($line);+ }
Urm, have you tested this? How does a while loop over a <$fd> make sense
when $/ is undef, the readline() operator will always return just one
record, so having a while loop doesn't make sense.
$/ is not undef, the opposite, it's the default value which results
in reading from <$fd> one line at a time. (where line is a sequence
terminated by CRLF or LF)
I'm not sure of the input here, but given that if you're expecting to
replace all e-mail addresses on all lines with this function that's not
how it'll work, the s/// doesn't have a /g, so it'll stop at the first
replacement.
So we are reading from <$fd> one line at a time and looking for the
first "expr1 <expr2>" occurrence in each line. This successfully redacts
the author, committer, signed-off-by lines.
Not doing a global replacement per line is intentional so as to avoid
false positives.
So far I haven't noticed missing any addresses that should have been
redacted.
quoted
close $fd
or print "Reading git-format-patch failed\n";
}
base-commit: a5828ae6b52137b913b978e16cd2334482eb4c1f
From: Georgios Kontaxis via GitGitGadget <hidden> Date: 2021-03-22 06:58:45
From: Georgios Kontaxis <redacted>
Gitweb extracts content from the Git log and makes it accessible
over HTTP. As a result, e-mail addresses found in commits are
exposed to web crawlers and they may not respect robots.txt.
This may result in unsolicited messages.
This is a feature for redacting e-mail addresses
from the generated HTML, etc. content.
This feature does not prevent someone from downloading the
unredacted commit log, e.g., by cloning the repository, and
extracting information from it.
It aims to hinder the low-effort bulk collection of e-mail
addresses by web crawlers.
Signed-off-by: Georgios Kontaxis <redacted>
---
gitweb: redacted e-mail addresses feature.
Gitweb extracts content from the Git log and makes it accessible over
HTTP. As a result, e-mail addresses found in commits are exposed to web
crawlers and they may not respect robots.txt. This may result in
unsolicited messages. This is a feature for redacting e-mail addresses
from the generated HTML, etc. content.
This feature does not prevent someone from downloading the unredacted
commit log, e.g., by cloning the repository, and extracting information
from it. It aims to hinder the low-effort bulk collection of e-mail
addresses by web crawlers.
Changes since v1:
* Turned off the feature by default.
* Removed duplicate code.
* Added note about Gitweb consumers receiving redacted logs.
Changes since v2:
* The feature can be set on a per-project basis. ('override' => 1)
Changes since v3:
* Renamed feature to "email-privacy" and improved documentation.
* Removed UI elements for git-format-patch since it won't be redacted.
* Simplified calls to the address redaction logic.
* Mail::Address is now used to reduce false-positive redactions.
Signed-off-by: Georgios Kontaxis geko1702+commits@99rst.org
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-910%2Fkontaxis%2Fkontaxis%2Femail_privacy-v4
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-910/kontaxis/kontaxis/email_privacy-v4
Pull-Request: https://github.com/gitgitgadget/git/pull/910
Range-diff vs v3:
1: 930cdefe7ee0 ! 1: 03a3f41c37ef gitweb: redacted e-mail addresses feature.
@@ Commit message
It aims to hinder the low-effort bulk collection of e-mail
addresses by web crawlers.
- Changes since v1:
- - Turned off the feature by default.
- - Removed duplicate code.
- - Added note about Gitweb consumers receiving redacted logs.
-
- Changes since v2:
- - The feature can be set on a per-project basis. ('override' => 1)
-
Signed-off-by: Georgios Kontaxis [off-list ref]
## Documentation/gitweb.conf.txt ##
-@@ Documentation/gitweb.conf.txt: same as of the snippet above:
- It is an error to specify a ref that does not pass "git check-ref-format"
- scrutiny. Duplicated values are filtered.
+@@ Documentation/gitweb.conf.txt: default font sizes or lineheights are changed (e.g. via adding extra
+ CSS stylesheet in `@stylesheets`), it may be appropriate to change
+ these values.
-+email_privacy::
-+ Redact e-mail addresses from the generated HTML, etc. content.
-+ This hides e-mail addresses found in the commit log from web crawlers.
-+ Disabled by default.
-++
-+It is highly recommended to enable this feature unless web crawlers are
-+hindered in some other way. Note that crawlers intent on harvesting e-mail
-+addresses may disregard robots.txt. You can enable this feature like so:
-++
-+---------------------------------------------------------------------------
-+$feature{'email_privacy'}{'default'} = [1];
-+---------------------------------------------------------------------------
-++
-+Note that if Gitweb is not the final step in a workflow then subsequent
-+steps may misbehave because of the redacted information they receive.
++email-privacy::
++ Redact e-mail addresses from the generated HTML, etc. content.
++ This hides e-mail addresses found in the commit log from HTTP clients.
++ It is meant to hinder web crawlers that harvest and abuse addresses.
++ Such crawlers may not respect robots.txt.
++ Note that users and user tools also see the addresses redacted.
++ If Gitweb is not the final step in a workflow then subsequent steps
++ may misbehave because of the redacted information they receive.
++ Disabled by default.
+
-
- EXAMPLES
- --------
+ highlight::
+ Server-side syntax highlight support in "blob" view. It requires
+ `$highlight_bin` program to be available (see the description of
## gitweb/gitweb.perl ##
+@@
+ use File::Basename qw(basename);
+ use Time::HiRes qw(gettimeofday tv_interval);
+ use Digest::MD5 qw(md5_hex);
++use Git::LoadCPAN::Mail::Address;
+
+ binmode STDOUT, ':utf8';
+
@@ gitweb/gitweb.perl: sub evaluate_uri {
'sub' => \&feature_extra_branch_refs,
'override' => 0,
@@ gitweb/gitweb.perl: sub evaluate_uri {
+ # Redact e-mail addresses.
+
+ # To enable system wide have in $GITWEB_CONFIG
-+ # $feature{'email_privacy'}{'default'} = [1];
-+ 'email_privacy' => {
-+ 'sub' => sub { feature_bool('email_privacy', @_) },
++ # $feature{'email-privacy'}{'default'} = [1];
++ 'email-privacy' => {
++ 'sub' => sub { feature_bool('email-privacy', @_) },
+ 'override' => 1,
+ 'default' => [0]},
);
@@ gitweb/gitweb.perl: sub parse_date {
return %date;
}
-+sub hide_mailaddr_if_private {
-+ my $line = shift;
-+ return $line unless (gitweb_check_feature('email_privacy') &&
-+ $line =~ m/^([^<]+) <([^>]*)>/);
-+ return hide_mailaddr($line)
++sub is_mailaddr {
++ my @addrs = Mail::Address->parse(shift);
++ if (!@addrs || !$addrs[0]->host || !$addrs[0]->user) {
++ return 0;
++ }
++ return 1;
+}
+
-+sub hide_mailaddr {
-+ my $mailaddr = shift;
-+ $mailaddr =~ s/<([^>]*)>/<private>/;
-+ return $mailaddr;
++sub hide_mailaddrs_if_private {
++ my $line = shift;
++ return $line unless gitweb_check_feature('email-privacy');
++ while ($line =~ m/(<[^>]+>)/g) {
++ my $match = $1;
++ if (!is_mailaddr($match)) {
++ next;
++ }
++ my $offset = pos $line;
++ my $head = substr $line, 0, $offset - length($match);
++ my $redaction = "<redacted>";
++ my $tail = substr $line, $offset;
++ $line = $head . $redaction . $tail;
++ pos $line = length($head) + length($redaction);
++ }
++ return $line;
+}
+
sub parse_tag {
my $tag_id = shift;
my %tag;
@@ gitweb/gitweb.perl: sub parse_tag {
+ } elsif ($line =~ m/^tag (.+)$/) {
+ $tag{'name'} = $1;
+ } elsif ($line =~ m/^tagger (.*) ([0-9]+) (.*)$/) {
+- $tag{'author'} = $1;
++ $tag{'author'} = hide_mailaddrs_if_private($1);
+ $tag{'author_epoch'} = $2;
+ $tag{'author_tz'} = $3;
if ($tag{'author'} =~ m/^([^<]+) <([^>]*)>/) {
- $tag{'author_name'} = $1;
- $tag{'author_email'} = $2;
-+ if (gitweb_check_feature('email_privacy')) {
-+ $tag{'author_email'} = "private";
-+ $tag{'author'} = hide_mailaddr($tag{'author'});
-+ }
- } else {
- $tag{'author_name'} = $tag{'author'};
- }
@@ gitweb/gitweb.perl: sub parse_commit_text {
+ } elsif ((!defined $withparents) && ($line =~ m/^parent ($oid_regex)$/)) {
+ push @parents, $1;
+ } elsif ($line =~ m/^author (.*) ([0-9]+) (.*)$/) {
+- $co{'author'} = to_utf8($1);
++ $co{'author'} = hide_mailaddrs_if_private(to_utf8($1));
+ $co{'author_epoch'} = $2;
+ $co{'author_tz'} = $3;
if ($co{'author'} =~ m/^([^<]+) <([^>]*)>/) {
- $co{'author_name'} = $1;
- $co{'author_email'} = $2;
-+ if (gitweb_check_feature('email_privacy')) {
-+ $co{'author_email'} = "private";
-+ $co{'author'} = hide_mailaddr($co{'author'});
-+ }
- } else {
+@@ gitweb/gitweb.perl: sub parse_commit_text {
$co{'author_name'} = $co{'author'};
}
-@@ gitweb/gitweb.perl: sub parse_commit_text {
+ } elsif ($line =~ m/^committer (.*) ([0-9]+) (.*)$/) {
+- $co{'committer'} = to_utf8($1);
++ $co{'committer'} = hide_mailaddrs_if_private(to_utf8($1));
+ $co{'committer_epoch'} = $2;
+ $co{'committer_tz'} = $3;
if ($co{'committer'} =~ m/^([^<]+) <([^>]*)>/) {
- $co{'committer_name'} = $1;
- $co{'committer_email'} = $2;
-+ if (gitweb_check_feature('email_privacy')) {
-+ $co{'committer_email'} = "private";
-+ $co{'committer'} = hide_mailaddr($co{'committer'});
-+ }
- } else {
- $co{'committer_name'} = $co{'committer'};
- }
@@ gitweb/gitweb.perl: sub parse_commit_text {
if (! defined $co{'title'} || $co{'title'} eq "") {
$co{'title'} = $co{'title_short'} = '(no commit message)';
@@ gitweb/gitweb.perl: sub parse_commit_text {
+ # remove added spaces, redact e-mail addresses if applicable.
foreach my $line (@commit_lines) {
$line =~ s/^ //;
-+ $line = hide_mailaddr_if_private($line);
++ $line = hide_mailaddrs_if_private($line);
}
$co{'comment'} = \@commit_lines;
-@@ gitweb/gitweb.perl: sub git_commitdiff {
- close $fd
- or print "Reading git-diff-tree failed\n";
- } elsif ($format eq 'patch') {
-- local $/ = undef;
-- print <$fd>;
-+ while (my $line = <$fd>) {
-+ print hide_mailaddr_if_private($line);
-+ }
- close $fd
- or print "Reading git-format-patch failed\n";
+@@ gitweb/gitweb.perl: sub git_log_generic {
+ -accesskey => "n", -title => "Alt-n"}, "next");
+ }
+ my $patch_max = gitweb_get_feature('patches');
+- if ($patch_max && !defined $file_name) {
++ if ($patch_max && !defined $file_name &&
++ !gitweb_check_feature('email-privacy')) {
+ if ($patch_max < 0 || @commitlist <= $patch_max) {
+ $paging_nav .= " ⋅ " .
+ $cgi->a({-href => href(action=>"patches", -replay=>1)},
+@@ gitweb/gitweb.perl: sub git_commit {
+ } @$parents ) .
+ ')';
}
+- if (gitweb_check_feature('patches') && @$parents <= 1) {
++ if (gitweb_check_feature('patches') && @$parents <= 1 &&
++ !gitweb_check_feature('email-privacy')) {
+ $formats_nav .= " | " .
+ $cgi->a({-href => href(action=>"patch", -replay=>1)},
+ "patch");
+@@ gitweb/gitweb.perl: sub git_commitdiff {
+ $formats_nav =
+ $cgi->a({-href => href(action=>"commitdiff_plain", -replay=>1)},
+ "raw");
+- if ($patch_max && @{$co{'parents'}} <= 1) {
++ if ($patch_max && @{$co{'parents'}} <= 1 &&
++ !gitweb_check_feature('email-privacy')) {
+ $formats_nav .= " | " .
+ $cgi->a({-href => href(action=>"patch", -replay=>1)},
+ "patch");
+
+ ## t/lib-gitweb.sh ##
+@@ t/lib-gitweb.sh: gitweb_run () {
+ GITWEB_CONFIG=$(pwd)/gitweb_config.perl
+ export GITWEB_CONFIG
+
++ PERL5LIB="$GIT_BUILD_DIR/perl:$GIT_BUILD_DIR/perl/FromCPAN"
++ export PERL5LIB
++
+ # some of git commands write to STDERR on error, but this is not
+ # written to web server logs, so we are not interested in that:
+ # we are interested only in properly formatted errors/warnings
Documentation/gitweb.conf.txt | 10 +++++++
gitweb/gitweb.perl | 54 ++++++++++++++++++++++++++++++-----
t/lib-gitweb.sh | 3 ++
3 files changed, 60 insertions(+), 7 deletions(-)
@@ -751,6 +751,16 @@ default font sizes or lineheights are changed (e.g. via adding extra CSS stylesheet in `@stylesheets`), it may be appropriate to change these values.+email-privacy::+ Redact e-mail addresses from the generated HTML, etc. content.+ This hides e-mail addresses found in the commit log from HTTP clients.+ It is meant to hinder web crawlers that harvest and abuse addresses.+ Such crawlers may not respect robots.txt.+ Note that users and user tools also see the addresses redacted.+ If Gitweb is not the final step in a workflow then subsequent steps+ may misbehave because of the redacted information they receive.+ Disabled by default.+ highlight:: Server-side syntax highlight support in "blob" view. It requires `$highlight_bin` program to be available (see the description of
@@ -569,6 +570,15 @@ sub evaluate_uri {'sub'=>\&feature_extra_branch_refs,'override'=>0,'default'=>[]},++# Redact e-mail addresses.++# To enable system wide have in $GITWEB_CONFIG+# $feature{'email-privacy'}{'default'} = [1];+'email-privacy'=>{+'sub'=>sub{feature_bool('email-privacy',@_)},+'override'=>1,+'default'=>[0]},);subgitweb_get_feature{
@@ -3449,6 +3459,32 @@ sub parse_date {return%date;}+subis_mailaddr{+my@addrs=Mail::Address->parse(shift);+if(!@addrs||!$addrs[0]->host||!$addrs[0]->user){+return0;+}+return1;+}++subhide_mailaddrs_if_private{+my$line=shift;+return$lineunlessgitweb_check_feature('email-privacy');+while($line=~m/(<[^>]+>)/g){+my$match=$1;+if(!is_mailaddr($match)){+next;+}+my$offset=pos$line;+my$head=substr$line,0,$offset-length($match);+my$redaction="<redacted>";+my$tail=substr$line,$offset;+$line=$head.$redaction.$tail;+pos$line=length($head)+length($redaction);+}+return$line;+}+subparse_tag{my$tag_id=shift;my%tag;
@@ -67,6 +67,9 @@ gitweb_run () {GITWEB_CONFIG=$(pwd)/gitweb_config.perlexportGITWEB_CONFIG+PERL5LIB="$GIT_BUILD_DIR/perl:$GIT_BUILD_DIR/perl/FromCPAN"+exportPERL5LIB+# some of git commands write to STDERR on error, but this is not# written to web server logs, so we are not interested in that:# we are interested only in properly formatted errors/warnings
From: Georgios Kontaxis via GitGitGadget <hidden> Date: 2021-03-27 03:57:15
From: Georgios Kontaxis <redacted>
Gitweb extracts content from the Git log and makes it accessible
over HTTP. As a result, e-mail addresses found in commits are
exposed to web crawlers and they may not respect robots.txt.
This can result in unsolicited messages.
Introduce an 'email-privacy' feature which redacts e-mail addresses
from the generated HTML content. Specifically, obscure addresses
retrieved from the the author/committer and comment sections of the
Git log. The feature is off by default.
This feature does not prevent someone from downloading the
unredacted commit log, e.g., by cloning the repository, and
extracting information from it. It aims to hinder the low-
effort, bulk collection of e-mail addresses by web crawlers.
Signed-off-by: Georgios Kontaxis <redacted>
---
gitweb: redacted e-mail addresses feature.
Changes since v1:
* Turned off the feature by default.
* Removed duplicate code.
* Added note about Gitweb consumers receiving redacted logs.
Changes since v2:
* The feature can be set on a per-project basis. ('override' => 1)
Changes since v3:
* Renamed feature to "email-privacy" and improved documentation.
* Removed UI elements for git-format-patch since it won't be redacted.
* Simplified calls to the address redaction logic.
* Mail::Address is now used to reduce false-positive redactions.
Changes since v4:
* Rephrased the commit comment.
* hide_mailaddrs_if_private is slighly more compact.
Signed-off-by: Georgios Kontaxis geko1702+commits@99rst.org
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-910%2Fkontaxis%2Fkontaxis%2Femail_privacy-v5
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-910/kontaxis/kontaxis/email_privacy-v5
Pull-Request: https://github.com/gitgitgadget/git/pull/910
Range-diff vs v4:
1: 03a3f41c37ef ! 1: 1427231f9db5 gitweb: redacted e-mail addresses feature.
@@ Commit message
Gitweb extracts content from the Git log and makes it accessible
over HTTP. As a result, e-mail addresses found in commits are
exposed to web crawlers and they may not respect robots.txt.
- This may result in unsolicited messages.
- This is a feature for redacting e-mail addresses
- from the generated HTML, etc. content.
+ This can result in unsolicited messages.
+
+ Introduce an 'email-privacy' feature which redacts e-mail addresses
+ from the generated HTML content. Specifically, obscure addresses
+ retrieved from the the author/committer and comment sections of the
+ Git log. The feature is off by default.
This feature does not prevent someone from downloading the
unredacted commit log, e.g., by cloning the repository, and
- extracting information from it.
- It aims to hinder the low-effort bulk collection of e-mail
- addresses by web crawlers.
+ extracting information from it. It aims to hinder the low-
+ effort, bulk collection of e-mail addresses by web crawlers.
Signed-off-by: Georgios Kontaxis [off-list ref]
@@ Documentation/gitweb.conf.txt: default font sizes or lineheights are changed (e.
+email-privacy::
+ Redact e-mail addresses from the generated HTML, etc. content.
-+ This hides e-mail addresses found in the commit log from HTTP clients.
++ This obscures e-mail addresses retrieved from the author/committer
++ and comment sections of the Git log.
+ It is meant to hinder web crawlers that harvest and abuse addresses.
+ Such crawlers may not respect robots.txt.
-+ Note that users and user tools also see the addresses redacted.
++ Note that users and user tools also see the addresses as redacted.
+ If Gitweb is not the final step in a workflow then subsequent steps
+ may misbehave because of the redacted information they receive.
+ Disabled by default.
@@ gitweb/gitweb.perl: sub parse_date {
+ if (!is_mailaddr($match)) {
+ next;
+ }
-+ my $offset = pos $line;
-+ my $head = substr $line, 0, $offset - length($match);
++ my $match_offset = pos($line) - length($match);
++ pos $line = $match_offset;
++
+ my $redaction = "<redacted>";
-+ my $tail = substr $line, $offset;
-+ $line = $head . $redaction . $tail;
-+ pos $line = length($head) + length($redaction);
++ $line =~ s/\G(<[^>]+>)/$redaction/;
++
++ pos $line = $match_offset + length($redaction);
+ }
+ return $line;
+}
Documentation/gitweb.conf.txt | 11 +++++++
gitweb/gitweb.perl | 55 ++++++++++++++++++++++++++++++-----
t/lib-gitweb.sh | 3 ++
3 files changed, 62 insertions(+), 7 deletions(-)
@@ -751,6 +751,17 @@ default font sizes or lineheights are changed (e.g. via adding extra CSS stylesheet in `@stylesheets`), it may be appropriate to change these values.+email-privacy::+ Redact e-mail addresses from the generated HTML, etc. content.+ This obscures e-mail addresses retrieved from the author/committer+ and comment sections of the Git log.+ It is meant to hinder web crawlers that harvest and abuse addresses.+ Such crawlers may not respect robots.txt.+ Note that users and user tools also see the addresses as redacted.+ If Gitweb is not the final step in a workflow then subsequent steps+ may misbehave because of the redacted information they receive.+ Disabled by default.+ highlight:: Server-side syntax highlight support in "blob" view. It requires `$highlight_bin` program to be available (see the description of
@@ -569,6 +570,15 @@ sub evaluate_uri {'sub'=>\&feature_extra_branch_refs,'override'=>0,'default'=>[]},++# Redact e-mail addresses.++# To enable system wide have in $GITWEB_CONFIG+# $feature{'email-privacy'}{'default'} = [1];+'email-privacy'=>{+'sub'=>sub{feature_bool('email-privacy',@_)},+'override'=>1,+'default'=>[0]},);subgitweb_get_feature{
@@ -3449,6 +3459,33 @@ sub parse_date {return%date;}+subis_mailaddr{+my@addrs=Mail::Address->parse(shift);+if(!@addrs||!$addrs[0]->host||!$addrs[0]->user){+return0;+}+return1;+}++subhide_mailaddrs_if_private{+my$line=shift;+return$lineunlessgitweb_check_feature('email-privacy');+while($line=~m/(<[^>]+>)/g){+my$match=$1;+if(!is_mailaddr($match)){+next;+}+my$match_offset=pos($line)-length($match);+pos$line=$match_offset;++my$redaction="<redacted>";+$line=~s/\G(<[^>]+>)/$redaction/;++pos$line=$match_offset+length($redaction);+}+return$line;+}+subparse_tag{my$tag_id=shift;my%tag;
@@ -67,6 +67,9 @@ gitweb_run () {GITWEB_CONFIG=$(pwd)/gitweb_config.perlexportGITWEB_CONFIG+PERL5LIB="$GIT_BUILD_DIR/perl:$GIT_BUILD_DIR/perl/FromCPAN"+exportPERL5LIB+# some of git commands write to STDERR on error, but this is not# written to web server logs, so we are not interested in that:# we are interested only in properly formatted errors/warnings
From: Georgios Kontaxis via GitGitGadget <hidden> Date: 2021-03-28 23:27:41
From: Georgios Kontaxis <redacted>
Gitweb extracts content from the Git log and makes it accessible
over HTTP. As a result, e-mail addresses found in commits are
exposed to web crawlers and they may not respect robots.txt.
This can result in unsolicited messages.
Introduce an 'email-privacy' feature which redacts e-mail addresses
from the generated HTML content. Specifically, obscure addresses
retrieved from the the author/committer and comment sections of the
Git log. The feature is off by default.
This feature does not prevent someone from downloading the
unredacted commit log, e.g., by cloning the repository, and
extracting information from it. It aims to hinder the low-
effort, bulk collection of e-mail addresses by web crawlers.
Signed-off-by: Georgios Kontaxis <redacted>
---
gitweb: redacted e-mail addresses feature.
Changes since v1:
* Turned off the feature by default.
* Removed duplicate code.
* Added note about Gitweb consumers receiving redacted logs.
Changes since v2:
* The feature can be set on a per-project basis. ('override' => 1)
Changes since v3:
* Renamed feature to "email-privacy" and improved documentation.
* Removed UI elements for git-format-patch since it won't be redacted.
* Simplified calls to the address redaction logic.
* Mail::Address is now used to reduce false-positive redactions.
Changes since v4:
* Rephrased the commit comment.
* hide_mailaddrs_if_private is slighly more compact.
Changes since v5:
* A simple <local@domain> filter is used instead of Mail::Address to
identify addresses.
Signed-off-by: Georgios Kontaxis geko1702+commits@99rst.org
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-910%2Fkontaxis%2Fkontaxis%2Femail_privacy-v6
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-910/kontaxis/kontaxis/email_privacy-v6
Pull-Request: https://github.com/gitgitgadget/git/pull/910
Range-diff vs v5:
1: 1427231f9db5 ! 1: 245cfed8ad58 gitweb: redacted e-mail addresses feature.
@@ Documentation/gitweb.conf.txt: default font sizes or lineheights are changed (e.
`$highlight_bin` program to be available (see the description of
## gitweb/gitweb.perl ##
-@@
- use File::Basename qw(basename);
- use Time::HiRes qw(gettimeofday tv_interval);
- use Digest::MD5 qw(md5_hex);
-+use Git::LoadCPAN::Mail::Address;
-
- binmode STDOUT, ':utf8';
-
@@ gitweb/gitweb.perl: sub evaluate_uri {
'sub' => \&feature_extra_branch_refs,
'override' => 0,
@@ gitweb/gitweb.perl: sub parse_date {
return %date;
}
-+sub is_mailaddr {
-+ my @addrs = Mail::Address->parse(shift);
-+ if (!@addrs || !$addrs[0]->host || !$addrs[0]->user) {
-+ return 0;
-+ }
-+ return 1;
-+}
-+
+sub hide_mailaddrs_if_private {
+ my $line = shift;
+ return $line unless gitweb_check_feature('email-privacy');
-+ while ($line =~ m/(<[^>]+>)/g) {
-+ my $match = $1;
-+ if (!is_mailaddr($match)) {
-+ next;
-+ }
-+ my $match_offset = pos($line) - length($match);
-+ pos $line = $match_offset;
-+
-+ my $redaction = "<redacted>";
-+ $line =~ s/\G(<[^>]+>)/$redaction/;
-+
-+ pos $line = $match_offset + length($redaction);
-+ }
++ $line =~ s/<[^@>]+@[^>]+>/<redacted>/ig;
+ return $line;
+}
+
@@ gitweb/gitweb.perl: sub git_commitdiff {
$formats_nav .= " | " .
$cgi->a({-href => href(action=>"patch", -replay=>1)},
"patch");
-
- ## t/lib-gitweb.sh ##
-@@ t/lib-gitweb.sh: gitweb_run () {
- GITWEB_CONFIG=$(pwd)/gitweb_config.perl
- export GITWEB_CONFIG
-
-+ PERL5LIB="$GIT_BUILD_DIR/perl:$GIT_BUILD_DIR/perl/FromCPAN"
-+ export PERL5LIB
-+
- # some of git commands write to STDERR on error, but this is not
- # written to web server logs, so we are not interested in that:
- # we are interested only in properly formatted errors/warnings
Documentation/gitweb.conf.txt | 11 +++++++++++
gitweb/gitweb.perl | 34 +++++++++++++++++++++++++++-------
2 files changed, 38 insertions(+), 7 deletions(-)
@@ -751,6 +751,17 @@ default font sizes or lineheights are changed (e.g. via adding extra CSS stylesheet in `@stylesheets`), it may be appropriate to change these values.+email-privacy::+ Redact e-mail addresses from the generated HTML, etc. content.+ This obscures e-mail addresses retrieved from the author/committer+ and comment sections of the Git log.+ It is meant to hinder web crawlers that harvest and abuse addresses.+ Such crawlers may not respect robots.txt.+ Note that users and user tools also see the addresses as redacted.+ If Gitweb is not the final step in a workflow then subsequent steps+ may misbehave because of the redacted information they receive.+ Disabled by default.+ highlight:: Server-side syntax highlight support in "blob" view. It requires `$highlight_bin` program to be available (see the description of
@@ -569,6 +569,15 @@ sub evaluate_uri {'sub'=>\&feature_extra_branch_refs,'override'=>0,'default'=>[]},++# Redact e-mail addresses.++# To enable system wide have in $GITWEB_CONFIG+# $feature{'email-privacy'}{'default'} = [1];+'email-privacy'=>{+'sub'=>sub{feature_bool('email-privacy',@_)},+'override'=>1,+'default'=>[0]},);subgitweb_get_feature{
@@ -3449,6 +3458,13 @@ sub parse_date {return%date;}+subhide_mailaddrs_if_private{+my$line=shift;+return$lineunlessgitweb_check_feature('email-privacy');+$line=~s/<[^@>]+@[^>]+>/<redacted>/ig;+return$line;+}+subparse_tag{my$tag_id=shift;my%tag;
From: Eric Wong <hidden> Date: 2021-03-29 01:48:19
Georgios Kontaxis via GitGitGadget [off-list ref] wrote:
Gitweb extracts content from the Git log and makes it accessible
over HTTP. As a result, e-mail addresses found in commits are
exposed to web crawlers and they may not respect robots.txt.
This can result in unsolicited messages.
Introduce an 'email-privacy' feature which redacts e-mail addresses
from the generated HTML content
A general reply to the topic: have you considered munging
addresses in a way that is still human readable, but obviously
obfuscated?
On some other project, I settled on HTML "•" as a replacement
for '.' for admins who enable that option. The $USER@$NO_DOT
remains as-is for easy identification+recognition of hosts.
I also considered Unicode homographs which can look identical
to replacement characters, too; but rejected that idea since
it would cause grief for legitimate users who would not notice
the homograph when pasting into their mail client.
Anyways, here's the list of candidates I tried:
homograph∂80x24.org
homograph@80x24ͺorg
homograph@80x24·org
homograph@80x24•org
homograph＠80x24.org
homograph﹫80x24.org
https://en.wikipedia.org/wiki/Ano_Teleia#Similar_symbolshttps://en.wikipedia.org/wiki/Enclosed_A
homographⒶ80x24.org
homograph@80x24 org
homograph@80x24․org
homograph@80x24ꓸorg
Georgios Kontaxis via GitGitGadget [off-list ref] wrote:
quoted
Gitweb extracts content from the Git log and makes it accessible
over HTTP. As a result, e-mail addresses found in commits are
exposed to web crawlers and they may not respect robots.txt.
This can result in unsolicited messages.
quoted
Introduce an 'email-privacy' feature which redacts e-mail addresses
from the generated HTML content
A general reply to the topic: have you considered munging
addresses in a way that is still human readable, but obviously
obfuscated?
On some other project, I settled on HTML "•" as a replacement
for '.' for admins who enable that option. The $USER@$NO_DOT
remains as-is for easy identification+recognition of hosts.
Thanks for the suggestion.
People have been trying to hinder address harvesting for a while now.
Replacing '@' with "at", the dot with "dot", adding spaces, etc.
was pretty common at some point. May still be.
I would expect crawlers to have caught up and this includes
all sorts of character encodings and unicode look-alike substitutions.
At the end of the day we are looking for something that's easy for humans
to read but hard for scripts to parse as an e-mail address.
(And that scripts cannot learn through an additional regex)
I'm not aware of anything like that. (I know CAPTCHAs, etc.)
I also considered Unicode homographs which can look identical
to replacement characters, too; but rejected that idea since
it would cause grief for legitimate users who would not notice
the homograph when pasting into their mail client.
Anyways, here's the list of candidates I tried:
homograph∂80x24.org
homograph@80x24ͺorg
homograph@80x24·org
homograph@80x24•org
homograph＠80x24.org
homograph﹫80x24.org
https://en.wikipedia.org/wiki/Ano_Teleia#Similar_symbolshttps://en.wikipedia.org/wiki/Enclosed_A
homographⒶ80x24.org
homograph@80x24 org
homograph@80x24․org
homograph@80x24ꓸorg
From: Eric Wong <hidden> Date: 2021-04-08 17:16:50
Georgios Kontaxis [off-list ref] wrote:
quoted
Georgios Kontaxis via GitGitGadget [off-list ref] wrote:
quoted
Introduce an 'email-privacy' feature which redacts e-mail addresses
from the generated HTML content
Eric Wong wrote:
quoted
A general reply to the topic: have you considered munging
addresses in a way that is still human readable, but obviously
obfuscated?
On some other project, I settled on HTML "•" as a replacement
for '.' for admins who enable that option. The $USER@$NO_DOT
remains as-is for easy identification+recognition of hosts.
Thanks for the suggestion.
People have been trying to hinder address harvesting for a while now.
Replacing '@' with "at", the dot with "dot", adding spaces, etc.
was pretty common at some point. May still be.
I would expect crawlers to have caught up and this includes
all sorts of character encodings and unicode look-alike substitutions.
I figure the crawlers hit a combinatorial explosion and
give up since they'd be wasting time with false-positives.
quoted
I also considered Unicode homographs which can look identical
to replacement characters, too; but rejected that idea since
it would cause grief for legitimate users who would not notice
the homograph when pasting into their mail client.
As a data point, none of the homograph@ candidates I posted here
on Mar 29 have attracted any attempts on my mail server.
On Mon, Mar 29 2021, Georgios Kontaxis via GitGitGadget wrote:
quoted hunk
[...]
+email-privacy::
+ Redact e-mail addresses from the generated HTML, etc. content.
+ This obscures e-mail addresses retrieved from the author/committer
+ and comment sections of the Git log.
+ It is meant to hinder web crawlers that harvest and abuse addresses.
+ Such crawlers may not respect robots.txt.
+ Note that users and user tools also see the addresses as redacted.
+ If Gitweb is not the final step in a workflow then subsequent steps
+ may misbehave because of the redacted information they receive.
+ Disabled by default.
+
highlight::
Server-side syntax highlight support in "blob" view. It requires
`$highlight_bin` program to be available (see the description of
@@ -569,6 +569,15 @@ sub evaluate_uri {'sub'=>\&feature_extra_branch_refs,'override'=>0,'default'=>[]},++# Redact e-mail addresses.++# To enable system wide have in $GITWEB_CONFIG+# $feature{'email-privacy'}{'default'} = [1];+'email-privacy'=>{+'sub'=>sub{feature_bool('email-privacy',@_)},+'override'=>1,+'default'=>[0]},);subgitweb_get_feature{
@@ -3449,6 +3458,13 @@ sub parse_date {return%date;}+subhide_mailaddrs_if_private{+my$line=shift;+return$lineunlessgitweb_check_feature('email-privacy');+$line=~s/<[^@>]+@[^>]+>/<redacted>/ig;
The /i here is redundant, since you have nothing that'll case-fold on
the LHS of the s///, doesn't harm anything either. Just a small note
since it's new in v6...
quoted hunk
+ return $line;+}+ sub parse_tag { my $tag_id = shift; my %tag;