[PATCH] add http.maxReceiveSpeed to limit git-receive-pack receiving speed

Sometimes need to limit the receive speed of git `clone/fetch`
because of the limited network bandwidth, otherwise will prevent
other applications from using the network normally.

Add `http.maxReceiveSpeed` to limit `git-receive-pack` receiving
speed, Can be overridden by `GIT_HTTP_MAX_RECEIVE_SPEED` eivironment
variable.

The default is unlimited, same if the value is 0 or negative. The
default unit is Bytes/s, common unit suffixes of k, m, or g are supported.

this configuration is valid for `clone`, `fetch`, `pull` commands of the
https protocol, and only supports libcurl 7.15.5 and above.
---
 http.c | 24 +++++++++++++++++++++++-
 1 file changed, 23 insertions(+), 1 deletion(-)

diff --git a/http.c b/http.c
index 8119247149..12030cf3bc 100644
--- a/http.c
+++ b/http.c

@@ -83,6 +83,9 @@ static const char *ssl_pinnedkey;
 static const char *ssl_cainfo;
 static long curl_low_speed_limit = -1;
 static long curl_low_speed_time = -1;
+#if LIBCURL_VERSION_NUM >= 0x070f05
+static ssize_t curl_max_receive_speed = -1;
+#endif
 static int curl_ftp_no_epsv;
 static const char *curl_http_proxy;
 static const char *http_proxy_authmethod;

@@ -361,7 +364,12 @@ static int http_options(const char *var, const char *value, void *cb)
 		curl_low_speed_time = (long)git_config_int(var, value);
 		return 0;
 	}
-
+#if LIBCURL_VERSION_NUM >= 0x070f05
+	if (!strcmp("http.maxreceivespeed", var)) {
+		curl_max_receive_speed = git_config_ssize_t(var, value);
+		return 0;
+	}
+#endif
 	if (!strcmp("http.noepsv", var)) {
 		curl_ftp_no_epsv = git_config_bool(var, value);
 		return 0;

@@ -974,6 +982,12 @@ static CURL *get_curl_handle(void)
 				 curl_low_speed_time);
 	}
 
+#if LIBCURL_VERSION_NUM >= 0x070f05
+	if (curl_max_receive_speed > 0)
+		curl_easy_setopt(result, CURLOPT_MAX_RECV_SPEED_LARGE,
+				 curl_max_receive_speed);
+#endif
+
 	curl_easy_setopt(result, CURLOPT_MAXREDIRS, 20);
 #if LIBCURL_VERSION_NUM >= 0x071301
 	curl_easy_setopt(result, CURLOPT_POSTREDIR, CURL_REDIR_POST_ALL);

@@ -1105,6 +1119,9 @@ void http_init(struct remote *remote, const char *url, int proactive_auth)
 {
 	char *low_speed_limit;
 	char *low_speed_time;
+#if LIBCURL_VERSION_NUM >= 0x070f05
+	char *max_receive_speed;
+#endif
 	char *normalized_url;
 	struct urlmatch_config config = { STRING_LIST_INIT_DUP };
 

@@ -1196,6 +1213,11 @@ void http_init(struct remote *remote, const char *url, int proactive_auth)
 	low_speed_time = getenv("GIT_HTTP_LOW_SPEED_TIME");
 	if (low_speed_time != NULL)
 		curl_low_speed_time = strtol(low_speed_time, NULL, 10);
+#if LIBCURL_VERSION_NUM >= 0x070f05
+	max_receive_speed = getenv("GIT_HTTP_MAX_RECEIVE_SPEED");
+	if (max_receive_speed && !git_parse_ssize_t(max_receive_speed, &curl_max_receive_speed))
+		warning("failed to parse GIT_HTTP_MAX_RECEIVE_SPEED: %s", max_receive_speed);
+#endif
 
 	if (curl_ssl_verify == -1)
 		curl_ssl_verify = 1;

-- 
2.28.1.49.gafcb914ae4.dirty.agit.6.3.1

On 19/08/21 16.14, Xia XiaoWen wrote:

Sometimes need to limit the receive speed of git `clone/fetch`
because of the limited network bandwidth, otherwise will prevent
other applications from using the network normally.

I mean "In some situations it is preferable to limit receiving speed of 
git clone/fetch due to network bandwidth constraint, otherwise other 
applications performance will suffer from degraded network performance.".

Add `http.maxReceiveSpeed` to limit `git-receive-pack` receiving
speed, Can be overridden by `GIT_HTTP_MAX_RECEIVE_SPEED` eivironment
variable.

s/eivironment/environment/

The default is unlimited, same if the value is 0 or negative. The
default unit is Bytes/s, common unit suffixes of k, m, or g are supported.

Wouldn't it make sense for negative speed? Why don't just return error 
if negative speed is set?

Why is there boilerplate #if #endif block for libcurl 7.15.5?

-- 
An old man doll... just what I always wanted! - Clara

I mean "In some situations it is preferable to limit receiving speed of
git clone/fetch due to network bandwidth constraint, otherwise other
applications performance will suffer from degraded network performance.".

Thanks!

Wouldn't it make sense for negative speed? Why don't just return error
if negative speed is set?

You are right, negative speed doesn't make sense. but I prefer printf
a warning and continue work instead of returning an error, negative
speed seems not a fatal error.

Why is there boilerplate #if #endif block for libcurl 7.15.5?

The speed limit depends on the `CURLOPT_MAX_RECV_SPEED_LARGE` provided
by libCURL, and libCURL added `CURLOPT_MAX_RECV_SPEED_LARGE` in
7.15.5: https://curl.se/changes.html#7_15_5 .

Bagas Sanjaya [off-list ref] 于2021年8月19日周四 下午5:36写道：

On 19/08/21 16.14, Xia XiaoWen wrote:

quoted

Sometimes need to limit the receive speed of git `clone/fetch`
because of the limited network bandwidth, otherwise will prevent
other applications from using the network normally.

I mean "In some situations it is preferable to limit receiving speed of
git clone/fetch due to network bandwidth constraint, otherwise other
applications performance will suffer from degraded network performance.".

quoted

Add `http.maxReceiveSpeed` to limit `git-receive-pack` receiving
speed, Can be overridden by `GIT_HTTP_MAX_RECEIVE_SPEED` eivironment
variable.

s/eivironment/environment/

quoted

The default is unlimited, same if the value is 0 or negative. The
default unit is Bytes/s, common unit suffixes of k, m, or g are supported.

Wouldn't it make sense for negative speed? Why don't just return error
if negative speed is set?

quoted

this configuration is valid for `clone`, `fetch`, `pull` commands of the
https protocol, and only supports libcurl 7.15.5 and above.
---
  http.c | 24 +++++++++++++++++++++++-
  1 file changed, 23 insertions(+), 1 deletion(-)

diff --git a/http.c b/http.c
index 8119247149..12030cf3bc 100644
--- a/http.c
+++ b/http.c

@@ -83,6 +83,9 @@ static const char *ssl_pinnedkey;
  static const char *ssl_cainfo;
  static long curl_low_speed_limit = -1;
  static long curl_low_speed_time = -1;
+#if LIBCURL_VERSION_NUM >= 0x070f05
+static ssize_t curl_max_receive_speed = -1;
+#endif
  static int curl_ftp_no_epsv;
  static const char *curl_http_proxy;
  static const char *http_proxy_authmethod;

@@ -361,7 +364,12 @@ static int http_options(const char *var, const char *value, void *cb)
              curl_low_speed_time = (long)git_config_int(var, value);
              return 0;
      }
-
+#if LIBCURL_VERSION_NUM >= 0x070f05
+     if (!strcmp("http.maxreceivespeed", var)) {
+             curl_max_receive_speed = git_config_ssize_t(var, value);
+             return 0;
+     }
+#endif
      if (!strcmp("http.noepsv", var)) {
              curl_ftp_no_epsv = git_config_bool(var, value);
              return 0;

@@ -974,6 +982,12 @@ static CURL *get_curl_handle(void)
                               curl_low_speed_time);
      }

+#if LIBCURL_VERSION_NUM >= 0x070f05
+     if (curl_max_receive_speed > 0)
+             curl_easy_setopt(result, CURLOPT_MAX_RECV_SPEED_LARGE,
+                              curl_max_receive_speed);
+#endif
+
      curl_easy_setopt(result, CURLOPT_MAXREDIRS, 20);
  #if LIBCURL_VERSION_NUM >= 0x071301
      curl_easy_setopt(result, CURLOPT_POSTREDIR, CURL_REDIR_POST_ALL);

@@ -1105,6 +1119,9 @@ void http_init(struct remote *remote, const char *url, int proactive_auth)
  {
      char *low_speed_limit;
      char *low_speed_time;
+#if LIBCURL_VERSION_NUM >= 0x070f05
+     char *max_receive_speed;
+#endif
      char *normalized_url;
      struct urlmatch_config config = { STRING_LIST_INIT_DUP };

@@ -1196,6 +1213,11 @@ void http_init(struct remote *remote, const char *url, int proactive_auth)
      low_speed_time = getenv("GIT_HTTP_LOW_SPEED_TIME");
      if (low_speed_time != NULL)
              curl_low_speed_time = strtol(low_speed_time, NULL, 10);
+#if LIBCURL_VERSION_NUM >= 0x070f05
+     max_receive_speed = getenv("GIT_HTTP_MAX_RECEIVE_SPEED");
+     if (max_receive_speed && !git_parse_ssize_t(max_receive_speed, &curl_max_receive_speed))
+             warning("failed to parse GIT_HTTP_MAX_RECEIVE_SPEED: %s", max_receive_speed);
+#endif

      if (curl_ssl_verify == -1)
              curl_ssl_verify = 1;

Why is there boilerplate #if #endif block for libcurl 7.15.5?

--
An old man doll... just what I always wanted! - Clara

In order to avoid hogging all the available network bandwidth, users may want to
limit the speed of receiving traffic for "git clone" or "git fetch".

Add `http.maxReceiveSpeed` to limit receiving speed of `git-receive-pack`.
Can be overridden by `GIT_HTTP_MAX_RECEIVE_SPEED` environment variable.

The default is unlimited, same if the value is 0. The default unit is Bytes/s,
common unit suffixes of k, m, or g are supported.

This configuration is valid for `clone`, `fetch`, `pull` commands of the https
protocol.

Signed-off-by: chenan.xxw <redacted>
---
 Documentation/config/http.txt |  4 ++++
 http.c                        | 21 +++++++++++++++++++++
 2 files changed, 25 insertions(+)

diff --git a/Documentation/config/http.txt b/Documentation/config/http.txt
index 7003661c0d..9b9fb5e9c7 100644
--- a/Documentation/config/http.txt
+++ b/Documentation/config/http.txt

@@ -235,6 +235,10 @@ http.lowSpeedLimit, http.lowSpeedTime::
 	Can be overridden by the `GIT_HTTP_LOW_SPEED_LIMIT` and
 	`GIT_HTTP_LOW_SPEED_TIME` environment variables.
 
+http.maxReceiveSpeed::
+	Limit the speed of receiving traffic, defaults to unlimited. Can be
+	overridden by the `GIT_HTTP_MAX_RECEIVE_SPEED` environment variable.
+
 http.noEPSV::
 	A boolean which disables using of EPSV ftp command by curl.
 	This can helpful with some "poor" ftp servers which don't

diff --git a/http.c b/http.c
index 8119247149..b12d192ffe 100644
--- a/http.c
+++ b/http.c

@@ -83,6 +83,7 @@ static const char *ssl_pinnedkey;
 static const char *ssl_cainfo;
 static long curl_low_speed_limit = -1;
 static long curl_low_speed_time = -1;
+static long curl_max_receive_speed;
 static int curl_ftp_no_epsv;
 static const char *curl_http_proxy;
 static const char *http_proxy_authmethod;

@@ -362,6 +363,13 @@ static int http_options(const char *var, const char *value, void *cb)
 		return 0;
 	}
 
+	if (!strcmp("http.maxreceivespeed", var)) {
+		curl_max_receive_speed = (long)git_config_int(var, value);
+		if (curl_max_receive_speed < 0)
+			die(_("negative values are not allowed for http.maxreceivespeed"));
+		return 0;
+	}
+
 	if (!strcmp("http.noepsv", var)) {
 		curl_ftp_no_epsv = git_config_bool(var, value);
 		return 0;

@@ -974,6 +982,10 @@ static CURL *get_curl_handle(void)
 				 curl_low_speed_time);
 	}
 
+	if (curl_max_receive_speed >= 0)
+		curl_easy_setopt(result, CURLOPT_MAX_RECV_SPEED_LARGE,
+				 (curl_off_t)curl_max_receive_speed);
+
 	curl_easy_setopt(result, CURLOPT_MAXREDIRS, 20);
 #if LIBCURL_VERSION_NUM >= 0x071301
 	curl_easy_setopt(result, CURLOPT_POSTREDIR, CURL_REDIR_POST_ALL);

@@ -1105,6 +1117,8 @@ void http_init(struct remote *remote, const char *url, int proactive_auth)
 {
 	char *low_speed_limit;
 	char *low_speed_time;
+	char *mrs;
+	static const char mrs_env[] = "GIT_HTTP_MAX_RECEIVE_SPEED";
 	char *normalized_url;
 	struct urlmatch_config config = { STRING_LIST_INIT_DUP };
 

@@ -1197,6 +1211,13 @@ void http_init(struct remote *remote, const char *url, int proactive_auth)
 	if (low_speed_time != NULL)
 		curl_low_speed_time = strtol(low_speed_time, NULL, 10);
 
+	mrs = getenv(mrs_env);
+	if (mrs != NULL) {
+		curl_max_receive_speed = strtol(mrs, NULL, 10);
+		if (curl_max_receive_speed < 0)
+			die(_("negative values are not allowed for %s"), mrs_env);
+	}
+
 	if (curl_ssl_verify == -1)
 		curl_ssl_verify = 1;
 

-- 
2.33.0.1.g26e5a845c1.dirty

On Tue, Aug 24, 2021 at 12:05 AM chenan.xxw [off-list ref] wrote:

In order to avoid hogging all the available network bandwidth, users may want to
limit the speed of receiving traffic for "git clone" or "git fetch".

Add `http.maxReceiveSpeed` to limit receiving speed of `git-receive-pack`.
Can be overridden by `GIT_HTTP_MAX_RECEIVE_SPEED` environment variable.

The default is unlimited, same if the value is 0. The default unit is Bytes/s,
common unit suffixes of k, m, or g are supported.

This configuration is valid for `clone`, `fetch`, `pull` commands of the https
protocol.

Signed-off-by: chenan.xxw <redacted>

Your email address in the s-o-b is different from the "From" header.
That will generate a commit with a different author email. This is
because you use gmail as the SMTP, but the value of "user.email"
config variable is not a valid alias email of Gmail. A workaround is
use a different "user.email" config for git-send-email, and
git-send-email will add a "From: your <real@email>" as the first line
of the body. E.g.:

    git -c user.email="haoyurenzhuxia@gmail.com" send-email <patch-file>

BTW, use your real username (Xia Xiaowen) instead of the nickname "chenan.xxw".

    git config --global user.name "Xia Xiaowen"
    git commit --reset-author --amend

You can set default value -1 for curl_max_receive_speed, just like
curl_low_speed_limit does.
I wonder if you can rename the variable name to curl_max_speed_limit
for both upload and download.

Can we use a better name, such as "http.maxspeedlimit" ?  And make
some changes to limit both upload and download over HTTP protocol.

You can also set "CURLOPT_MAX_SEND_SPEED_LARGE" to limit bandwidth for git-push.

You introduced a new l10n message for translation. Why not make it as
simple as parsing env "GIT_HTTP_LOW_SPEED_LIMIT" or
"GIT_HTTP_LOW_SPEED_TIME" above?

--
Jiang Xin

Thanks for Jiang Xin's reply, it's very helpful.

But I carefully considered the purpose of this patch this week. As
Jeff King and Randall said, speed-limiting should be the function of
the operating system or the switch/router, and I agree with it. Users
may not need to limit the speed for `git-receive-pack` in most cases.
On the other side, for the ssh protocol, there is no way to limiting
speed in git. Please ignore this patch In general.