infra/pkg-generic: use site method for same-site extra downloads

Started by Yann E. MORIN on Nov 8, 2019 · 3 messages from 3 authors · Last reply Nov 9, 2019

Subsystems (1)

Maintainers (1)

STALE2554d

open the first message on its own page

  1. [Buildroot] [PATCH] infra/pkg-generic: use site method for same-site extra downloads
    Related patches
  2. [Buildroot] [PATCH] infra/pkg-generic: use site method for same-site extra downloads
    Related patches
Message details and related content
When a package specifies extra downloads, it has the option to only name
the basename of the extra download, in which case that extra download
will be retrieved from the same location the main download is retrieved
from.

In that case, if the extra download contains a '+', it would confuse the
dl-wrapper, which believes the LHS of the '+' is the site method, and
the RHS the actual URI, and so the dl-wrapper mangles and damages the
URI when fetching such extra downloads, like that happens with android
tools, where the proper URI and mangled URIs of the extra download are,
respectively:

    https://launchpad.net/ubuntu/+archive/primary/+files/android-tools_4.2.2+git20130218-3ubuntu41.debian.tar.gz
    http://archive/primary/+files/android-tools_4.2.2+git20130218-3ubuntu41.debian.tar.gz

We fix that by always propagating the site method to extra downloads,
but only when they are specified as relative to the main download URI.

For the extra downloads that specify a full URI, it is not systematic
that it is the same site method. For example, a main download could be a
git clone, but an extra download a pure http download; in that case we
can't replicate the site method for extra downloads, so they'll have to
take appropriate care to specify the required method and encoding if
needed.

Reported-by: Jemy Zhang <redacted>
Signed-off-by: Yann E. MORIN <redacted>
Cc: Jemy Zhang <redacted>
---
 package/pkg-generic.mk | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk
index 893faba22a..7d6fa08418 100644
--- a/package/pkg-generic.mk
+++ b/package/pkg-generic.mk
@@ -524,7 +524,7 @@ $(2)_ALL_DOWNLOADS = \
 	$$(if $$($(2)_SOURCE),$$($(2)_SITE_METHOD)+$$($(2)_SITE)/$$($(2)_SOURCE)) \
 	$$(foreach p,$$($(2)_PATCH) $$($(2)_EXTRA_DOWNLOADS),\
 		$$(if $$(findstring ://,$$(p)),$$(p),\
-			$$($(2)_SITE)/$$(p)))
+			$$($(2)_SITE_METHOD)+$$($(2)_SITE)/$$(p)))
 
 ifndef $(2)_SITE
  ifdef $(3)_SITE
-- 
2.20.1
Message details and related content
On Fri,  8 Nov 2019 18:26:45 +0100
"Yann E. MORIN" [off-list ref] wrote:
When a package specifies extra downloads, it has the option to only name
the basename of the extra download, in which case that extra download
will be retrieved from the same location the main download is retrieved
from.

In that case, if the extra download contains a '+', it would confuse the
dl-wrapper, which believes the LHS of the '+' is the site method, and
the RHS the actual URI, and so the dl-wrapper mangles and damages the
URI when fetching such extra downloads, like that happens with android
tools, where the proper URI and mangled URIs of the extra download are,
respectively:

    https://launchpad.net/ubuntu/+archive/primary/+files/android-tools_4.2.2+git20130218-3ubuntu41.debian.tar.gz
    http://archive/primary/+files/android-tools_4.2.2+git20130218-3ubuntu41.debian.tar.gz

We fix that by always propagating the site method to extra downloads,
but only when they are specified as relative to the main download URI.

For the extra downloads that specify a full URI, it is not systematic
that it is the same site method. For example, a main download could be a
git clone, but an extra download a pure http download; in that case we
can't replicate the site method for extra downloads, so they'll have to
take appropriate care to specify the required method and encoding if
needed.

Reported-by: Jemy Zhang <redacted>
Signed-off-by: Yann E. MORIN <redacted>
Cc: Jemy Zhang <redacted>
---
 package/pkg-generic.mk | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Applied to master, thanks.

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
Message details and related content
quoted
quoted
quoted
quoted
"Yann" == Yann E MORIN [off-list ref] writes:
 > When a package specifies extra downloads, it has the option to only name
 > the basename of the extra download, in which case that extra download
 > will be retrieved from the same location the main download is retrieved
 > from.

 > In that case, if the extra download contains a '+', it would confuse the
 > dl-wrapper, which believes the LHS of the '+' is the site method, and
 > the RHS the actual URI, and so the dl-wrapper mangles and damages the
 > URI when fetching such extra downloads, like that happens with android
 > tools, where the proper URI and mangled URIs of the extra download are,
 > respectively:

 >     https://launchpad.net/ubuntu/+archive/primary/+files/android-tools_4.2.2+git20130218-3ubuntu41.debian.tar.gz
 >     http://archive/primary/+files/android-tools_4.2.2+git20130218-3ubuntu41.debian.tar.gz

 > We fix that by always propagating the site method to extra downloads,
 > but only when they are specified as relative to the main download URI.

 > For the extra downloads that specify a full URI, it is not systematic
 > that it is the same site method. For example, a main download could be a
 > git clone, but an extra download a pure http download; in that case we
 > can't replicate the site method for extra downloads, so they'll have to
 > take appropriate care to specify the required method and encoding if
 > needed.

 > Reported-by: Jemy Zhang [off-list ref]
 > Signed-off-by: Yann E. MORIN [off-list ref]
 > Cc: Jemy Zhang [off-list ref]

Committed to 2019.02.x and 2019.08.x, thanks.

-- 
Bye, Peter Korsgaard
Keyboard shortcuts
hback out one level
jnext message in thread / next series-diff position
kprevious message in thread / previous series-diff position
ldrill in
Escclose help / fold thread tree
?toggle this help