From: Maciej Soltysiak <hidden> Date: 2003-01-08 13:17:52
tornado.wiggy.net is my client running Linux 2.4.19 (unpatched, UP machine),
and 2001:968:1::2 is the icecast server running Linux 2.4.20-rc2-ac3 (SMP).
If you want to test the stream yourself, please stream from
http://ipv6.lkml.org:8000/difm .
Which ipv6 client should i be using ?
Regards,
Maciej
I am using a patched version of xmms using the patch from
http://bugs.debian.org/155955 . (Don't forget to rerun autoconf
after applying the patch). If you want I can create an ipv6
enabled xmms.deb for you if you are using Debian.
Wichert.
--
Wichert Akkerman [off-list ref] http://www.wiggy.net/
A random hacker
From: Maciej Soltysiak <hidden> Date: 2003-01-08 13:42:25
I am using a patched version of xmms using the patch from
http://bugs.debian.org/155955 . (Don't forget to rerun autoconf
after applying the patch). If you want I can create an ipv6
enabled xmms.deb for you if you are using Debian.
Well, i have xmms from deb package, so i would rather use an
ip6 enabled deb. If that's not a problem.
Maciej
From: Maciej Soltysiak <hidden> Date: 2003-01-08 14:34:46
I had no problems listening to the stream, except a gap after about 3mins.
tcpdump showed the client closed the connection, and quickly initiated a
new one. Since then i had 15mins of nonstop playback and it stopped,
similarily to your dump.
The tcpdump is similar to yours, except i do not have traffic class info.
And rarely sack was used.
Is there a ip6 mangling router in your route to the icecast server?
I have been listening on an ip6 enabled host behind my ip6 tunnelling
router to my MAN.
Client: linux-2.4.21-pre1
Router: linux-2.4.20-grsec
I have to go now, i will look into that later.
Regards,
Maciej
Is there a ip6 mangling router in your route to the icecast server?
Not to my knowledge. traceroute6 shows:
traceroute to ipv6.lkml.org (2001:968:1::2) from
3ffe:8280:10:1d0:290:27ff:fe2d:968c, 30 hops max, 16 byte packets
1 thunder.wiggy.net (3ffe:8280:10:1d0:250:4ff:fe0b:dd79) 0.666 ms 0.22 ms 0.199 ms
2 xs4all-29.ipv6.xs4all.nl (3ffe:8280:0:2001::58) 27.568 ms 28.012 ms 30.177 ms
3 26.ge-0-2-0.xr1.pbw.xs4all.net (2001:888:0:3::1) 22.035 ms 19.528 ms 44.644 ms
4 0.ge-0-3-0.xr1.sara.xs4all.net (2001:888:2:1::1) 19.519 ms 19.002 ms 21.974 ms
5 fe-0-0-0.ams-core-01.network6.isp-services.nl (2001:7f8:1::a502:4875:1) 19.978 ms 30.278 ms 20.248 ms
6 2001:968::2 (2001:968::2) 24.246 ms 24.083 ms 22.918 ms
7 2001:968:1::2 (2001:968:1::2) 24.978 ms 23.866 ms 23.661 ms
thunder.wiggy.net is my Linux router running 2.4.19-pre5-ac3-freeswan196
currently. The second hop is a normal sit tunnel and all the other
hops are native ipv6 using Cisco and Juniper routers as far as I know.
If you want I can get a detailed list of the routers and the IOS/JunOS
versions they are running.
Wichert.
--
Wichert Akkerman [off-list ref] http://www.wiggy.net/
A random hacker
traceroute to ipv6.lkml.org (2001:968:1::2) from
3ffe:8280:10:1d0:290:27ff:fe2d:968c, 30 hops max, 16 byte packets
1 thunder.wiggy.net (3ffe:8280:10:1d0:250:4ff:fe0b:dd79) 0.666 ms 0.22 ms 0.199 ms
2 xs4all-29.ipv6.xs4all.nl (3ffe:8280:0:2001::58) 27.568 ms 28.012 ms 30.177 ms
3 26.ge-0-2-0.xr1.pbw.xs4all.net (2001:888:0:3::1) 22.035 ms 19.528 ms 44.644 ms
4 0.ge-0-3-0.xr1.sara.xs4all.net (2001:888:2:1::1) 19.519 ms 19.002 ms 21.974 ms
5 fe-0-0-0.ams-core-01.network6.isp-services.nl (2001:7f8:1::a502:4875:1) 19.978 ms 30.278 ms 20.248 ms
6 2001:968::2 (2001:968::2) 24.246 ms 24.083 ms 22.918 ms
7 2001:968:1::2 (2001:968:1::2) 24.978 ms 23.866 ms 23.661 ms
thunder.wiggy.net is my Linux router running 2.4.19-pre5-ac3-freeswan196
currently. The second hop is a normal sit tunnel and all the other
hops are native ipv6 using Cisco and Juniper routers as far as I know.
Slight correction to that: xs4all-29.ipv6.xs4all.nl is a FreeBSD-4.5
tunnel server. The toher xs4all.net hops are Junipers running JunOS 5.3
or 5.5.
Wichert.
--
Wichert Akkerman [off-list ref] http://www.wiggy.net/
A random hacker
From: Fabio Massimo Di Nitto <hidden> Date: 2003-01-08 15:13:29
Hi
On Wed, 8 Jan 2003, Wichert Akkerman wrote:
tornado.wiggy.net is my client running Linux 2.4.19 (unpatched, UP machine),
and 2001:968:1::2 is the icecast server running Linux 2.4.20-rc2-ac3 (SMP).
If you want to test the stream yourself, please stream from
http://ipv6.lkml.org:8000/difm .
I have no problem to stream from there. kernel-source-2.4.19 here. several
tunnels in the middle and different brand of routers...
anyway Im farly sure that the xmms patch is not the problem. We have been
testing it for more than 6 months now (for inclusion in debian, we are
not the upstream) and yes we hit a similar problems with one icecast
server but at that time we didn't care too much since it was basically at
the first tests round. I will see if i can get it up and running again
(it is not under my control) and investigate a bit more into it.
Thanks
Fabio
From: Maciej Soltysiak <hidden> Date: 2003-01-08 16:30:55
Previously Wichert Akkerman wrote:
quoted
traceroute to ipv6.lkml.org (2001:968:1::2) from
3ffe:8280:10:1d0:290:27ff:fe2d:968c, 30 hops max, 16 byte packets
1 thunder.wiggy.net (3ffe:8280:10:1d0:250:4ff:fe0b:dd79) 0.666 ms 0.22 ms 0.199 ms
2 xs4all-29.ipv6.xs4all.nl (3ffe:8280:0:2001::58) 27.568 ms 28.012 ms 30.177 ms
3 26.ge-0-2-0.xr1.pbw.xs4all.net (2001:888:0:3::1) 22.035 ms 19.528 ms 44.644 ms
4 0.ge-0-3-0.xr1.sara.xs4all.net (2001:888:2:1::1) 19.519 ms 19.002 ms 21.974 ms
5 fe-0-0-0.ams-core-01.network6.isp-services.nl (2001:7f8:1::a502:4875:1) 19.978 ms 30.278 ms 20.248 ms
6 2001:968::2 (2001:968::2) 24.246 ms 24.083 ms 22.918 ms
7 2001:968:1::2 (2001:968:1::2) 24.978 ms 23.866 ms 23.661 ms
thunder.wiggy.net is my Linux router running 2.4.19-pre5-ac3-freeswan196
currently. The second hop is a normal sit tunnel and all the other
hops are native ipv6 using Cisco and Juniper routers as far as I know.
Slight correction to that: xs4all-29.ipv6.xs4all.nl is a FreeBSD-4.5
tunnel server. The toher xs4all.net hops are Junipers running JunOS 5.3
or 5.5.
I had four contiguous listenings:
3 mins
10mins
19mins
13mins
When i increased the buffer in xmms i got better uninterrupted timings.
And survived data gaps better.
I seem to be getting better results than you, i think that it is not an
issue of ipv6 implementation but simply the case of time sensitive
traffic fighting with other Internet traffic over tunnels through ipv4
networks.
I do not know how many tunnels are in my path, i know that hop distance to
my tunnel is exactly 1 hop (ipv6 broker and ipv4 provider are the same)
If there is immense traffic at one of the routers (total traffic on an
interface) stream packets can be simply dropped if there are no queuing
disciplines that would take eg. flow control into account.
What do you think?
btw. what the hell is JunOs ?
Regards,
Maciej Soltysiak
I do not know how many tunnels are in my path, i know that hop distance to
my tunnel is exactly 1 hop (ipv6 broker and ipv4 provider are the same)
My tunnel provider is 5 hops away. To my knowledge non of the ipv4 or
ipv6 hops in the path are congested and no traffic shaping is done.
If there is immense traffic at one of the routers (total traffic on an
interface) stream packets can be simply dropped if there are no queuing
disciplines that would take eg. flow control into account.
I'll ask the ISPs involved to check if this might be happening, but I
highly doubt it.
btw. what the hell is JunOs ?
Juniper OS, running on Juniper routers.
Wichert.
--
Wichert Akkerman [off-list ref] http://www.wiggy.net/
A random hacker
I seem to be getting better results than you, i think that it is not an
issue of ipv6 implementation but simply the case of time sensitive
traffic fighting with other Internet traffic over tunnels through ipv4
networks.
Actually, I don't follow this. How could any kind of traffic shaping
result in my client not sending ACKs, which is what the tcpdump
seems to indicate? I can understand packets being dropped which
would result in retransmits, but that is not the case here.
Wichert.
(usual I'm-no-network-guru-and-might-be-misreading-things disclaimer here)
--
Wichert Akkerman [off-list ref] http://www.wiggy.net/
A random hacker
From: Maciej Soltysiak <hidden> Date: 2003-01-08 17:35:07
Actually, I don't follow this. How could any kind of traffic shaping
result in my client not sending ACKs, which is what the tcpdump
seems to indicate?
Well, i think i made a mistake, writing about routers dropping the
packets, it's not the case here, you are right.
Maciej
From: Fabio Massimo Di Nitto <hidden> Date: 2003-01-08 17:56:57
I was able to reproduce the problem again. I have been using ethereal to
sniff instead of tcpdump and gave out some more info.
basically the icecast server at certain time (but i can't predict
exactly in which situations) just send a FIN, ACK packet to the client.
Basically to close the connection and after a few packets the client of
course answer.What is strange that in the meanwhile there are still 3/4
data packets coming from the server to the client.
Regarding the network side I noticed the following:
an average of 500ms to ping6 the server and 0 pkt loss
few seconds before the FIN, ACK (server->client) and for about 6 pkts the
average jumped to 2000ms
I suspect that this network flap made the server thinking about
<insert_here_whatever_term_is_more_appropriate> and decided to close
the connection.
The full ethereal dump is available at
http://www.fabbione.net/ice-xmms-ipv6.dump.bz2
but PLEASE note that it is a 10MB file and Im on a slow adsl line so be
"nice".
Fabio
PS Im afraid/happy that anyway the problem is not related to the kernel
version we are running.
--
vega:~# apt-get install life
Reading Package Lists... Done
Building Dependency Tree... Done
E: Couldn't find package life
From: Andrew McGregor <hidden> Date: 2003-01-08 19:44:00
Selective ACK is mandatory in IPv6 and uses a somewhat different algorithm, so you shouldn't be seeing nearly as many ACKs as an IPv4 client would do by default.
Andrew
--On Wednesday, January 08, 2003 18:01:39 +0100 Wichert Akkerman [off-list ref] wrote:
Previously Maciej Soltysiak wrote:
quoted
I seem to be getting better results than you, i think that it is not an
issue of ipv6 implementation but simply the case of time sensitive
traffic fighting with other Internet traffic over tunnels through ipv4
networks.
Actually, I don't follow this. How could any kind of traffic shaping
result in my client not sending ACKs, which is what the tcpdump
seems to indicate? I can understand packets being dropped which
would result in retransmits, but that is not the case here.
Wichert.
(usual I'm-no-network-guru-and-might-be-misreading-things disclaimer here)
--
Wichert Akkerman [off-list ref] http://www.wiggy.net/
A random hacker
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: Andrew McGregor <hidden> Date: 2003-01-08 19:51:40
--On Wednesday, January 08, 2003 19:05:36 +0100 Fabio Massimo Di Nitto [off-list ref] wrote:
I was able to reproduce the problem again. I have been using ethereal to
sniff instead of tcpdump and gave out some more info.
basically the icecast server at certain time (but i can't predict
exactly in which situations) just send a FIN, ACK packet to the client.
Basically to close the connection and after a few packets the client of
course answer.What is strange that in the meanwhile there are still 3/4
data packets coming from the server to the client.
Regarding the network side I noticed the following:
an average of 500ms to ping6 the server and 0 pkt loss
few seconds before the FIN, ACK (server->client) and for about 6 pkts the
average jumped to 2000ms
I suspect that this network flap made the server thinking about
<insert_here_whatever_term_is_more_appropriate> and decided to close
the connection.
Probably on the server's side it got an ICMP Host Unreachable or two as some router updated its tables, and decided to close the connection. The FIN jumped the queue in one/several of the routers in the path, so it got reordered relative to the data. This would imply that the router in question had its route to you back by the time the FIN got there.
Wierd, but far from impossible.
From: Fabio Massimo Di Nitto <hidden> Date: 2003-01-08 20:22:26
Definitly it is somehow unstable. Difficult to find the reason.
Anyway Im sure that there is asimmetric routing between me and
ipv6.lkml.org and I could see pkts coming back. It might not have been the
case for Wichert
Fabio
On Wed, 8 Jan 2003, Maciej Soltysiak wrote:
quoted
Probably on the server's side it got an ICMP Host Unreachable or two as
some router updated its tables, and decided to close the connection.
Sounds reasonable to me. Could it mean that this router that we are
talking about is simply slow or overloaded ?
Maciej
From: Andrew McGregor <hidden> Date: 2003-01-08 20:31:01
Probably not slow and/or overloaded. I'd think it's more likely that it has a routing update problem or an unreliable link. But whatever, this seemed to me to be a classic 'dodgy box in the middle' rather than an end host problem.
Andrew
--On Wednesday, January 08, 2003 21:31:04 +0100 Fabio Massimo Di Nitto [off-list ref] wrote:
Definitly it is somehow unstable. Difficult to find the reason.
Anyway Im sure that there is asimmetric routing between me and
ipv6.lkml.org and I could see pkts coming back. It might not have been the
case for Wichert
Fabio
On Wed, 8 Jan 2003, Maciej Soltysiak wrote:
quoted
quoted
Probably on the server's side it got an ICMP Host Unreachable or two as
some router updated its tables, and decided to close the connection.
Sounds reasonable to me. Could it mean that this router that we are
talking about is simply slow or overloaded ?
Maciej
Probably on the server's side it got an ICMP Host Unreachable or two as
some router updated its tables, and decided to close the connection.
The fact that this problem does not seem to occur when using a window
XP client seems to contradict the suggestions that it may be a router
problem.
Wichert.
--
Wichert Akkerman [off-list ref] http://www.wiggy.net/
A random hacker
From: Fabio Massimo Di Nitto <hidden> Date: 2003-01-09 07:20:41
On Wed, 8 Jan 2003, Wichert Akkerman wrote:
Previously Andrew McGregor wrote:
quoted
Probably on the server's side it got an ICMP Host Unreachable or two as
some router updated its tables, and decided to close the connection.
The fact that this problem does not seem to occur when using a window
XP client seems to contradict the suggestions that it may be a router
problem.
Wichert.
Is the WinXP client located in the same place where you are?
From my side the ISP that is giving me problems is xs26.net
at 2 different points. One is flapping and one is the link between them
and another ISP (i can't even reach it now) where pkts get seriously
delayed (from 100ms to more than 350ms) probably due to a slow link.
But what is seriously annoying is that xs26.net keeps announcing a network
that it can't reach, fscking the entire routing.
Fabio
Is the WinXP client located in the same place where you are?
Yes. (Well, not my place but a friend who had both linux and XP did
that test).
Wichert.
--
Wichert Akkerman [off-list ref] http://www.wiggy.net/
A random hacker
From: Fabio Massimo Di Nitto <hidden> Date: 2003-01-09 09:34:29
On Thu, 9 Jan 2003, Wichert Akkerman wrote:
Previously Fabio Massimo Di Nitto wrote:
quoted
Is the WinXP client located in the same place where you are?
Yes. (Well, not my place but a friend who had both linux and XP did
that test).
Wichert.
hmmmm strange because now I have forced the route to ipv6.lkml.org via
another ISP (bypassing xs26.net) and it is more than 3 hours that xmms is
streaming without interruptions.
Fabio
From: Maciej Soltysiak <hidden> Date: 2003-01-09 10:24:08
The fact that this problem does not seem to occur when using a window
XP client seems to contradict the suggestions that it may be a router
problem.
What other linux clients support streaming on ip6 ? patched mpg123 maybe?
What XP client are you using ?
Maybe it is a client issue, you say the client stops sending ACKs, maybe
the client code is buggy.
Another packet, but this time it SACKs the just-recieved packet. It looks
as if the two packets inbetween somehow were not recognized as belonging
with this connection.
again, again, again.
It looks as if somehow those two packets 9359225:9360433 and 9360433:9360653 get
mangled in a way as to invalidate the checksum. This would cause "silent drop"
of these packets before they were acked....
Can you check the stats counters, to see if they are indeed dropped?
Roger.
--
** R.E.Wolff@BitWizard.nl ** http://www.BitWizard.nl/ ** +31-15-2600998 **
*-- BitWizard writes Linux device drivers for any device you may have! --*
* The Worlds Ecosystem is a stable system. Stable systems may experience *
* excursions from the stable situation. We are currently in such an *
* excursion: The stable situation does not include humans. ***************
Another packet, but this time it SACKs the just-recieved packet. It looks
as if the two packets inbetween somehow were not recognized as belonging
with this connection.
or SACK forgot about them?
Two more packets, and still more hints towards the other machine that
we're missing 9359225-9360653
So, it retransmits the first. but we don't see it as beloging to
this connection or something, so it gets ignored.
or we're waiting for the other one to ACK them both in one go?
It looks as if somehow those two packets 9359225:9360433 and
9360433:9360653 get mangled in a way as to invalidate the checksum. This
would cause "silent drop" of these packets before they were acked....
Could be data dependant, so there's a pattern in the packet contents that causes this?
Can you check the stats counters, to see if they are indeed dropped?
Roger.
Can you check the stats counters, to see if they are indeed dropped?
If you can tell me to which counter I am looking for and where I can
find that, sure.
Wichert.
--
Wichert Akkerman [off-list ref] http://www.wiggy.net/
A random hacker
From: Mika Liljeberg <hidden> Date: 2003-01-09 22:03:50
Hi Wichert,
Looking at your trace it seems that the receiving machine is dropping
all packets that do not have traffic class set. Note that all segments
received with [class 0x2] get properly acked. The others probably don't
get to TCP at all. You might want to check your filters and QoS
policies.
BR,
MikaL
On Wed, 2003-01-08 at 19:01, Wichert Akkerman wrote:
Previously Maciej Soltysiak wrote:
quoted
I seem to be getting better results than you, i think that it is not an
issue of ipv6 implementation but simply the case of time sensitive
traffic fighting with other Internet traffic over tunnels through ipv4
networks.
Actually, I don't follow this. How could any kind of traffic shaping
result in my client not sending ACKs, which is what the tcpdump
seems to indicate? I can understand packets being dropped which
would result in retransmits, but that is not the case here.
Wichert.
(usual I'm-no-network-guru-and-might-be-misreading-things disclaimer here)
Looking at your trace it seems that the receiving machine is dropping
all packets that do not have traffic class set. Note that all segments
received with [class 0x2] get properly acked. The others probably don't
get to TCP at all. You might want to check your filters and QoS
policies.
I don't have any filters and QoS support isn't even enabled in this
kernel.
Wichert.
--
Wichert Akkerman [off-list ref] http://www.wiggy.net/
A random hacker
From: Bill Davidsen <hidden> Date: 2003-01-09 22:44:04
On Thu, 9 Jan 2003, Andrew McGregor wrote:
Probably on the server's side it got an ICMP Host Unreachable or two as
some router updated its tables, and decided to close the connection. The
FIN jumped the queue in one/several of the routers in the path, so it got
reordered relative to the data. This would imply that the router in
question had its route to you back by the time the FIN got there.
Wierd, but far from impossible.
Nothing is impossible with routers, including sending ICMP packets using
different routing than TCP (and doing odd things with UDP as well).
Sometimes ICMP will be sent over a low bandwidth link with hopefully less
latency. The phrase "the less traveled way" comes to mind.
--
bill davidsen [off-list ref]
CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.
From: Paul Jakma <hidden> Date: 2003-01-10 20:51:18
On Thu, 9 Jan 2003, Fabio Massimo Di Nitto wrote:
hmmmm strange because now I have forced the route to ipv6.lkml.org
via another ISP (bypassing xs26.net)
xs26.net posted to the 6bone list not so long that ago that they have
IPv6 routing stability problems. (they're ditching OSPFv3/ospfd and
implementing an inhouse IPv6 link-state IGP instead apparently).
Fabio
regards,
--
Paul Jakma Sys Admin Alphyra
paulj@alphyra.ie
Warning: /never/ send email to spam@dishone.st or trap@dishone.st