Re: WARNING in __static_key_slow_dec

From
Willem de Bruijn <willemdebruijn.kernel@gmail.com>
Date
2018-05-18 20:31:29
Also in
lkml
Reply to
DaeRyong Jeong, 2018-05-18
On Fri, May 18, 2018 at 4:03 AM, DaeRyong Jeong [off-list ref] wrote:
We report the crash: WARNING in __static_key_slow_dec

This crash has been found in v4.8 using RaceFuzzer (a modified
version of Syzkaller), which we describe more at the end of this
report.
Even though v4.8 is the relatively old version, we did manual verification
and we think the bug still exists.
Our analysis shows that the race occurs when invoking two syscalls
concurrently, setsockopt() with optname SO_TIMESTAMPING and ioctl() with
cmd SIOCGSTAMPNS.


Diagnosis:
We think if timestamp was previously enabled with
SOCK_TIMESTAMPING_RX_SOFTWARE flag, the concurrent execution of
sock_disable_timestamp() and sock_enable_timestamp() causes the crash.


Thread interleaving:
(Assume sk->flag has the SOCK_TIMESTAMPING_RX_SOFTWARE flag by the
previous setsockopt() call with SO_TIMESTAMPING)

CPU0 (sock_disable_timestamp())                 CPU1 (sock_enable_timestamp())
=====                                           =====
(flag == 1UL << SOCK_TIMESTAMPING_RX_SOFTWARE)  (flag == SOCK_TIMESTAMP)

                                                if (!sock_flag(sk, flag)) {
                                                        unsigned long previous_flags = sk->sk_flags;

if (sk->sk_flags & flags) {
        sk->sk_flags &= ~flags;
        if (sock_needs_netstamp(sk) &&
            !(sk->sk_flags & SK_FLAGS_TIMESTAMP))
                net_disable_timestamp();
                                                        sock_set_flag(sk, flag);

                                                        if (sock_needs_netstamp(sk) &&
                                                            !(previous_flags & SK_FLAGS_TIMESTAMP))
                                                                net_enable_timestamp();
                                                        /* Here, net_enable_timestamp() is not called because
                                                         * previous_flags has the SOCK_TIMESTAMPING_RX_SOFTWARE
                                                         * flag
                                                         */
/* After the race, sk->sk has the flag SOCK_TIMESTAMP, but
 * net_enable_timestamp() is not called one more time.
 * Consequently, when the socket is closed, __sk_destruct()
 * calls net_disable_timestamp() that leads WARNING.
 */
Thanks for the detailed analysis.

Indeed the updates to sk->sk_flags and calls to net_(dis|en)able_timestamp
should happen atomically, but this is not the case. The setsockopt
path holds the socket lock, but not all ioctl paths.

Perhaps we can take lock_sock_fast in sock_get_timestamp and
variants.
Thread (4 messages) 4 messages, 2 authors, last reply 2018-05-20
Keyboard shortcuts
hback out one level
jnext message in thread / next series-diff position
kprevious message in thread / previous series-diff position
ldrill in
Escclose help / fold thread tree
?toggle this help