This patchset impelements support of SOCK_SEQPACKET for virtio
transport.
As SOCK_SEQPACKET guarantees to save record boundaries, so to
do it, new packet operation was added: it marks start of record (with
record length in header), such packet doesn't carry any data. To send
record, packet with start marker is sent first, then all data is sent
as usual 'RW' packets. On receiver's side, length of record is known
from packet with start record marker. Now as packets of one socket
are not reordered neither on vsock nor on vhost transport layers, such
marker allows to restore original record on receiver's side. If user's
buffer is smaller that record length, when all out of size data is
dropped.
Maximum length of datagram is not limited as in stream socket,
because same credit logic is used. Difference with stream socket is
that user is not woken up until whole record is received or error
occurred. Implementation also supports 'MSG_EOR' and 'MSG_TRUNC' flags.
Tests also implemented.
Arseny Krasnov (13):
af_vsock: prepare for SOCK_SEQPACKET support
af_vsock: prepare 'vsock_connectible_recvmsg()'
af_vsock: implement SEQPACKET rx loop
af_vsock: implement send logic for SOCK_SEQPACKET
af_vsock: rest of SEQPACKET support
af_vsock: update comments for stream sockets
virtio/vsock: dequeue callback for SOCK_SEQPACKET
virtio/vsock: fetch length for SEQPACKET record
virtio/vsock: add SEQPACKET receive logic
virtio/vsock: rest of SOCK_SEQPACKET support
virtio/vsock: setup SEQPACKET ops for transport
vhost/vsock: setup SEQPACKET ops for transport
vsock_test: add SOCK_SEQPACKET tests
drivers/vhost/vsock.c | 7 +-
include/linux/virtio_vsock.h | 12 +
include/net/af_vsock.h | 6 +
include/uapi/linux/virtio_vsock.h | 9 +
net/vmw_vsock/af_vsock.c | 543 ++++++++++++++++------
net/vmw_vsock/virtio_transport.c | 4 +
net/vmw_vsock/virtio_transport_common.c | 295 ++++++++++--
tools/testing/vsock/util.c | 32 +-
tools/testing/vsock/util.h | 3 +
tools/testing/vsock/vsock_test.c | 126 +++++
10 files changed, 862 insertions(+), 175 deletions(-)
TODO:
- Support for record integrity control. As transport could drop some
packets, something like "record-id" and record end marker need to
be implemented. Idea is that SEQ_BEGIN packet carries both record
length and record id, end marker(let it be SEQ_END) carries only
record id. To be sure that no one packet was lost, receiver checks
length of data between SEQ_BEGIN and SEQ_END(it must be same with
value in SEQ_BEGIN) and record ids of SEQ_BEGIN and SEQ_END(this
means that both markers were not dropped. I think that easiest way
to implement record id for SEQ_BEGIN is to reuse another field of
packet header(SEQ_BEGIN already uses 'flags' as record length).For
SEQ_END record id could be stored in 'flags'.
Another way to implement it, is to move metadata of both SEQ_END
and SEQ_BEGIN to payload. But this approach has problem, because
if we move something to payload, such payload is accounted by
credit logic, which fragments payload, while payload with record
length and id couldn't be fragmented. One way to overcome it is to
ignore credit update for SEQ_BEGIN/SEQ_END packet.Another solution
is to update 'stream_has_space()' function: current implementation
return non-zero when at least 1 byte is allowed to use,but updated
version will have extra argument, which is needed length. For 'RW'
packet this argument is 1, for SEQ_BEGIN it is sizeof(record len +
record id) and for SEQ_END it is sizeof(record id).
- What to do, when server doesn't support SOCK_SEQPACKET. In current
implementation RST is replied in the same way when listening port
is not found. I think that current RST is enough,because case when
server doesn't support SEQ_PACKET is same when listener missed(e.g.
no listener in both cases).
v2 -> v3:
- patches reorganized: split for prepare and implementation patches
- local variables are declared in "Reverse Christmas tree" manner
- virtio_transport_common.c: valid leXX_to_cpu() for vsock header
fields access
- af_vsock.c: 'vsock_connectible_*sockopt()' added as shared code
between stream and seqpacket sockets.
- af_vsock.c: loops in '__vsock_*_recvmsg()' refactored.
- af_vsock.c: 'vsock_wait_data()' refactored.
v1 -> v2:
- patches reordered: af_vsock.c related changes now before virtio vsock
- patches reorganized: more small patches, where +/- are not mixed
- tests for SOCK_SEQPACKET added
- all commit messages updated
- af_vsock.c: 'vsock_pre_recv_check()' inlined to
'vsock_connectible_recvmsg()'
- af_vsock.c: 'vsock_assign_transport()' returns ENODEV if transport
was not found
- virtio_transport_common.c: transport callback for seqpacket dequeue
- virtio_transport_common.c: simplified
'virtio_transport_recv_connected()'
- virtio_transport_common.c: send reset on socket and packet type
mismatch.
Signed-off-by: Arseny Krasnov <redacted>
--
2.25.1
This adds some logic to current stream enqueue function for SEQPACKET
support:
1) Send record begin marker with length of record.
2) Return value from enqueue function is wholevrecord length or error
for SOCK_SEQPACKET.
Signed-off-by: Arseny Krasnov <redacted>
---
include/net/af_vsock.h | 1 +
net/vmw_vsock/af_vsock.c | 16 ++++++++++++++--
2 files changed, 15 insertions(+), 2 deletions(-)
@@ -1845,8 +1851,14 @@ static int vsock_connectible_sendmsg(struct socket *sock, struct msghdr *msg,}out_err:-if(total_written>0)-err=total_written;+if(total_written>0){+/* Return number of written bytes only if:+*1)SOCK_STREAMsocket.+*2)SOCK_SEQPACKETsocketwhenwholebufferissent.+*/+if(sk->sk_type==SOCK_STREAM||total_written==len)+err=total_written;+}out:release_sock(sk);returnerr;
This replaces 'stream' to 'connect oriented' in comments as SEQPACKET is
also connect oriented.
Signed-off-by: Arseny Krasnov <redacted>
---
net/vmw_vsock/af_vsock.c | 31 +++++++++++++++++--------------
1 file changed, 17 insertions(+), 14 deletions(-)
@@ -415,8 +415,8 @@ static void vsock_deassign_transport(struct vsock_sock *vsk)/* Assign a transport to a socket and call the .init transport callback.*-*Note:forstreamsocketthismustbecalledwhenvsk->remote_addrisset-*(e.g.duringtheconnect()orwhenaconnectionrequestonalistener+*Note:forconnectorientedsocketthismustbecalledwhenvsk->remote_addr+*isset(e.g.duringtheconnect()orwhenaconnectionrequestonalistener*socketisreceived).*Thevsk->remote_addrisusedtodecidewhichtransporttouse:*-remoteCID==VMADDR_CID_LOCALorg2h->local_cidorVMADDR_CID_HOSTif
@@ -477,10 +477,10 @@ int vsock_assign_transport(struct vsock_sock *vsk, struct vsock_sock *psk)return0;/* transport->release() must be called with sock lock acquired.-*Thispathcanonlybetakenduringvsock_stream_connect(),-*wherewehavealreadyheldthesocklock.-*Intheothercases,thisfunctioniscalledonanewsocket-*whichisnotassignedtoanytransport.+*Thispathcanonlybetakenduringvsock_connect(),wherewe+*havealreadyheldthesocklock.Intheothercases,this+*functioniscalledonanewsocketwhichisnotassignedto+*anytransport.*/vsk->transport->release(vsk);vsock_deassign_transport(vsk);
@@ -657,9 +657,10 @@ static int __vsock_bind_connectible(struct vsock_sock *vsk,vsock_addr_init(&vsk->local_addr,new_addr.svm_cid,new_addr.svm_port);-/* Remove stream sockets from the unbound list and add them to the hash-*tableforeasylookupbyitsaddress.Theunboundlistissimplyan-*extraentryattheendofthehashtable,atrickusedbyAF_UNIX.+/* Remove connect oriented sockets from the unbound list and add them+*tothehashtableforeasylookupbyitsaddress.Theunboundlist+*issimplyanextraentryattheendofthehashtable,atrickused+*byAF_UNIX.*/__vsock_remove_bound(vsk);__vsock_insert_bound(vsock_bound_sockets(&vsk->local_addr),vsk);
@@ -950,10 +951,10 @@ static int vsock_shutdown(struct socket *sock, int mode)if((mode&~SHUTDOWN_MASK)||!mode)return-EINVAL;-/* If this is a STREAM socket and it is not connected then bail out-*immediately.IfitisaDGRAMsocketthenwemustfirstkickthe-*socketsothatitwakesupfromanysleepingcalls,forexample-*recv(),andthenafterwardsreturntheerror.+/* If this is a connect oriented socket and it is not connected then+*bailoutimmediately.IfitisaDGRAMsocketthenwemustfirst+*kickthesocketsothatitwakesupfromanysleepingcalls,for+*examplerecv(),andthenafterwardsreturntheerror.*/sk=sock->sk;
@@ -1770,7 +1771,9 @@ static int vsock_connectible_sendmsg(struct socket *sock, struct msghdr *msg,lock_sock(sk);-/* Callers should not provide a destination with stream sockets. */+/* Callers should not provide a destination with connect oriented+*sockets.+*/if(msg->msg_namelen){err=sk->sk_state==TCP_ESTABLISHED?-EISCONN:-EOPNOTSUPP;gotoout;
This adds receive loop for SEQPACKET. It looks like receive loop for
SEQPACKET, but there is a little bit difference:
1) It doesn't call notify callbacks.
2) It doesn't care about 'SO_SNDLOWAT' and 'SO_RCVLOWAT' values, because
there is no sense for these values in SEQPACKET case.
3) It waits until whole record is received or error is found during
receiving.
4) It processes and sets 'MSG_TRUNC' flag.
So to avoid extra conditions for two types of socket inside one loop, two
independent functions were created.
Signed-off-by: Arseny Krasnov <redacted>
---
include/net/af_vsock.h | 5 ++
net/vmw_vsock/af_vsock.c | 102 ++++++++++++++++++++++++++++++++++++++-
2 files changed, 106 insertions(+), 1 deletion(-)
@@ -2006,7 +2006,107 @@ static int __vsock_stream_recvmsg(struct sock *sk, struct msghdr *msg,staticint__vsock_seqpacket_recvmsg(structsock*sk,structmsghdr*msg,size_tlen,intflags){-return-1;+conststructvsock_transport*transport;+conststructiovec*orig_iov;+unsignedlongorig_nr_segs;+ssize_tdequeued_total=0;+structvsock_sock*vsk;+size_trecord_len;+longtimeout;+interr=0;+DEFINE_WAIT(wait);++vsk=vsock_sk(sk);+transport=vsk->transport;++timeout=sock_rcvtimeo(sk,flags&MSG_DONTWAIT);+msg->msg_flags&=~MSG_EOR;+orig_nr_segs=msg->msg_iter.nr_segs;+orig_iov=msg->msg_iter.iov;++while(1){+ssize_tdequeued;+s64ready;++prepare_to_wait(sk_sleep(sk),&wait,TASK_INTERRUPTIBLE);+ready=vsock_stream_has_data(vsk);++if(ready==0){+if(vsock_wait_data(sk,&wait,timeout,NULL,0)){+/* In case of any loop break(timeout, signal+*interruptorshutdown),wereportuserthat+*nothingwascopied.+*/+dequeued_total=0;+break;+}+continue;+}++finish_wait(sk_sleep(sk),&wait);++if(ready<0){+err=-ENOMEM;+gotoout;+}++if(dequeued_total==0){+record_len=+transport->seqpacket_seq_get_len(vsk);++if(record_len==0)+continue;+}++/* 'msg_iter.count' is number of unused bytes in iov.+*Oneverycopytoioviteratoritisdecrementedat+*sizeofdata.+*/+dequeued=transport->seqpacket_dequeue(vsk,msg,+msg->msg_iter.count,flags);++if(dequeued<0){+dequeued_total=0;++if(dequeued==-EAGAIN){+iov_iter_init(&msg->msg_iter,READ,+orig_iov,orig_nr_segs,+len);+msg->msg_flags&=~MSG_EOR;+continue;+}++err=-ENOMEM;+break;+}++dequeued_total+=dequeued;++if(dequeued_total>=record_len)+break;+}+if(sk->sk_err)+err=-sk->sk_err;+elseif(sk->sk_shutdown&RCV_SHUTDOWN)+err=0;++if(dequeued_total>0){+/* User sets MSG_TRUNC, so return real length of+*packet.+*/+if(flags&MSG_TRUNC)+err=record_len;+else+err=len-msg->msg_iter.count;++/* Always set MSG_TRUNC if real length of packet is+*biggerthatuserbuffer.+*/+if(record_len>len)+msg->msg_flags|=MSG_TRUNC;+}+out:+returnerr;}staticint
This adds rest of logic for SEQPACKET:
1) Shared functions for packet sending now set valid type of packet
according socket type.
2) SEQPACKET specific function like SEQ_BEGIN send and data dequeue.
3) TAP support for SEQPACKET is not so easy if it is necessary to
send whole record to TAP interface. This could be done by allocating
new packet when whole record is received, data of record must be
copied to TAP packet.
Signed-off-by: Arseny Krasnov <redacted>
---
include/linux/virtio_vsock.h | 7 ++++
net/vmw_vsock/virtio_transport_common.c | 55 +++++++++++++++++++++----
2 files changed, 55 insertions(+), 7 deletions(-)
@@ -793,10 +824,11 @@ int virtio_transport_connect(struct vsock_sock *vsk){structvirtio_vsock_pkt_infoinfo={.op=VIRTIO_VSOCK_OP_REQUEST,-.type=VIRTIO_VSOCK_TYPE_STREAM,.vsk=vsk,};+info.type=virtio_transport_get_type(sk_vsock(vsk));+returnvirtio_transport_send_pkt_info(vsk,&info);}EXPORT_SYMBOL_GPL(virtio_transport_connect);
@@ -805,7 +837,6 @@ int virtio_transport_shutdown(struct vsock_sock *vsk, int mode){structvirtio_vsock_pkt_infoinfo={.op=VIRTIO_VSOCK_OP_SHUTDOWN,-.type=VIRTIO_VSOCK_TYPE_STREAM,.flags=(mode&RCV_SHUTDOWN?VIRTIO_VSOCK_SHUTDOWN_RCV:0)|(mode&SEND_SHUTDOWN?
@@ -813,6 +844,8 @@ int virtio_transport_shutdown(struct vsock_sock *vsk, int mode).vsk=vsk,};+info.type=virtio_transport_get_type(sk_vsock(vsk));+returnvirtio_transport_send_pkt_info(vsk,&info);}EXPORT_SYMBOL_GPL(virtio_transport_shutdown);
This adds transport callback which tries to fetch record begin marker
from socket's rx queue. It is called from af_vsock.c before reading data
packets of record.
Signed-off-by: Arseny Krasnov <redacted>
---
include/linux/virtio_vsock.h | 1 +
net/vmw_vsock/virtio_transport_common.c | 33 +++++++++++++++++++++++++
2 files changed, 34 insertions(+)
This prepares 'vsock_connectible_recvmg()' to call SEQPACKET receive
loop:
1) Some shared check left in this function, then socket type
specific receive loop is called.
2) Stream receive loop is moved to separate function.
Signed-off-by: Arseny Krasnov <redacted>
---
net/vmw_vsock/af_vsock.c | 242 ++++++++++++++++++++++-----------------
1 file changed, 138 insertions(+), 104 deletions(-)
@@ -1858,65 +1858,69 @@ static int vsock_stream_sendmsg(struct socket *sock, struct msghdr *msg,returnvsock_connectible_sendmsg(sock,msg,len);}--staticint-vsock_connectible_recvmsg(structsocket*sock,structmsghdr*msg,size_tlen,-intflags)+staticintvsock_wait_data(structsock*sk,structwait_queue_entry*wait,+longtimeout,+structvsock_transport_recv_notify_data*recv_data,+size_ttarget){-structsock*sk;+interr=0;structvsock_sock*vsk;conststructvsock_transport*transport;-interr;-size_ttarget;-ssize_tcopied;-longtimeout;-structvsock_transport_recv_notify_datarecv_data;--DEFINE_WAIT(wait);-sk=sock->sk;vsk=vsock_sk(sk);transport=vsk->transport;-err=0;--lock_sock(sk);--if(!transport||sk->sk_state!=TCP_ESTABLISHED){-/* Recvmsg is supposed to return 0 if a peer performs an-*orderlyshutdown.Differentiatebetweenthatcaseandwhena-*peerhasnotconnectedoralocalshutdownoccuredwiththe-*SOCK_DONEflag.-*/-if(sock_flag(sk,SOCK_DONE))-err=0;-else-err=-ENOTCONN;+if(sk->sk_err!=0||+(sk->sk_shutdown&RCV_SHUTDOWN)||+(vsk->peer_shutdown&SEND_SHUTDOWN)){+err=-1;gotoout;}--if(flags&MSG_OOB){-err=-EOPNOTSUPP;+/* Don't wait for non-blocking sockets. */+if(timeout==0){+err=-EAGAIN;gotoout;}-/* We don't check peer_shutdown flag here since peer may actually shut-*down,buttherecanbedatainthequeuethatalocalsocketcan-*receive.-*/-if(sk->sk_shutdown&RCV_SHUTDOWN){-err=0;-gotoout;+if(recv_data){+err=transport->notify_recv_pre_block(vsk,target,recv_data);+if(err<0)+gotoout;}-/* It is valid on Linux to pass in a zero-length receive buffer. This-*isnotanerror.Wemayaswellbailoutnow.-*/-if(!len){-err=0;+release_sock(sk);+timeout=schedule_timeout(timeout);+lock_sock(sk);++if(signal_pending(current)){+err=sock_intr_errno(timeout);+gotoout;+}elseif(timeout==0){+err=-EAGAIN;gotoout;}+out:+finish_wait(sk_sleep(sk),wait);+returnerr;+}++staticint__vsock_stream_recvmsg(structsock*sk,structmsghdr*msg,+size_tlen,intflags)+{+structvsock_transport_recv_notify_datarecv_data;+conststructvsock_transport*transport;+structvsock_sock*vsk;+ssize_tcopied;+size_ttarget;+longtimeout;+interr;++DEFINE_WAIT(wait);++vsk=vsock_sk(sk);+transport=vsk->transport;+/* We must not copy less than target bytes into the user's buffer*beforereturningsuccessfully,sowewaitfortheconsumequeueto*havethatmuchdatatoconsumebeforedequeueing.Notethatthis
@@ -2031,6 +2003,68 @@ vsock_connectible_recvmsg(struct socket *sock, struct msghdr *msg, size_t len,returnerr;}+staticint__vsock_seqpacket_recvmsg(structsock*sk,structmsghdr*msg,+size_tlen,intflags)+{+return-1;+}++staticint+vsock_connectible_recvmsg(structsocket*sock,structmsghdr*msg,size_tlen,+intflags)+{+conststructvsock_transport*transport;+structvsock_sock*vsk;+structsock*sk;+interr=0;++sk=sock->sk;++lock_sock(sk);++vsk=vsock_sk(sk);+transport=vsk->transport;++if(!transport||sk->sk_state!=TCP_ESTABLISHED){+/* Recvmsg is supposed to return 0 if a peer performs an+*orderlyshutdown.Differentiatebetweenthatcaseandwhena+*peerhasnotconnectedoralocalshutdownoccurredwiththe+*SOCK_DONEflag.+*/+if(!sock_flag(sk,SOCK_DONE))+err=-ENOTCONN;++gotoout;+}++if(flags&MSG_OOB){+err=-EOPNOTSUPP;+gotoout;+}++/* We don't check peer_shutdown flag here since peer may actually shut+*down,buttherecanbedatainthequeuethatalocalsocketcan+*receive.+*/+if(sk->sk_shutdown&RCV_SHUTDOWN)+gotoout;++/* It is valid on Linux to pass in a zero-length receive buffer. This+*isnotanerror.Wemayaswellbailoutnow.+*/+if(!len)+gotoout;++if(sk->sk_type==SOCK_STREAM)+err=__vsock_stream_recvmsg(sk,msg,len,flags);+else+err=__vsock_seqpacket_recvmsg(sk,msg,len,flags);++out:+release_sock(sk);+returnerr;+}+staticintvsock_stream_recvmsg(structsocket*sock,structmsghdr*msg,size_tlen,intflags)
This also removes ignore of non-stream type of packets.
Signed-off-by: Arseny Krasnov <redacted>
---
drivers/vhost/vsock.c | 7 +++++--
1 file changed, 5 insertions(+), 2 deletions(-)
This prepares af_vsock.c for SEQPACKET support:
1) As both stream and seqpacket sockets are connection oriented, add
check for SOCK_SEQPACKET to conditions where SOCK_STREAM is checked.
2) Some functions such as setsockopt(), getsockopt(), connect(),
recvmsg(), sendmsg() are shared between both types of sockets, so
rename them in general manner and create entry points for each type
of socket to call these functions(for stream in this patch, for
seqpacket in further patches).
Signed-off-by: Arseny Krasnov <redacted>
---
net/vmw_vsock/af_vsock.c | 91 +++++++++++++++++++++++++++++-----------
1 file changed, 67 insertions(+), 24 deletions(-)
This adds two tests of SOCK_SEQPACKET socket: both transfer data and then
test MSG_EOR and MSG_TRUNC flags. Cases for connect(), bind(), etc. are
not tested, because it is same as for stream socket.
Signed-off-by: Arseny Krasnov <redacted>
---
tools/testing/vsock/util.c | 32 ++++++--
tools/testing/vsock/util.h | 3 +
tools/testing/vsock/vsock_test.c | 126 +++++++++++++++++++++++++++++++
3 files changed, 156 insertions(+), 5 deletions(-)
@@ -84,7 +84,7 @@ void vsock_wait_remote_close(int fd)}/* Connect to <cid, port> and return the file descriptor. */-intvsock_stream_connect(unsignedintcid,unsignedintport)+staticintvsock_connect(unsignedintcid,unsignedintport,inttype){union{structsockaddrsa;
@@ -101,7 +101,7 @@ int vsock_stream_connect(unsigned int cid, unsigned int port)control_expectln("LISTENING");-fd=socket(AF_VSOCK,SOCK_STREAM,0);+fd=socket(AF_VSOCK,type,0);timeout_begin(TIMEOUT);do{
@@ -120,11 +120,21 @@ int vsock_stream_connect(unsigned int cid, unsigned int port)returnfd;}+intvsock_stream_connect(unsignedintcid,unsignedintport)+{+returnvsock_connect(cid,port,SOCK_STREAM);+}++intvsock_seqpacket_connect(unsignedintcid,unsignedintport)+{+returnvsock_connect(cid,port,SOCK_SEQPACKET);+}+/* Listen on <cid, port> and return the first incoming connection. The remote*addressisstoredtoclientaddrp.clientaddrpmaybeNULL.*/-intvsock_stream_accept(unsignedintcid,unsignedintport,-structsockaddr_vm*clientaddrp)+staticintvsock_accept(unsignedintcid,unsignedintport,+structsockaddr_vm*clientaddrp,inttype){union{structsockaddrsa;
@@ -145,7 +155,7 @@ int vsock_stream_accept(unsigned int cid, unsigned int port,intclient_fd;intold_errno;-fd=socket(AF_VSOCK,SOCK_STREAM,0);+fd=socket(AF_VSOCK,type,0);if(bind(fd,&addr.sa,sizeof(addr.svm))<0){perror("bind");
@@ -189,6 +199,18 @@ int vsock_stream_accept(unsigned int cid, unsigned int port,returnclient_fd;}+intvsock_stream_accept(unsignedintcid,unsignedintport,+structsockaddr_vm*clientaddrp)+{+returnvsock_accept(cid,port,clientaddrp,SOCK_STREAM);+}++intvsock_seqpacket_accept(unsignedintcid,unsignedintport,+structsockaddr_vm*clientaddrp)+{+returnvsock_accept(cid,port,clientaddrp,SOCK_SEQPACKET);+}+/* Transmit one byte and check the return value.**expected_ret:
Hi Arseny,
thanks for this new series!
I'm a bit busy but I hope to review it tomorrow or on Thursday.
Stefano
On Mon, Jan 25, 2021 at 02:09:00PM +0300, Arseny Krasnov wrote:
This patchset impelements support of SOCK_SEQPACKET for virtio
transport.
As SOCK_SEQPACKET guarantees to save record boundaries, so to
do it, new packet operation was added: it marks start of record (with
record length in header), such packet doesn't carry any data. To send
record, packet with start marker is sent first, then all data is sent
as usual 'RW' packets. On receiver's side, length of record is known
from packet with start record marker. Now as packets of one socket
are not reordered neither on vsock nor on vhost transport layers, such
marker allows to restore original record on receiver's side. If user's
buffer is smaller that record length, when all out of size data is
dropped.
Maximum length of datagram is not limited as in stream socket,
because same credit logic is used. Difference with stream socket is
that user is not woken up until whole record is received or error
occurred. Implementation also supports 'MSG_EOR' and 'MSG_TRUNC' flags.
Tests also implemented.
Arseny Krasnov (13):
af_vsock: prepare for SOCK_SEQPACKET support
af_vsock: prepare 'vsock_connectible_recvmsg()'
af_vsock: implement SEQPACKET rx loop
af_vsock: implement send logic for SOCK_SEQPACKET
af_vsock: rest of SEQPACKET support
af_vsock: update comments for stream sockets
virtio/vsock: dequeue callback for SOCK_SEQPACKET
virtio/vsock: fetch length for SEQPACKET record
virtio/vsock: add SEQPACKET receive logic
virtio/vsock: rest of SOCK_SEQPACKET support
virtio/vsock: setup SEQPACKET ops for transport
vhost/vsock: setup SEQPACKET ops for transport
vsock_test: add SOCK_SEQPACKET tests
drivers/vhost/vsock.c | 7 +-
include/linux/virtio_vsock.h | 12 +
include/net/af_vsock.h | 6 +
include/uapi/linux/virtio_vsock.h | 9 +
net/vmw_vsock/af_vsock.c | 543 ++++++++++++++++------
net/vmw_vsock/virtio_transport.c | 4 +
net/vmw_vsock/virtio_transport_common.c | 295 ++++++++++--
tools/testing/vsock/util.c | 32 +-
tools/testing/vsock/util.h | 3 +
tools/testing/vsock/vsock_test.c | 126 +++++
10 files changed, 862 insertions(+), 175 deletions(-)
TODO:
- Support for record integrity control. As transport could drop some
packets, something like "record-id" and record end marker need to
be implemented. Idea is that SEQ_BEGIN packet carries both record
length and record id, end marker(let it be SEQ_END) carries only
record id. To be sure that no one packet was lost, receiver checks
length of data between SEQ_BEGIN and SEQ_END(it must be same with
value in SEQ_BEGIN) and record ids of SEQ_BEGIN and SEQ_END(this
means that both markers were not dropped. I think that easiest way
to implement record id for SEQ_BEGIN is to reuse another field of
packet header(SEQ_BEGIN already uses 'flags' as record length).For
SEQ_END record id could be stored in 'flags'.
Another way to implement it, is to move metadata of both SEQ_END
and SEQ_BEGIN to payload. But this approach has problem, because
if we move something to payload, such payload is accounted by
credit logic, which fragments payload, while payload with record
length and id couldn't be fragmented. One way to overcome it is to
ignore credit update for SEQ_BEGIN/SEQ_END packet.Another solution
is to update 'stream_has_space()' function: current implementation
return non-zero when at least 1 byte is allowed to use,but updated
version will have extra argument, which is needed length. For 'RW'
packet this argument is 1, for SEQ_BEGIN it is sizeof(record len +
record id) and for SEQ_END it is sizeof(record id).
- What to do, when server doesn't support SOCK_SEQPACKET. In current
implementation RST is replied in the same way when listening port
is not found. I think that current RST is enough,because case when
server doesn't support SEQ_PACKET is same when listener missed(e.g.
no listener in both cases).
v2 -> v3:
- patches reorganized: split for prepare and implementation patches
- local variables are declared in "Reverse Christmas tree" manner
- virtio_transport_common.c: valid leXX_to_cpu() for vsock header
fields access
- af_vsock.c: 'vsock_connectible_*sockopt()' added as shared code
between stream and seqpacket sockets.
- af_vsock.c: loops in '__vsock_*_recvmsg()' refactored.
- af_vsock.c: 'vsock_wait_data()' refactored.
v1 -> v2:
- patches reordered: af_vsock.c related changes now before virtio vsock
- patches reorganized: more small patches, where +/- are not mixed
- tests for SOCK_SEQPACKET added
- all commit messages updated
- af_vsock.c: 'vsock_pre_recv_check()' inlined to
'vsock_connectible_recvmsg()'
- af_vsock.c: 'vsock_assign_transport()' returns ENODEV if transport
was not found
- virtio_transport_common.c: transport callback for seqpacket dequeue
- virtio_transport_common.c: simplified
'virtio_transport_recv_connected()'
- virtio_transport_common.c: send reset on socket and packet type
mismatch.
Signed-off-by: Arseny Krasnov <redacted>
--
2.25.1
This modifies current receive logic for SEQPACKET support:
1) Inserts 'SEQ_BEGIN' packet to socket's rx queue.
2) Inserts 'RW' packet to socket's rx queue, but without merging with
buffer of last packet in queue.
3) Performs check for packet and socket types on receive(if mismatch,
then reset connection).
Signed-off-by: Arseny Krasnov <redacted>
---
net/vmw_vsock/virtio_transport_common.c | 79 ++++++++++++++++++-------
1 file changed, 58 insertions(+), 21 deletions(-)
@@ -1050,39 +1058,49 @@ virtio_transport_recv_enqueue(struct vsock_sock *vsk,structvirtio_vsock_pkt*pkt){structvirtio_vsock_sock*vvs=vsk->trans;-boolcan_enqueue,free_pkt=false;+boolfree_pkt=false;pkt->len=le32_to_cpu(pkt->hdr.len);pkt->off=0;spin_lock_bh(&vvs->rx_lock);-can_enqueue=virtio_transport_inc_rx_pkt(vvs,pkt);-if(!can_enqueue){+if(!virtio_transport_inc_rx_pkt(vvs,pkt)){free_pkt=true;gotoout;}-/* Try to copy small packets into the buffer of last packet queued,-*toavoidwastingmemoryqueueingtheentirebufferwithasmall-*payload.-*/-if(pkt->len<=GOOD_COPY_LEN&&!list_empty(&vvs->rx_queue)){-structvirtio_vsock_pkt*last_pkt;+switch(le16_to_cpu(pkt->hdr.type)){+caseVIRTIO_VSOCK_TYPE_STREAM:{+/* Try to copy small packets into the buffer of last packet queued,+*toavoidwastingmemoryqueueingtheentirebufferwithasmall+*payload.+*/+if(pkt->len<=GOOD_COPY_LEN&&!list_empty(&vvs->rx_queue)){+structvirtio_vsock_pkt*last_pkt;-last_pkt=list_last_entry(&vvs->rx_queue,-structvirtio_vsock_pkt,list);+last_pkt=list_last_entry(&vvs->rx_queue,+structvirtio_vsock_pkt,list);-/* If there is space in the last packet queued, we copy the-*newpacketinitsbuffer.-*/-if(pkt->len<=last_pkt->buf_len-last_pkt->len){-memcpy(last_pkt->buf+last_pkt->len,pkt->buf,-pkt->len);-last_pkt->len+=pkt->len;-free_pkt=true;-gotoout;+/* If there is space in the last packet queued, we copy the+*newpacketinitsbuffer.+*/+if(pkt->len<=last_pkt->buf_len-last_pkt->len){+memcpy(last_pkt->buf+last_pkt->len,pkt->buf,+pkt->len);+last_pkt->len+=pkt->len;+free_pkt=true;+gotoout;+}}++break;+}+caseVIRTIO_VSOCK_TYPE_SEQPACKET:{+break;+}+default:+gotoout;}list_add_tail(&pkt->list,&vvs->rx_queue);
@@ -1247,6 +1273,12 @@ virtio_transport_recv_listen(struct sock *sk, struct virtio_vsock_pkt *pkt,return0;}+staticboolvirtio_transport_valid_type(u16type)+{+return(type==VIRTIO_VSOCK_TYPE_STREAM)||+(type==VIRTIO_VSOCK_TYPE_SEQPACKET);+}+/* We are under the virtio-vsock's vsock->rx_lock or vhost-vsock's vq->mutex*lock.*/
This adds transport callback and it's logic for SEQPACKET dequeue.
Callback fetches RW packets from rx queue of socket until whole record
is copied(if user's buffer is full, user is not woken up). This is done
to not stall sender, because if we wake up user and it leaves syscall,
nobody will send credit update for rest of record, and sender will wait
for next enter of read syscall at receiver's side. So if user buffer is
full, we just send credit update and drop data. If during copy SEQ_BEGIN
was found(and not all data was copied), copying is restarted by reset
user's iov iterator(previous unfinished data is dropped).
Signed-off-by: Arseny Krasnov <redacted>
---
include/linux/virtio_vsock.h | 4 +
include/uapi/linux/virtio_vsock.h | 9 ++
net/vmw_vsock/virtio_transport_common.c | 128 ++++++++++++++++++++++++
3 files changed, 141 insertions(+)
@@ -83,6 +84,9 @@ enum virtio_vsock_op {VIRTIO_VSOCK_OP_CREDIT_UPDATE=6,/* Request the peer to send the credit info to us */VIRTIO_VSOCK_OP_CREDIT_REQUEST=7,++/* Record begin for SOCK_SEQPACKET */+VIRTIO_VSOCK_OP_SEQ_BEGIN=8,};/* VIRTIO_VSOCK_OP_SHUTDOWN flags values */
@@ -91,4 +95,9 @@ enum virtio_vsock_shutdown {VIRTIO_VSOCK_SHUTDOWN_SEND=2,};+/* VIRTIO_VSOCK_OP_RW flags values for SOCK_SEQPACKET type */+enumvirtio_vsock_rw_seqpacket{+VIRTIO_VSOCK_RW_EOR=1,+};+#endif /* _UAPI_LINUX_VIRTIO_VSOCK_H */
@@ -397,6 +397,132 @@ virtio_transport_stream_do_dequeue(struct vsock_sock *vsk,returnerr;}+staticinlinevoidvirtio_transport_del_n_free_pkt(structvirtio_vsock_pkt*pkt)+{+list_del(&pkt->list);+virtio_transport_free_pkt(pkt);+}++staticsize_tvirtio_transport_drop_until_seq_begin(structvirtio_vsock_sock*vvs)+{+structvirtio_vsock_pkt*pkt,*n;+size_tbytes_dropped=0;++list_for_each_entry_safe(pkt,n,&vvs->rx_queue,list){+if(le16_to_cpu(pkt->hdr.op)==VIRTIO_VSOCK_OP_SEQ_BEGIN)+break;++bytes_dropped+=le32_to_cpu(pkt->hdr.len);+virtio_transport_dec_rx_pkt(vvs,pkt);+virtio_transport_del_n_free_pkt(pkt);+}++returnbytes_dropped;+}++staticssize_tvirtio_transport_seqpacket_do_dequeue(structvsock_sock*vsk,+structmsghdr*msg,+size_tuser_buf_len)+{+structvirtio_vsock_sock*vvs=vsk->trans;+structvirtio_vsock_pkt*pkt;+size_tbytes_handled=0;+interr=0;++spin_lock_bh(&vvs->rx_lock);++if(user_buf_len==0){+/* User's buffer is full, we processing rest of+*recordanddropit.If'SEQ_BEGIN'isfound+*whileiterating,userwillbewokenup,+*becauserecordisalreadycopied,andwe+*don'tcareaboutabsentofsometailRWpackets+*ofit.Returnnumberofbytes(restofrecord),+*butignorecreditupdateforsuchabsentbytes.+*/+bytes_handled=virtio_transport_drop_until_seq_begin(vvs);+vvs->user_read_copied+=bytes_handled;++if(!list_empty(&vvs->rx_queue)&&+vvs->user_read_copied<vvs->user_read_seq_len){+/* 'SEQ_BEGIN' found, but record isn't complete.+*Setnumberofcopiedbytestofitrecordsize+*andforcecounterstofinishreceiving.+*/+bytes_handled+=(vvs->user_read_seq_len-vvs->user_read_copied);+vvs->user_read_copied=vvs->user_read_seq_len;+}+}++/* Now start copying. */+while(vvs->user_read_copied<vvs->user_read_seq_len&&+vvs->rx_bytes&&+user_buf_len&&+!err){+pkt=list_first_entry(&vvs->rx_queue,structvirtio_vsock_pkt,list);++switch(le16_to_cpu(pkt->hdr.op)){+caseVIRTIO_VSOCK_OP_SEQ_BEGIN:{+/* Unexpected 'SEQ_BEGIN' during record copy:+*Leavereceiveloop,'EAGAIN'willrestartitfrom+*outerreceiveloop,packetisstillinqueueand+*countersarecleared.Soinnextloopenter,+*'SEQ_BEGIN'willbedequeuedfirst.User'siov+*iteratorwillberesetinouterloop.Also+*sendcreditupdate,becausesomebytescouldbe+*copied.Userwillneverseeunfinishedrecord.+*/+err=-EAGAIN;+break;+}+caseVIRTIO_VSOCK_OP_RW:{+size_tbytes_to_copy;+size_tpkt_len;++pkt_len=(size_t)le32_to_cpu(pkt->hdr.len);+bytes_to_copy=min(user_buf_len,pkt_len);++/* sk_lock is held by caller so no one else can dequeue.+*Unlockrx_locksincememcpy_to_msg()maysleep.+*/+spin_unlock_bh(&vvs->rx_lock);++if(memcpy_to_msg(msg,pkt->buf,bytes_to_copy)){+spin_lock_bh(&vvs->rx_lock);+err=-EINVAL;+break;+}++spin_lock_bh(&vvs->rx_lock);+user_buf_len-=bytes_to_copy;+bytes_handled+=pkt->len;+vvs->user_read_copied+=bytes_to_copy;++if(le32_to_cpu(pkt->hdr.flags)&VIRTIO_VSOCK_RW_EOR)+msg->msg_flags|=MSG_EOR;+break;+}+default:+;+}++/* For unexpected 'SEQ_BEGIN', keep such packet in queue,+*butdropanyothertypeofpacket.+*/+if(le16_to_cpu(pkt->hdr.op)!=VIRTIO_VSOCK_OP_SEQ_BEGIN){+virtio_transport_dec_rx_pkt(vvs,pkt);+virtio_transport_del_n_free_pkt(pkt);+}+}++spin_unlock_bh(&vvs->rx_lock);++virtio_transport_send_credit_update(vsk,VIRTIO_VSOCK_TYPE_SEQPACKET,+NULL);++returnerr?:bytes_handled;+}+ssize_tvirtio_transport_stream_dequeue(structvsock_sock*vsk,structmsghdr*msg,
@@ -481,6 +607,8 @@ int virtio_transport_do_socket_init(struct vsock_sock *vsk,spin_lock_init(&vvs->rx_lock);spin_lock_init(&vvs->tx_lock);INIT_LIST_HEAD(&vvs->rx_queue);+vvs->user_read_copied=0;+vvs->user_read_seq_len=0;return0;}
I think the patch title should be more explicit, so something like
vsock: generalize function to manage connectible sockets
On Mon, Jan 25, 2021 at 02:11:28PM +0300, Arseny Krasnov wrote:
quoted hunk
This prepares af_vsock.c for SEQPACKET support:
1) As both stream and seqpacket sockets are connection oriented, add
check for SOCK_SEQPACKET to conditions where SOCK_STREAM is checked.
2) Some functions such as setsockopt(), getsockopt(), connect(),
recvmsg(), sendmsg() are shared between both types of sockets, so
rename them in general manner and create entry points for each type
of socket to call these functions(for stream in this patch, for
seqpacket in further patches).
Signed-off-by: Arseny Krasnov <redacted>
---
net/vmw_vsock/af_vsock.c | 91 +++++++++++++++++++++++++++++-----------
1 file changed, 67 insertions(+), 24 deletions(-)
I think it's okay to add this function in this patch, but until
SOCK_SEQPACKET is not supported, I would check only SOCK_STREAM and add
SOCK_SEQPACKET only when you add 'vsock_seqpacket_ops' later.
quoted hunk
static void __vsock_release(struct sock *sk, int level)
{
if (sk) {
if (vsk->transport)
vsk->transport->release(vsk);
- else if (sk->sk_type == SOCK_STREAM)
+ else if (sock_type_connectible(sk->sk_type))
vsock_remove_sock(vsk);
sock_orphan(sk);
@@ -945,7 +950,7 @@ static int vsock_shutdown(struct socket *sock, int mode)
sk = sock->sk;
if (sock->state == SS_UNCONNECTED) {
err = -ENOTCONN;
- if (sk->sk_type == SOCK_STREAM)
+ if (sock_type_connectible(sk->sk_type))
return err;
} else {
sock->state = SS_DISCONNECTING;
@@ -960,7 +965,7 @@ static int vsock_shutdown(struct socket *sock, int mode)
sk->sk_state_change(sk);
release_sock(sk);
- if (sk->sk_type == SOCK_STREAM) {
+ if (sock_type_connectible(sk->sk_type)) {
sock_reset_flag(sk, SOCK_DONE);
vsock_send_shutdown(sk, mode);
}
sock_put(sk);
}
-static int vsock_stream_connect(struct socket *sock, struct sockaddr *addr,
- int addr_len, int flags)
+static int vsock_connect(struct socket *sock, struct sockaddr *addr,
+ int addr_len, int flags)
{
int err;
struct sock *sk;
vsk->buffer_size = val;
}
-static int vsock_stream_setsockopt(struct socket *sock,
- int level,
- int optname,
- sockptr_t optval,
- unsigned int optlen)
+static int vsock_connectible_setsockopt(struct socket *sock,
+ int level,
+ int optname,
+ sockptr_t optval,
+ unsigned int optlen)
{
int err;
struct sock *sk;
@@ -1612,10 +1623,20 @@ static int vsock_stream_setsockopt(struct socket *sock,
return err;
}
-static int vsock_stream_getsockopt(struct socket *sock,
- int level, int optname,
- char __user *optval,
- int __user *optlen)
+static int vsock_stream_setsockopt(struct socket *sock,
+ int level,
+ int optname,
+ sockptr_t optval,
+ unsigned int optlen)
+{
+ return vsock_connectible_setsockopt(sock, level, optname, optval,
+ optlen);
+}
As before, I think you can directly use vsock_connectible_setsockopt in
'vsock_stream_ops'.
quoted hunk
+
+static int vsock_connectible_getsockopt(struct socket *sock,
+ int level, int optname,
+ char __user *optval,
+ int __user *optlen)
{
int err;
int len;
@@ -1683,8 +1704,17 @@ static int vsock_stream_getsockopt(struct socket *sock,
return 0;
}
-static int vsock_stream_sendmsg(struct socket *sock, struct msghdr *msg,
- size_t len)
+static int vsock_stream_getsockopt(struct socket *sock,
+ int level, int optname,
+ char __user *optval,
+ int __user *optlen)
+{
+ return vsock_connectible_getsockopt(sock, level, optname, optval,
+ optlen);
+}
+
On Mon, Jan 25, 2021 at 02:11:57PM +0300, Arseny Krasnov wrote:
quoted hunk
This prepares 'vsock_connectible_recvmg()' to call SEQPACKET receive
loop:
1) Some shared check left in this function, then socket type
specific receive loop is called.
2) Stream receive loop is moved to separate function.
Signed-off-by: Arseny Krasnov <redacted>
---
net/vmw_vsock/af_vsock.c | 242 ++++++++++++++++++++++-----------------
1 file changed, 138 insertions(+), 104 deletions(-)
return vsock_connectible_sendmsg(sock, msg, len);
}
-
-static int
-vsock_connectible_recvmsg(struct socket *sock, struct msghdr *msg, size_t len,
- int flags)
+static int vsock_wait_data(struct sock *sk, struct wait_queue_entry *wait,
+ long timeout,
+ struct vsock_transport_recv_notify_data *recv_data,
+ size_t target)
{
- struct sock *sk;
+ int err = 0;
struct vsock_sock *vsk;
const struct vsock_transport *transport;
- int err;
- size_t target;
- ssize_t copied;
- long timeout;
- struct vsock_transport_recv_notify_data recv_data;
-
- DEFINE_WAIT(wait);
- sk = sock->sk;
vsk = vsock_sk(sk);
transport = vsk->transport;
- err = 0;
-
- lock_sock(sk);
-
- if (!transport || sk->sk_state != TCP_ESTABLISHED) {
- /* Recvmsg is supposed to return 0 if a peer performs an
- * orderly shutdown. Differentiate between that case and when a
- * peer has not connected or a local shutdown occured with the
- * SOCK_DONE flag.
- */
- if (sock_flag(sk, SOCK_DONE))
- err = 0;
- else
- err = -ENOTCONN;
+ if (sk->sk_err != 0 ||
+ (sk->sk_shutdown & RCV_SHUTDOWN) ||
+ (vsk->peer_shutdown & SEND_SHUTDOWN)) {
+ err = -1;
goto out;
}
-
- if (flags & MSG_OOB) {
- err = -EOPNOTSUPP;
+ /* Don't wait for non-blocking sockets. */
+ if (timeout == 0) {
+ err = -EAGAIN;
goto out;
}
- /* We don't check peer_shutdown flag here since peer may actually shut
- * down, but there can be data in the queue that a local socket can
- * receive.
- */
- if (sk->sk_shutdown & RCV_SHUTDOWN) {
- err = 0;
- goto out;
+ if (recv_data) {
+ err = transport->notify_recv_pre_block(vsk, target, recv_data);
+ if (err < 0)
+ goto out;
}
- /* It is valid on Linux to pass in a zero-length receive buffer. This
- * is not an error. We may as well bail out now.
- */
- if (!len) {
- err = 0;
+ release_sock(sk);
+ timeout = schedule_timeout(timeout);
+ lock_sock(sk);
+
+ if (signal_pending(current)) {
+ err = sock_intr_errno(timeout);
+ goto out;
+ } else if (timeout == 0) {
+ err = -EAGAIN;
goto out;
}
+out:
+ finish_wait(sk_sleep(sk), wait);
+ return err;
+}
+
+static int __vsock_stream_recvmsg(struct sock *sk, struct msghdr *msg,
+ size_t len, int flags)
+{
+ struct vsock_transport_recv_notify_data recv_data;
+ const struct vsock_transport *transport;
+ struct vsock_sock *vsk;
+ ssize_t copied;
+ size_t target;
+ long timeout;
+ int err;
+
+ DEFINE_WAIT(wait);
+
+ vsk = vsock_sk(sk);
+ transport = vsk->transport;
+
/* We must not copy less than target bytes into the user's buffer
* before returning successfully, so we wait for the consume queue to
* have that much data to consume before dequeueing. Note that this
You can add this function later, when you implement it...
+static int
+vsock_connectible_recvmsg(struct socket *sock, struct msghdr *msg, size_t len,
+ int flags)
+{
+ const struct vsock_transport *transport;
+ struct vsock_sock *vsk;
+ struct sock *sk;
+ int err = 0;
+
+ sk = sock->sk;
+
+ lock_sock(sk);
+
+ vsk = vsock_sk(sk);
+ transport = vsk->transport;
+
+ if (!transport || sk->sk_state != TCP_ESTABLISHED) {
+ /* Recvmsg is supposed to return 0 if a peer performs an
+ * orderly shutdown. Differentiate between that case and when a
+ * peer has not connected or a local shutdown occurred
with the
+ * SOCK_DONE flag.
+ */
+ if (!sock_flag(sk, SOCK_DONE))
+ err = -ENOTCONN;
+
+ goto out;
+ }
+
+ if (flags & MSG_OOB) {
+ err = -EOPNOTSUPP;
+ goto out;
+ }
+
+ /* We don't check peer_shutdown flag here since peer may actually shut
+ * down, but there can be data in the queue that a local socket can
+ * receive.
+ */
+ if (sk->sk_shutdown & RCV_SHUTDOWN)
+ goto out;
+
+ /* It is valid on Linux to pass in a zero-length receive buffer. This
+ * is not an error. We may as well bail out now.
+ */
+ if (!len)
+ goto out;
+
+ if (sk->sk_type == SOCK_STREAM)
+ err = __vsock_stream_recvmsg(sk, msg, len, flags);
+ else
+ err = __vsock_seqpacket_recvmsg(sk, msg, len, flags);
On Mon, Jan 25, 2021 at 02:12:36PM +0300, Arseny Krasnov wrote:
quoted hunk
This adds receive loop for SEQPACKET. It looks like receive loop for
SEQPACKET, but there is a little bit difference:
1) It doesn't call notify callbacks.
2) It doesn't care about 'SO_SNDLOWAT' and 'SO_RCVLOWAT' values, because
there is no sense for these values in SEQPACKET case.
3) It waits until whole record is received or error is found during
receiving.
4) It processes and sets 'MSG_TRUNC' flag.
So to avoid extra conditions for two types of socket inside one loop, two
independent functions were created.
Signed-off-by: Arseny Krasnov <redacted>
---
include/net/af_vsock.h | 5 ++
net/vmw_vsock/af_vsock.c | 102 ++++++++++++++++++++++++++++++++++++++-
2 files changed, 106 insertions(+), 1 deletion(-)
+ if (sk->sk_err)
+ err = -sk->sk_err;
+ else if (sk->sk_shutdown & RCV_SHUTDOWN)
+ err = 0;
+
+ if (dequeued_total > 0) {
+ /* User sets MSG_TRUNC, so return real length of
+ * packet.
+ */
+ if (flags & MSG_TRUNC)
+ err = record_len;
+ else
+ err = len - msg->msg_iter.count;
+
+ /* Always set MSG_TRUNC if real length of packet is
+ * bigger that user buffer.
I'm not sure about ENODEV is the better choice in this case, since the
transport exists, but it doesn't support SOCK_SEQPACKET, so maybe is
better ESOCKTNOSUPPORT.
As I said, I think you don't need to implement all of this helpers and
you can directly assign the vsock_connectible_* functions in the
'vsock_seqpacket_ops'.
Hi Arseny,
I reviewed a part, tomorrow I hope to finish the other patches.
Just a couple of comments in the TODOs below.
On Mon, Jan 25, 2021 at 02:09:00PM +0300, Arseny Krasnov wrote:
This patchset impelements support of SOCK_SEQPACKET for virtio
transport.
As SOCK_SEQPACKET guarantees to save record boundaries, so to
do it, new packet operation was added: it marks start of record (with
record length in header), such packet doesn't carry any data. To send
record, packet with start marker is sent first, then all data is sent
as usual 'RW' packets. On receiver's side, length of record is known
from packet with start record marker. Now as packets of one socket
are not reordered neither on vsock nor on vhost transport layers, such
marker allows to restore original record on receiver's side. If user's
buffer is smaller that record length, when all out of size data is
dropped.
Maximum length of datagram is not limited as in stream socket,
because same credit logic is used. Difference with stream socket is
that user is not woken up until whole record is received or error
occurred. Implementation also supports 'MSG_EOR' and 'MSG_TRUNC' flags.
Tests also implemented.
Arseny Krasnov (13):
af_vsock: prepare for SOCK_SEQPACKET support
af_vsock: prepare 'vsock_connectible_recvmsg()'
af_vsock: implement SEQPACKET rx loop
af_vsock: implement send logic for SOCK_SEQPACKET
af_vsock: rest of SEQPACKET support
af_vsock: update comments for stream sockets
virtio/vsock: dequeue callback for SOCK_SEQPACKET
virtio/vsock: fetch length for SEQPACKET record
virtio/vsock: add SEQPACKET receive logic
virtio/vsock: rest of SOCK_SEQPACKET support
virtio/vsock: setup SEQPACKET ops for transport
vhost/vsock: setup SEQPACKET ops for transport
vsock_test: add SOCK_SEQPACKET tests
drivers/vhost/vsock.c | 7 +-
include/linux/virtio_vsock.h | 12 +
include/net/af_vsock.h | 6 +
include/uapi/linux/virtio_vsock.h | 9 +
net/vmw_vsock/af_vsock.c | 543 ++++++++++++++++------
net/vmw_vsock/virtio_transport.c | 4 +
net/vmw_vsock/virtio_transport_common.c | 295 ++++++++++--
tools/testing/vsock/util.c | 32 +-
tools/testing/vsock/util.h | 3 +
tools/testing/vsock/vsock_test.c | 126 +++++
10 files changed, 862 insertions(+), 175 deletions(-)
TODO:
- Support for record integrity control. As transport could drop some
packets, something like "record-id" and record end marker need to
be implemented. Idea is that SEQ_BEGIN packet carries both record
length and record id, end marker(let it be SEQ_END) carries only
record id. To be sure that no one packet was lost, receiver checks
length of data between SEQ_BEGIN and SEQ_END(it must be same with
value in SEQ_BEGIN) and record ids of SEQ_BEGIN and SEQ_END(this
means that both markers were not dropped. I think that easiest way
to implement record id for SEQ_BEGIN is to reuse another field of
packet header(SEQ_BEGIN already uses 'flags' as record length).For
SEQ_END record id could be stored in 'flags'.
I don't really like the idea of reusing the 'flags' field for this
purpose.
Another way to implement it, is to move metadata of both SEQ_END
and SEQ_BEGIN to payload. But this approach has problem, because
if we move something to payload, such payload is accounted by
credit logic, which fragments payload, while payload with record
length and id couldn't be fragmented. One way to overcome it is to
ignore credit update for SEQ_BEGIN/SEQ_END packet.Another solution
is to update 'stream_has_space()' function: current implementation
return non-zero when at least 1 byte is allowed to use,but updated
version will have extra argument, which is needed length. For 'RW'
packet this argument is 1, for SEQ_BEGIN it is sizeof(record len +
record id) and for SEQ_END it is sizeof(record id).
Is the payload accounted by credit logic also if hdr.op is not
VIRTIO_VSOCK_OP_RW?
I think that we can define a specific header to put after the
virtio_vsock_hdr when hdr.op is SEQ_BEGIN or SEQ_END, and in this header
we can store the id and the length of the message.
- What to do, when server doesn't support SOCK_SEQPACKET. In current
implementation RST is replied in the same way when listening port
is not found. I think that current RST is enough,because case when
server doesn't support SEQ_PACKET is same when listener missed(e.g.
no listener in both cases).
I think so, but I'll check better if we can have some issues.
Thanks,
Stefano
v2 -> v3:
- patches reorganized: split for prepare and implementation patches
- local variables are declared in "Reverse Christmas tree" manner
- virtio_transport_common.c: valid leXX_to_cpu() for vsock header
fields access
- af_vsock.c: 'vsock_connectible_*sockopt()' added as shared code
between stream and seqpacket sockets.
- af_vsock.c: loops in '__vsock_*_recvmsg()' refactored.
- af_vsock.c: 'vsock_wait_data()' refactored.
v1 -> v2:
- patches reordered: af_vsock.c related changes now before virtio vsock
- patches reorganized: more small patches, where +/- are not mixed
- tests for SOCK_SEQPACKET added
- all commit messages updated
- af_vsock.c: 'vsock_pre_recv_check()' inlined to
'vsock_connectible_recvmsg()'
- af_vsock.c: 'vsock_assign_transport()' returns ENODEV if transport
was not found
- virtio_transport_common.c: transport callback for seqpacket dequeue
- virtio_transport_common.c: simplified
'virtio_transport_recv_connected()'
- virtio_transport_common.c: send reset on socket and packet type
mismatch.
Signed-off-by: Arseny Krasnov <redacted>
--
2.25.1
On Mon, Jan 25, 2021 at 02:12:36PM +0300, Arseny Krasnov wrote:
quoted
This adds receive loop for SEQPACKET. It looks like receive loop for
SEQPACKET, but there is a little bit difference:
1) It doesn't call notify callbacks.
2) It doesn't care about 'SO_SNDLOWAT' and 'SO_RCVLOWAT' values, because
there is no sense for these values in SEQPACKET case.
3) It waits until whole record is received or error is found during
receiving.
4) It processes and sets 'MSG_TRUNC' flag.
So to avoid extra conditions for two types of socket inside one loop, two
independent functions were created.
Signed-off-by: Arseny Krasnov <redacted>
---
include/net/af_vsock.h | 5 ++
net/vmw_vsock/af_vsock.c | 102 ++++++++++++++++++++++++++++++++++++++-
2 files changed, 106 insertions(+), 1 deletion(-)
+ if (sk->sk_err)
+ err = -sk->sk_err;
+ else if (sk->sk_shutdown & RCV_SHUTDOWN)
+ err = 0;
+
+ if (dequeued_total > 0) {
+ /* User sets MSG_TRUNC, so return real length of
+ * packet.
+ */
+ if (flags & MSG_TRUNC)
+ err = record_len;
+ else
+ err = len - msg->msg_iter.count;
+
+ /* Always set MSG_TRUNC if real length of packet is
+ * bigger that user buffer.
Hi Arseny,
I reviewed a part, tomorrow I hope to finish the other patches.
Just a couple of comments in the TODOs below.
On Mon, Jan 25, 2021 at 02:09:00PM +0300, Arseny Krasnov wrote:
quoted
This patchset impelements support of SOCK_SEQPACKET for virtio
transport.
As SOCK_SEQPACKET guarantees to save record boundaries, so to
do it, new packet operation was added: it marks start of record (with
record length in header), such packet doesn't carry any data. To send
record, packet with start marker is sent first, then all data is sent
as usual 'RW' packets. On receiver's side, length of record is known
from packet with start record marker. Now as packets of one socket
are not reordered neither on vsock nor on vhost transport layers, such
marker allows to restore original record on receiver's side. If user's
buffer is smaller that record length, when all out of size data is
dropped.
Maximum length of datagram is not limited as in stream socket,
because same credit logic is used. Difference with stream socket is
that user is not woken up until whole record is received or error
occurred. Implementation also supports 'MSG_EOR' and 'MSG_TRUNC' flags.
Tests also implemented.
Arseny Krasnov (13):
af_vsock: prepare for SOCK_SEQPACKET support
af_vsock: prepare 'vsock_connectible_recvmsg()'
af_vsock: implement SEQPACKET rx loop
af_vsock: implement send logic for SOCK_SEQPACKET
af_vsock: rest of SEQPACKET support
af_vsock: update comments for stream sockets
virtio/vsock: dequeue callback for SOCK_SEQPACKET
virtio/vsock: fetch length for SEQPACKET record
virtio/vsock: add SEQPACKET receive logic
virtio/vsock: rest of SOCK_SEQPACKET support
virtio/vsock: setup SEQPACKET ops for transport
vhost/vsock: setup SEQPACKET ops for transport
vsock_test: add SOCK_SEQPACKET tests
drivers/vhost/vsock.c | 7 +-
include/linux/virtio_vsock.h | 12 +
include/net/af_vsock.h | 6 +
include/uapi/linux/virtio_vsock.h | 9 +
net/vmw_vsock/af_vsock.c | 543 ++++++++++++++++------
net/vmw_vsock/virtio_transport.c | 4 +
net/vmw_vsock/virtio_transport_common.c | 295 ++++++++++--
tools/testing/vsock/util.c | 32 +-
tools/testing/vsock/util.h | 3 +
tools/testing/vsock/vsock_test.c | 126 +++++
10 files changed, 862 insertions(+), 175 deletions(-)
TODO:
- Support for record integrity control. As transport could drop some
packets, something like "record-id" and record end marker need to
be implemented. Idea is that SEQ_BEGIN packet carries both record
length and record id, end marker(let it be SEQ_END) carries only
record id. To be sure that no one packet was lost, receiver checks
length of data between SEQ_BEGIN and SEQ_END(it must be same with
value in SEQ_BEGIN) and record ids of SEQ_BEGIN and SEQ_END(this
means that both markers were not dropped. I think that easiest way
to implement record id for SEQ_BEGIN is to reuse another field of
packet header(SEQ_BEGIN already uses 'flags' as record length).For
SEQ_END record id could be stored in 'flags'.
I don't really like the idea of reusing the 'flags' field for this
purpose.
quoted
Another way to implement it, is to move metadata of both SEQ_END
and SEQ_BEGIN to payload. But this approach has problem, because
if we move something to payload, such payload is accounted by
credit logic, which fragments payload, while payload with record
length and id couldn't be fragmented. One way to overcome it is to
ignore credit update for SEQ_BEGIN/SEQ_END packet.Another solution
is to update 'stream_has_space()' function: current implementation
return non-zero when at least 1 byte is allowed to use,but updated
version will have extra argument, which is needed length. For 'RW'
packet this argument is 1, for SEQ_BEGIN it is sizeof(record len +
record id) and for SEQ_END it is sizeof(record id).
Is the payload accounted by credit logic also if hdr.op is not
VIRTIO_VSOCK_OP_RW?
Yes, on send any packet with payload could be fragmented if
there is not enough space at receiver. On receive 'fwd_cnt' and
'buf_alloc' are updated with header of every packet. Of course,
to every such case i've described i can add check for 'RW'
packet, to exclude payload from credit accounting, but this is
bunch of dumb checks.
I think that we can define a specific header to put after the
virtio_vsock_hdr when hdr.op is SEQ_BEGIN or SEQ_END, and in this header
we can store the id and the length of the message.
I think it is better than use payload and touch credit logic
quoted
- What to do, when server doesn't support SOCK_SEQPACKET. In current
implementation RST is replied in the same way when listening port
is not found. I think that current RST is enough,because case when
server doesn't support SEQ_PACKET is same when listener missed(e.g.
no listener in both cases).
I think so, but I'll check better if we can have some issues.
Thanks,
Stefano
quoted
v2 -> v3:
- patches reorganized: split for prepare and implementation patches
- local variables are declared in "Reverse Christmas tree" manner
- virtio_transport_common.c: valid leXX_to_cpu() for vsock header
fields access
- af_vsock.c: 'vsock_connectible_*sockopt()' added as shared code
between stream and seqpacket sockets.
- af_vsock.c: loops in '__vsock_*_recvmsg()' refactored.
- af_vsock.c: 'vsock_wait_data()' refactored.
v1 -> v2:
- patches reordered: af_vsock.c related changes now before virtio vsock
- patches reorganized: more small patches, where +/- are not mixed
- tests for SOCK_SEQPACKET added
- all commit messages updated
- af_vsock.c: 'vsock_pre_recv_check()' inlined to
'vsock_connectible_recvmsg()'
- af_vsock.c: 'vsock_assign_transport()' returns ENODEV if transport
was not found
- virtio_transport_common.c: transport callback for seqpacket dequeue
- virtio_transport_common.c: simplified
'virtio_transport_recv_connected()'
- virtio_transport_common.c: send reset on socket and packet type
mismatch.
Signed-off-by: Arseny Krasnov <redacted>
--
2.25.1
On Fri, Jan 29, 2021 at 09:41:50AM +0300, Arseny Krasnov wrote:
On 28.01.2021 20:19, Stefano Garzarella wrote:
quoted
Hi Arseny,
I reviewed a part, tomorrow I hope to finish the other patches.
Just a couple of comments in the TODOs below.
On Mon, Jan 25, 2021 at 02:09:00PM +0300, Arseny Krasnov wrote:
quoted
This patchset impelements support of SOCK_SEQPACKET for virtio
transport.
As SOCK_SEQPACKET guarantees to save record boundaries, so to
do it, new packet operation was added: it marks start of record (with
record length in header), such packet doesn't carry any data. To send
record, packet with start marker is sent first, then all data is sent
as usual 'RW' packets. On receiver's side, length of record is known
from packet with start record marker. Now as packets of one socket
are not reordered neither on vsock nor on vhost transport layers, such
marker allows to restore original record on receiver's side. If user's
buffer is smaller that record length, when all out of size data is
dropped.
Maximum length of datagram is not limited as in stream socket,
because same credit logic is used. Difference with stream socket is
that user is not woken up until whole record is received or error
occurred. Implementation also supports 'MSG_EOR' and 'MSG_TRUNC' flags.
Tests also implemented.
Arseny Krasnov (13):
af_vsock: prepare for SOCK_SEQPACKET support
af_vsock: prepare 'vsock_connectible_recvmsg()'
af_vsock: implement SEQPACKET rx loop
af_vsock: implement send logic for SOCK_SEQPACKET
af_vsock: rest of SEQPACKET support
af_vsock: update comments for stream sockets
virtio/vsock: dequeue callback for SOCK_SEQPACKET
virtio/vsock: fetch length for SEQPACKET record
virtio/vsock: add SEQPACKET receive logic
virtio/vsock: rest of SOCK_SEQPACKET support
virtio/vsock: setup SEQPACKET ops for transport
vhost/vsock: setup SEQPACKET ops for transport
vsock_test: add SOCK_SEQPACKET tests
drivers/vhost/vsock.c | 7 +-
include/linux/virtio_vsock.h | 12 +
include/net/af_vsock.h | 6 +
include/uapi/linux/virtio_vsock.h | 9 +
net/vmw_vsock/af_vsock.c | 543 ++++++++++++++++------
net/vmw_vsock/virtio_transport.c | 4 +
net/vmw_vsock/virtio_transport_common.c | 295 ++++++++++--
tools/testing/vsock/util.c | 32 +-
tools/testing/vsock/util.h | 3 +
tools/testing/vsock/vsock_test.c | 126 +++++
10 files changed, 862 insertions(+), 175 deletions(-)
TODO:
- Support for record integrity control. As transport could drop some
packets, something like "record-id" and record end marker need to
be implemented. Idea is that SEQ_BEGIN packet carries both record
length and record id, end marker(let it be SEQ_END) carries only
record id. To be sure that no one packet was lost, receiver checks
length of data between SEQ_BEGIN and SEQ_END(it must be same with
value in SEQ_BEGIN) and record ids of SEQ_BEGIN and SEQ_END(this
means that both markers were not dropped. I think that easiest way
to implement record id for SEQ_BEGIN is to reuse another field of
packet header(SEQ_BEGIN already uses 'flags' as record length).For
SEQ_END record id could be stored in 'flags'.
I don't really like the idea of reusing the 'flags' field for this
purpose.
quoted
Another way to implement it, is to move metadata of both SEQ_END
and SEQ_BEGIN to payload. But this approach has problem, because
if we move something to payload, such payload is accounted by
credit logic, which fragments payload, while payload with record
length and id couldn't be fragmented. One way to overcome it is to
ignore credit update for SEQ_BEGIN/SEQ_END packet.Another solution
is to update 'stream_has_space()' function: current implementation
return non-zero when at least 1 byte is allowed to use,but updated
version will have extra argument, which is needed length. For 'RW'
packet this argument is 1, for SEQ_BEGIN it is sizeof(record len +
record id) and for SEQ_END it is sizeof(record id).
Is the payload accounted by credit logic also if hdr.op is not
VIRTIO_VSOCK_OP_RW?
Yes, on send any packet with payload could be fragmented if
there is not enough space at receiver. On receive 'fwd_cnt' and
'buf_alloc' are updated with header of every packet. Of course,
to every such case i've described i can add check for 'RW'
packet, to exclude payload from credit accounting, but this is
bunch of dumb checks.
quoted
I think that we can define a specific header to put after the
virtio_vsock_hdr when hdr.op is SEQ_BEGIN or SEQ_END, and in this header
we can store the id and the length of the message.
I think it is better than use payload and touch credit logic
Cool, so let's try this option, hoping there aren't a lot of issues.
Another item for TODO could be to add the SOCK_SEQPACKET support also
for vsock_loopback. Should be simple since it also uses
virtio_transport_common APIs and it can be useful for testing and
debugging.
Thanks,
Stefano