Thread (5 messages) 5 messages, 3 authors, 2003-02-18

Re: 2.4.20 TCP server + solaris client performance

From: Fabrizio Nesti <hidden>
Date: 2003-02-18 15:51:58

Thanks for the replies, but:

yes,
- the UDP server is a (SCSI) 4disks RAID5. Very fast.
- The TCP (experimental) server is all the same very fast (dma enabled).
- Typical time for local operation (the same tar xf) is 1 sec or less.

solaris is : SunOS 5.8 Generic_108528-09 sun4u sparc SUNW,Ultra-5_10
linux are :
  udp server: Linux 2.4.18-24.8.0smp #1 SMP i686 i686
  tcp server: Linux 2.4.20 i686 athlon

Plain read or write (dd) is quite satisfactory, so it seems that problems
arise with many near reads and writes (like tar, cvs etc).
Also, it seems that we have no retransmission problems.
(see below nfsstat -c on solaris).

Puzzled.

Thanks to all,
cheers,
Fabrizio

PS: root@caslon:/root$ nfsstat -c

Client rpc:
Connection oriented:
calls        badcalls     badxids      timeouts     newcreds     badverfs
10994822     1645         30           16           0            0
timers       cantconn     nomem        interrupts
0            1598         0            29
Connectionless:
calls        badcalls     retrans      badxids      timeouts     newcreds
37885148     10303        6058         238          16180        0
badverfs     timers       nomem        cantsend
0            4267         0            0

Client nfs:
calls        badcalls     clgets       cltoomany
48204358     218          48204358     28
Version 2: (4019 calls)
null         getattr      setattr      root         lookup       readlink
0 0%         3909 97%     42 1%        0 0%         55 1%        6 0%
read         wrcache      write        create       remove       rename
0 0%         0 0%         0 0%         0 0%         0 0%         0 0%
link         symlink      mkdir        rmdir        readdir      statfs
0 0%         0 0%         0 0%         0 0%         6 0%         1 0%
Version 3: (48171346 calls)
null         getattr      setattr      lookup       access       readlink
0 0%         18461695 38% 482002 1%    8829223 18%  7849906 16%  16660 0%
read         write        create       mkdir        symlink      mknod
7750211 16%  3143812 6%   244856 0%    25973 0%     8847 0%      0 0%
remove       rmdir        rename       link         readdir      readdirplus
228254 0%    18425 0%     13163 0%     14493 0%     259741 0%    361588 0%
fsstat       fsinfo       pathconf     commit
50919 0%     740 0%       2075 0%      408763 0%

Client nfs_acl:
Version 2: (1 calls)
null         getacl       setacl       getattr      access
0 0%         0 0%         0 0%         1 100%       0 0%
Version 3: (28992 calls)
null         getacl       setacl
0 0%         28992 100%   0 0%



On Tue, 18 Feb 2003, Eric Whiting wrote:
Have you verified that your storage is properly setup? Is it IDE/SCSI/RAID? IDE
storage with DMA disabled will cause terrible performance. Please verify disk
speed on local writes and make sure they are 'faster' than the network.

What version of SOlaris are you running? Solaris 8 and newer has a lot of NFS
fixes...

We are getting good NFS numbers with 2.4.20 UDP NFS servers against solaris [89]
clients.

eric

Fabrizio Nesti wrote:
quoted
Sorry,
the reason to switch to TCP was exactly the same: poor nfs performance as
seen from any solaris client. And by poor I mean three-five times slower.

The same "tar xf" (a typical high load r/w usage) gives

                     linux server           solaris server
linux   client      1 sec (udp)(caching?)    8 sec (for both tcp and udp)
solaris client      22/40 sec (tcp/udp)      10/8 sec (tcp/udp)

We worry about these ^^^  figures, since we bought a new linux server
to switch to, and we have some solaris clients.

Since solaris nfs clients sseems to prefer TCP, and following some messages
on the list, we tried that. (Even tuning can not improve this situation).
Is there something wrong we are doing or we have to switch back to soalris
server?

Thanks again,
Fabrizio Nesti

On Mon, 17 Feb 2003, Alan Powell wrote:
quoted
8192 block size is a Linux daemon limitation. Also,
don't switch to TCP unless you need to. UDP will be
faster if you have a decent network. Revert back to
UDP, and on the client side, run "nfsstat -c" and
monitor the number of retransmissions. If that number
doesn't increase, you have a clean network, and you
should stay on UDP.

--- Fabrizio Nesti <nesti@medialab.sissa.it> wrote:
quoted
Hello to everybody.
we are reporting a very low performance for nfs
access from Solaris clients
to the linux nfs server on RH8.0. We thought it was
udp and upgraded to
kernel 2.4.20.

Now the performance is still low, compared to a
solaris server:
    # time gtar xf /var/tmp/cvs-1.11.5.tar
    Writing on Linux_2.4.20> real    0m22.132s
    Writing on Solaris_8>    real    0m7.174s

Both filesystems are mounted with
proto=tcp,rsize=32768,wsize=32768.
Snooping the traffic however, it appears that the
linux server is not
serving with size=32768, but with a maximum size of
8192.

- Is there a reson for this?
- May this be the reason for the poor performance
above?

Thanks in advance,
Fabrizio Nesti


PS: Some snoop traffic:
...
  solaris -> linux  NFS C CREATE3 FH=884D
(EXCLUSIVE) check_cvs.in
linux -> solaris    NFS R CREATE3 OK FH=174A
 solaris -> linux   NFS C SETATTR3 FH=174A
linux -> solaris    NFS R SETATTR3 OK
  solaris -> linux  NFS C WRITE3 FH=174A at 0 for
8192 (ASYNC)
  solaris -> linux  TCP D=2049 S=793 Ack=633960980
Seq=849797604 Len=1460
  solaris -> linux  TCP D=2049 S=793 Ack=633960980
Seq=849799064 Len=1460
  solaris -> linux  TCP D=2049 S=793 Ack=633960980
Seq=849800524 Len=1460
  solaris -> linux  TCP D=2049 S=793 Ack=633960980
Seq=849801984 Len=1460
  solaris -> linux  TCP D=2049 S=793 Ack=633960980
Seq=849803444 Len=1056
linux -> solaris    TCP D=793 S=2049 Ack=849804500
Seq=633960980 Len=0
...


-------------------------------------------------------
quoted
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

__________________________________________________
Do you Yahoo!?
Yahoo! Shopping - Send Flowers for Valentine's Day
http://shopping.yahoo.com
-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs




-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs
Keyboard shortcuts
hback out one level
jnext message in thread
kprevious message in thread
ldrill in
Escclose help / fold thread tree
?toggle this help