Re: 2.4.20 TCP server + solaris client performance
From: Fabrizio Nesti <hidden>
Date: 2003-02-18 15:51:58
Thanks for the replies, but: yes, - the UDP server is a (SCSI) 4disks RAID5. Very fast. - The TCP (experimental) server is all the same very fast (dma enabled). - Typical time for local operation (the same tar xf) is 1 sec or less. solaris is : SunOS 5.8 Generic_108528-09 sun4u sparc SUNW,Ultra-5_10 linux are : udp server: Linux 2.4.18-24.8.0smp #1 SMP i686 i686 tcp server: Linux 2.4.20 i686 athlon Plain read or write (dd) is quite satisfactory, so it seems that problems arise with many near reads and writes (like tar, cvs etc). Also, it seems that we have no retransmission problems. (see below nfsstat -c on solaris). Puzzled. Thanks to all, cheers, Fabrizio PS: root@caslon:/root$ nfsstat -c Client rpc: Connection oriented: calls badcalls badxids timeouts newcreds badverfs 10994822 1645 30 16 0 0 timers cantconn nomem interrupts 0 1598 0 29 Connectionless: calls badcalls retrans badxids timeouts newcreds 37885148 10303 6058 238 16180 0 badverfs timers nomem cantsend 0 4267 0 0 Client nfs: calls badcalls clgets cltoomany 48204358 218 48204358 28 Version 2: (4019 calls) null getattr setattr root lookup readlink 0 0% 3909 97% 42 1% 0 0% 55 1% 6 0% read wrcache write create remove rename 0 0% 0 0% 0 0% 0 0% 0 0% 0 0% link symlink mkdir rmdir readdir statfs 0 0% 0 0% 0 0% 0 0% 6 0% 1 0% Version 3: (48171346 calls) null getattr setattr lookup access readlink 0 0% 18461695 38% 482002 1% 8829223 18% 7849906 16% 16660 0% read write create mkdir symlink mknod 7750211 16% 3143812 6% 244856 0% 25973 0% 8847 0% 0 0% remove rmdir rename link readdir readdirplus 228254 0% 18425 0% 13163 0% 14493 0% 259741 0% 361588 0% fsstat fsinfo pathconf commit 50919 0% 740 0% 2075 0% 408763 0% Client nfs_acl: Version 2: (1 calls) null getacl setacl getattr access 0 0% 0 0% 0 0% 1 100% 0 0% Version 3: (28992 calls) null getacl setacl 0 0% 28992 100% 0 0% On Tue, 18 Feb 2003, Eric Whiting wrote:
Have you verified that your storage is properly setup? Is it IDE/SCSI/RAID? IDE storage with DMA disabled will cause terrible performance. Please verify disk speed on local writes and make sure they are 'faster' than the network. What version of SOlaris are you running? Solaris 8 and newer has a lot of NFS fixes... We are getting good NFS numbers with 2.4.20 UDP NFS servers against solaris [89] clients. eric Fabrizio Nesti wrote:quoted
Sorry, the reason to switch to TCP was exactly the same: poor nfs performance as seen from any solaris client. And by poor I mean three-five times slower. The same "tar xf" (a typical high load r/w usage) gives linux server solaris server linux client 1 sec (udp)(caching?) 8 sec (for both tcp and udp) solaris client 22/40 sec (tcp/udp) 10/8 sec (tcp/udp) We worry about these ^^^ figures, since we bought a new linux server to switch to, and we have some solaris clients. Since solaris nfs clients sseems to prefer TCP, and following some messages on the list, we tried that. (Even tuning can not improve this situation). Is there something wrong we are doing or we have to switch back to soalris server? Thanks again, Fabrizio Nesti On Mon, 17 Feb 2003, Alan Powell wrote:quoted
8192 block size is a Linux daemon limitation. Also, don't switch to TCP unless you need to. UDP will be faster if you have a decent network. Revert back to UDP, and on the client side, run "nfsstat -c" and monitor the number of retransmissions. If that number doesn't increase, you have a clean network, and you should stay on UDP.--- Fabrizio Nesti <nesti@medialab.sissa.it> wrote:quoted
Hello to everybody. we are reporting a very low performance for nfs access from Solaris clients to the linux nfs server on RH8.0. We thought it was udp and upgraded to kernel 2.4.20. Now the performance is still low, compared to a solaris server: # time gtar xf /var/tmp/cvs-1.11.5.tar Writing on Linux_2.4.20> real 0m22.132s Writing on Solaris_8> real 0m7.174s Both filesystems are mounted with proto=tcp,rsize=32768,wsize=32768. Snooping the traffic however, it appears that the linux server is not serving with size=32768, but with a maximum size of 8192. - Is there a reson for this? - May this be the reason for the poor performance above? Thanks in advance, Fabrizio Nesti PS: Some snoop traffic: ... solaris -> linux NFS C CREATE3 FH=884D (EXCLUSIVE) check_cvs.in linux -> solaris NFS R CREATE3 OK FH=174A solaris -> linux NFS C SETATTR3 FH=174A linux -> solaris NFS R SETATTR3 OK solaris -> linux NFS C WRITE3 FH=174A at 0 for 8192 (ASYNC) solaris -> linux TCP D=2049 S=793 Ack=633960980 Seq=849797604 Len=1460 solaris -> linux TCP D=2049 S=793 Ack=633960980 Seq=849799064 Len=1460 solaris -> linux TCP D=2049 S=793 Ack=633960980 Seq=849800524 Len=1460 solaris -> linux TCP D=2049 S=793 Ack=633960980 Seq=849801984 Len=1460 solaris -> linux TCP D=2049 S=793 Ack=633960980 Seq=849803444 Len=1056 linux -> solaris TCP D=793 S=2049 Ack=849804500 Seq=633960980 Len=0 ...-------------------------------------------------------quoted
This sf.net email is sponsored by:ThinkGeek Welcome to geek heaven. http://thinkgeek.com/sf _______________________________________________ NFS maillist - NFS@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/nfs__________________________________________________ Do you Yahoo!? Yahoo! Shopping - Send Flowers for Valentine's Day http://shopping.yahoo.com------------------------------------------------------- This sf.net email is sponsored by:ThinkGeek Welcome to geek heaven. http://thinkgeek.com/sf _______________________________________________ NFS maillist - NFS@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/nfs
------------------------------------------------------- This sf.net email is sponsored by:ThinkGeek Welcome to geek heaven. http://thinkgeek.com/sf _______________________________________________ NFS maillist - NFS@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/nfs