Skip to content
  • Ilya Dryomov's avatar
    libceph: use MSG_SENDPAGE_NOTLAST with ceph_tcp_sendpage() · 433b0a12
    Ilya Dryomov authored
    Prevent do_tcp_sendpages() from calling tcp_push() (at least) once per
    page.  Instead, arrange for tcp_push() to be called (at least) once per
    data payload.  This results in more MSS-sized packets and fewer packets
    overall (5-10% reduction in my tests with typical OSD request sizes).
    See commits 2f533844 ("tcp: allow splice() to build full TSO
    packets"), 35f9c09f ("tcp: tcp_sendpages() should call tcp_push()
    once") and ae62ca7b
    
     ("tcp: fix MSG_SENDPAGE_NOTLAST logic") for
    details.
    
    Here is an example of a packet size histogram for 128K OSD requests
    (MSS = 1448, top 5):
    
    Before:
    
         SIZE    COUNT
         1448   777700
          952   127915
         1200    39238
         1219     9806
           21     5675
    
    After:
    
         SIZE    COUNT
         1448   897280
           21     6201
         1019     2797
          643     2739
          376     2479
    
    We could do slightly better by explicitly corking the socket but it's
    not clear it's worth it.
    
    Signed-off-by: default avatarIlya Dryomov <idryomov@gmail.com>
    433b0a12