使用GNU / Linux系统调用`splice`在Haskell中进行Zero-Copy Socket到Socket的数据传输

更新:Nemo先生的回答帮助解决了这个问题! 下面的代码包含修复! 请参阅下面的nb Falsenb True调用。

还有一个叫做splice的新的Haskell软件包 ,它具有最好的套接字到套接字数据传输循环的操作系统特定的和可移植的实现

我有以下(Haskell)代码:

 #ifdef LINUX_SPLICE #include <fcntl.h> {-# LANGUAGE CPP #-} {-# LANGUAGE ForeignFunctionInterface #-} #endif module Network.Socket.Splice ( Length , zeroCopy , splice #ifdef LINUX_SPLICE , c_splice #endif ) where import Data.Word import Foreign.Ptr import Network.Socket import Control.Monad import Control.Exception import System.Posix.Types import System.Posix.IO #ifdef LINUX_SPLICE import Data.Int import Data.Bits import Unsafe.Coerce import Foreign.C.Types import Foreign.C.Error import System.Posix.Internals #else import System.IO import Foreign.Marshal.Alloc #endif zeroCopy :: Bool zeroCopy = #ifdef LINUX_SPLICE True #else False #endif type Length = #ifdef LINUX_SPLICE (#type size_t) #else Int #endif -- | The 'splice' function pipes data from -- one socket to another in a loop. -- On Linux this happens in kernel space with -- zero copying between kernel and user spaces. -- On other operating systems, a portable -- implementation utilizes a user space buffer -- allocated with 'mallocBytes'; 'hGetBufSome' -- and 'hPut' are then used to avoid repeated -- tiny allocations as would happen with 'recv' -- 'sendAll' calls from the 'bytestring' package. splice :: Length -> Socket -> Socket -> IO () splice l (MkSocket x _ _ _ _) (MkSocket y _ _ _ _) = do let e = error "splice ended" #ifdef LINUX_SPLICE (r,w) <- createPipe print ('+',r,w) let s = Fd x -- source let t = Fd y -- target let c = throwErrnoIfMinus1 "Network.Socket.Splice.splice" let u = unsafeCoerce :: (#type ssize_t) -> (#type size_t) let fs = sPLICE_F_MOVE .|. sPLICE_F_MORE let nb v = do setNonBlockingFD xv setNonBlockingFD yv nb False finally (forever $ do b <- c $ c_splice s nullPtr w nullPtr l fs if b > 0 then c_splice r nullPtr t nullPtr (ub) fs) else e (do closeFd r closeFd w nb True print ('-',r,w)) #else -- .. #endif #ifdef LINUX_SPLICE -- SPLICE -- fcntl.h -- ssize_t splice( -- int fd_in, -- loff_t* off_in, -- int fd_out, -- loff_t* off_out, -- size_t len, -- unsigned int flags -- ); foreign import ccall "splice" c_splice :: Fd -> Ptr (#type loff_t) -> Fd -> Ptr (#type loff_t) -> (#type size_t) -> Word -> IO (#type ssize_t) sPLICE_F_MOVE :: Word sPLICE_F_MOVE = (#const "SPLICE_F_MOVE") sPLICE_F_MORE :: Word sPLICE_F_MORE = (#const "SPLICE_F_MORE") #endif 

注意: 上面的代码现在正常工作! 感谢Nemo,下面不再有效了!

我用上面定义的splice调用了两个打开和连接的套接字(它们已经被用来传输最less量的握手数据,使用套接字API sendrecv调用或转换为句柄,并与hGetLinehPut ),我不断得到:

 Network.Socket.Splice.splice: resource exhausted (Resource temporarily unavailable) 

在第一个c_splice调用站点: c_splice返回-1并设置一些errno值(可能是EAGAIN ),读取resource exhausted | resource temporarily unavailable resource exhausted | resource temporarily unavailable在查找时resource exhausted | resource temporarily unavailable

我testing了不同Length值的调用splice8192

Solutions Collecting From Web of "使用GNU / Linux系统调用`splice`在Haskell中进行Zero-Copy Socket到Socket的数据传输"

我不知道Haskell,但“资源暂时不可用”是EAGAIN

看起来Haskell默认将其套接字设置为非阻塞模式 。 因此,如果您在没有数据的情况下尝试从一个中读取数据,或者在其缓冲区已满时尝试写入一个数据,则会使用EAGAIN失败。

找出如何将套接字更改为阻塞模式,我打赌你会解决你的问题。

[更新]

或者,在尝试读取或写入套接字之前调用selectpoll 。 但是您仍然需要处理EAGAIN ,因为在Linux select将会指示套接字已经准备好的情况下,实际上并不存在。

sendfile()系统调用会为你工作吗? 如果是这样,你可以使用sendfile包 。