A Comprehensive Zero-Copy Architecture for High Performance Distributed Data Acquisition Over Advanced Network Technologies for the CMS Experiment

作者:Bauer Gerry*; Behrens Ulf; Branson James; Bukowiec Sebastian; Chaze Olivier; Cittolin Sergio; Perez Jose Antonio Coarasa; Deldicque Christian; Dobson Marc; Dupont Aymeric; Erhan Samim; Gigi Dominique; Glege Frank; Gomez Reino Robert; Hartl Christian; Holzner Andre; Masetti Lorenzo; Meijers Frans; Meschi Emilio; Mommsen Remigius K; Nunez Barranco Fernandez Carlos; O'Dell Vivian; Orsini Luciano; Paus Christoph; Petrucci Andrea; Pieri Marco; Polese Giovanni; Racz Attila
来源:IEEE Transactions on Nuclear Science, 2013, 60(6): 4595-4602.
DOI:10.1109/TNS.2013.2282340

摘要

This paper outlines a software architecture where zero-copy operations are used comprehensively at every processing point from the Application layer to the Physical layer. The proposed architecture is being used during feasibility studies on advanced networking technologies for the CMS experiment at CERN. The design relies on a homogeneous peer-to-peer message passing system, which is built around memory pool caches allowing efficient and deterministic latency handling of messages of any size through the different software layers. In this scheme portable distributed applications can be programmed to process input to output operations by mere pointer arithmetic and DMA operations only. The approach combined with the open fabric protocol stack (OFED) allows one to attain near wire-speed message transfer at application level. The architecture supports full portability of user applications by encapsulating the protocol details and network into modular peer transport services whereas a transparent replacement of the underlying protocol facilitates deployment of several network technologies like Gigabit Ethernet, Myrinet, Infiniband, etc. Therefore, this solution provides a protocol-independent communication framework and prevents having to deal with potentially difficult couplings when the underlying communication infrastructure is changed. We demonstrate the feasibility of this approach by giving efficiency and performance measurements of the software in the context of the CMS distributed event building studies.

  • 出版日期2013-12