Multicast Still King for Market Data
Despite its age, multicast remains critical for low-latency trading and market data distribution. In a recent podcast, an HPE engineer called it a 'walled garden' technology that thrives in controlled bank and exchange networks where sending one stream to many recipients with minimal latency is paramount.
The efficiency of multicast, which uses a one-to-many data transmission model, significantly reduces network congestion and improves data delivery efficiency. This contrasts with unicast, a one-to-one method, and broadcast, a one-to-all method. In high-frequency trading, this reduction in latency is critical for executing trades based on the most current market data. Financial exchanges like the Shenzhen Stock Exchange (SZSE) utilize specific multicast protocols, such as the Multicast Market Data Distribution Protocol (MDDP), to ensure the real-time distribution of market data. These protocols often build upon the User Datagram Protocol (UDP), which allows for fast, connectionless communication. However, this can lead to challenges like packet loss and out-of-order delivery, known as multicast gaps, which can have significant financial implications. To combat the inherent latency in software-based processing, trading firms are increasingly turning to hardware-level solutions. Field-Programmable Gate Arrays (FPGAs) allow trading algorithms to be executed directly on the hardware, enabling faster data analysis and decision-making. This approach offers deterministic, nanosecond-level latency, a significant improvement over the millisecond or microsecond latencies of traditional CPU-based systems. Kernel bypass techniques are another critical component in reducing latency. By allowing trading applications to communicate directly with the network interface card (NIC), these methods circumvent the processing overhead of the operating system's kernel. Technologies like DPDK (Data Plane Development Kit) facilitate this direct access, further minimizing delays. The debate between on-premises and cloud deployments continues to be a key consideration for trading infrastructure. On-premises setups have traditionally been favored for their low latency and high degree of control. However, cloud solutions are becoming increasingly competitive, offering scalability and flexibility, though potential latency issues remain a concern for high-frequency trading where microseconds matter. Some firms are adopting a hybrid approach, combining the control of on-premises infrastructure for research and latency-sensitive operations with the scalability of the cloud for live trading. The choice ultimately depends on specific needs regarding latency, cost, scalability, and security. For applications where sub-millisecond latency is paramount, co-locating servers within the same data centers as exchanges remains a common strategy.
