Building High-Performance Engineering Teams
A recent podcast highlighted sociologist Omar Lizardo's theory of culture creating social "bridges" and "fences" within organizations. The framework suggests that effective engineering leadership requires building a broad, inclusive culture (bridges) for collaboration while also cultivating specialized, high-performance groups (fences) around niche domains like FPGAs or kernel bypass.
Omar Lizardo's research suggests that "highbrow" cultural consumption, which requires a significant investment in learning, tends to create dense, strong-tie social networks—an academic parallel to the specialized engineering "fences". In contrast, consumption of popular culture is associated with a higher density of weak ties, acting as social "bridges" that connect diverse groups. This aligns with the need for broad, cross-functional collaboration on larger platform initiatives. The "fenced-off" teams working on FPGAs and kernel bypass are critical for achieving the lowest possible latency in trade execution. Kernel bypass techniques, for example, can reduce latency from over 20 microseconds in a traditional network stack to under one microsecond by allowing applications to communicate directly with the network interface card. This is a domain where deep, shared expertise and a specialized culture are paramount for success. Competitors like Citadel Securities and Jane Street have heavily invested in such specialized teams. Citadel Securities processes over a billion market data messages per second with median latencies under one microsecond, a feat achieved by dedicating specific CPU cores to market data processing and order generation. Jane Street combines kernel bypass with FPGAs to filter 95% of market data messages in hardware before they even reach the CPU. The decision to house this infrastructure on-premises versus in the cloud is a significant architectural trade-off. On-premises deployments offer the lowest possible latency and the greatest control over hardware and network configurations, which is crucial for high-frequency trading. However, this comes at the cost of higher capital expenditures and less flexibility compared to cloud solutions. While the public cloud is often perceived as too high-latency for the most critical trading functions, it is increasingly used for less latency-sensitive workloads in finance, such as risk calculations and big data analytics. Morgan Stanley itself employs a multi-cloud architecture with Azure and AWS for various functions, indicating a hybrid approach to infrastructure modernization. This highlights the need for "bridges" between the "fenced" on-premises teams and the broader organization that leverages cloud technologies.
