Uber Details Real-Time ML Infrastructure
Uber Engineering has detailed its scalable Kafka consumer proxy, named "Uforwarder," designed to support its real-time, event-driven microservices. The system abstracts the complexity of managing Kafka topics, enabling product and ML teams to process user events in real time. This infrastructure is foundational for Uber's dynamic systems like surge pricing, fraud detection, and real-time recommendations.
- Uber's Apache Kafka deployment is one of the largest in the world, processing trillions of messages and multiple petabytes of data daily across more than 1,000 consumer services. - Uforwarder was developed to solve specific scaling challenges with traditional Kafka consumers, including head-of-line blocking (where a single slow message delays all subsequent ones in a partition), complex partition management, and inconsistent client support across different programming languages. - The system functions as a push-based proxy, fetching messages from Kafka and pushing them to consumer services via gRPC, a high-performance RPC framework. This inverts the standard pull-based model of Kafka consumers. - Uforwarder is a key component of the infrastructure supporting "Michelangelo," Uber's end-to-end Machine Learning platform. Michelangelo currently manages around 400 active ML projects and serves 10 million real-time predictions per second at its peak. - The real-time features powered by this event-driven architecture include calculating ETAs, ranking Uber Eats homefeed items, rider-driver matching, and detecting fraud. - To improve efficiency and workload isolation, Uforwarder uses "context-aware routing," which propagates metadata in Kafka message headers to make infrastructure-level routing decisions, ensuring events are delivered only to the relevant consumer instances. - The project has been fully open-sourced by Uber Engineering and has become the primary method for consuming Kafka data internally.
