The feature store is a critical part of how we rank and retrieve the right context across your work.
AI Summary
Dropbox Dash, a real-time AI-powered workspace, relies on a feature store to manage and deliver data signals to its ranking system. To meet its sub-100ms latency requirements and massive parallel read performance, Dropbox built a hybrid feature store using Feast, AWS DynamoDB, and Spark. The system serves features quickly, adapting to changing user behavior, and integrates with their existing infrastructure. The feature store architecture combines Feast's orchestration layer and serving APIs with a Go service for feature serving, cloud-based storage for offline indexing, and Spark jobs for feature ingestion and computation. This setup ensures a streamlined experience for engineers while abstracting away offline and online data management, pipeline orchestration, and data freshness guarantees. A three-part ingestion system was developed to balance freshness with reliability, allowing for real-time incorporation of new user signals while handling complex transformations and reducing infrastructure overload. The system consistently achieves p95 latencies in the 25-35ms range, making it possible to reliably meet Dash's latency targets
Get the top 10 engineering articles delivered every Monday.