Data Pipelines at Scale – How Citadel Innovates with Data
At Citadel, data is the core of the investment process. Data informs our decisions to deploy billions of dollars of capital across a wide array of markets. Anyone can buy a dataset though. For Citadel, the differentiation is that getting the data is only the beginning. In this talk, Rob and Andy from Citadel’s Core Data Engineering Team will walk through Apache’s Airflow project and share their experiences leveraging and extending open source software to manage ETL processes for the vast expanse of data made available to our investment professionals.