Member of Technical Staff - Engineering Lead, Data Ingestion

What this role involves

Reflection AI is hiring a Data Ingestion Lead to build and lead the team responsible for the ingestion layer that supplies training data for frontier models. The role involves managing web crawling, data pipelines, and data lakes, while staying hands-on and collaborating with research teams.

Skills this posting asks for

• web crawling
• data ingestion
• data pipelines
• data lakes
• distributed compute
• ray
• beam
• spark
• airflow
• prefect
• parquet
• jsonl
• warc
• object stores
• lakehouse formats
• deduplication
• dataset versioning
• python
• scala
• java

Requirements

• Level: lead

From the employer’s posting

OUR MISSION Reflection is a research lab making intelligence open and accessible for everyone to use, customize, and build on. We build open models that let anyone control their intelligence and help shape the future of AI. Our mission: make intelligence open and accessible to all. ABOUT THE ROL…