Senior Staff Software Engineer - Lakeflow Pipelines Datasets

What this role involves

Databricks seeks a Senior Staff Software Engineer to lead the technical direction of the Lakeflow Pipelines team, focusing on declarative dataflow processing, stream processing, and large-scale distributed systems. The role involves architecting a fault-tolerant platform for pipelines and materialized views, working across areas like resource management and efficient storage structures.

Skills this posting asks for

• distributed systems
• stream processing
• data pipelines
• database internals
• storage systems
• performance engineering
• algorithmic foundations
• declarative dataflow
• resource management
• system optimization

Requirements

• 10 years of experience
• Level: staff

From the employer’s posting

Databricks builds the world's leading data and AI platform, used by tens of thousands of organizations to turn raw data into decisions that matter. The Lakeflow Engineering team owns the entirety of the ETL product line: Declarative Dataflow Graphs, Materialized Views, Structured Streamin…