Staff Software Engineer - Database Engine Internals

What this role involves

Staff Software Engineer role focusing on designing and implementing the next generation query engine and structured storage system for Databricks' Lakehouse platform. Involves working on query compilation, distributed execution, vectorized execution, data security, resource management, transaction coordination, storage structures, and automatic physical data optimization.

Skills this posting asks for

• query compilation
• optimization
• distributed query execution
• scheduling
• vectorized execution engine
• data security
• resource management
• transaction coordination
• efficient storage structures
• encodings
• indexes
• automatic physical data optimization
• apache spark
• database systems
• storage systems
• distributed systems
• language design
• performance optimization

Requirements

• Level: staff

From the employer’s posting

P-188 Our mission at Databricks is to radically simplify the whole data lifecycle from ingestion to ETL, BI, and all the way up to ML/AI with a unified platform. To achieve this goal, we believe the data warehouse architecture as we know it today will be replaced by a new architectural pa…