unity-catalog jobs in San Diego

Design and build scalable, distributed data platform systems on AWS and Databricks, owning workspace architecture, governance, and lifecycle management. Implement infrastructure as code with Terraform, develop end-to-end data pipelines from ingestion to analytics and ML, and operate Amazon EKS clusters.…See more Define SLIs/SLOs for platform reliability, lead incident response and on-call rotations, implement security and compliance controls, and mentor engineers on platform standards. This senior staff role requires 8+ years building large-scale cloud platforms, 5+ years in Data Platform/DevOps/SRE roles, expert AWS knowledge, and deep hands-on experience with Databricks Lakehouse (Unity Catalog, Delta Lake, MLflow), Terraform, CI/CD pipelines, and Python/Bash scripting.

Staff-level data engineer who designs, develops, and maintains scalable ETL/ELT pipelines using Databricks, PySpark, and Python for enterprise analytics and AI use cases. Builds curated data layers following Lakehouse Medallion architecture, develops Databricks-native applications (notebooks, dashboards, APIs), and optimizes Apache Spark workloads for performance and cost efficiency.…See more Owns production data pipelines end-to-end, ensures data quality and reliability, and acts as a technical leader mentoring teams on data engineering best practices and modern Databricks capabilities.