unity-catalog jobs in San Diego

Senior-level data engineer responsible for designing and implementing cloud-based data solutions on Databricks (AWS, Azure, or GCP), with expertise in Lakehouse architecture, Apache Spark, Delta Lake, and distributed compute platforms. Lead cross-functional data engineering teams and projects, oversee CI/CD automation, and consult on data governance, security, and compliance strategies.…See more Manage complex data pipelines using Delta Live Tables, Autoloader, Structured Streaming, Databricks Workflows, and Apache Airflow, with demonstrated ability to optimize performance and mentor technical staff.

Lead Data Engineer to design, build, and deliver Databricks-based data and analytics solutions for Deloitte's consulting clients. You will manage delivery teams through requirements gathering, solution design, development, testing, and deployment while translating business and technical requirements into scalable data pipelines and platform capabilities.…See more The role requires 6+ years of data engineering or cloud data platform experience, 3+ years with Databricks and Apache Spark, 3+ years with Python and SQL, and 2+ years leading project teams. You will advise client stakeholders on architecture, implementation, and operating models while overseeing solution quality and project delivery.

Principal Engineer leading the design, build, and operation of a Microsoft Fabric–based unified data platform (UDP) serving 50+ enterprise source systems across R&D, Commercial, Manufacturing, and Supply Chain. You will architect production-grade data ingestion pipelines, mentor data and analytics engineers, enforce engineering standards and SLAs, manage external implementation partners, and build the semantic layer and governance controls.…See more The role requires 10+ years of data engineering and platform architecture experience, 2+ years of technical leadership in a lakehouse environment, and deep hands-on expertise in Microsoft Fabric (lakehouses, Data Factory, deployment pipelines), PySpark, T-SQL, Python, medallion-architecture design, CI/CD, and data governance tooling—with biopharmaceutical or regulated-industry GxP experience strongly preferred.

Staff Data Engineer at Qualcomm responsible for designing, building, and maintaining scalable batch and streaming data pipelines on AWS and Databricks. Develop reusable data engineering frameworks, libraries, and templates while implementing AI-driven automation for schema inference, data quality checks, pipeline orchestration, and self-healing mechanisms.…See more Lead reliability and operations efforts by defining SLIs/SLOs, participating in incident response, and driving performance tuning and cost optimization. Mentor junior engineers, collaborate across analytics and platform teams, and drive adoption of standardized data frameworks and best practices.

Lead large-scale enterprise data transformation programs at Qualcomm, managing multi-year initiatives across data modernization, cloud migration, and analytics platforms. Coordinate work across distributed data engineering, architecture, governance, security, and analytics teams while managing multi-million-dollar budgets and vendor relationships.…See more Drive adoption of modern data architectures (data lakehouse, data mesh, medallion patterns) and cloud platforms (Databricks, AWS, Azure, GCP), and establish federated data governance frameworks. Influence executives and technical stakeholders to deliver measurable business outcomes while handling legacy system retirement and enterprise data integration from CRM, ERP, HCM, and supply-chain systems.

Principal Engineer leading the design, build, and operation of an enterprise data platform serving 50+ source systems across R&D, manufacturing, supply chain, and business functions. Responsible for hands-on architecture of medallion-layer pipelines using Microsoft Fabric, technical leadership and mentorship of data engineers, vendor management, data governance implementation, and platform reliability—balancing feature delivery with security, compliance (GxP), and operational excellence in a regulated biopharmaceutical environment.See more

Design, build, and operate enterprise-scale Databricks lakehouse platforms for a global semiconductor company. Lead AI-native development practices, machine learning enablement, and AIOps automation while mentoring engineering teams and establishing technical standards.…See more Drive platform modernization, cloud-native practices, and intelligent data/AI infrastructure at scale. Requires 4–5 years of hands-on Databricks experience, strong Python/SQL/Spark skills, MLOps expertise, and proven technical leadership.

Lead the design, build, and operation of enterprise-scale Databricks Lakehouse platforms at Qualcomm, driving AI-native development, MLOps practices, and cloud modernization. This hands-on Staff IT Engineer role requires 4–5 years of Databricks experience and 8–12 years total IT background, with responsibility for platform architecture (Delta Lake, Unity Catalog, Workflows), machine learning enablement (MLflow, Mosaic AI), AIOps automation, and technical mentoring of engineers.…See more You will establish engineering standards, optimize platform performance and cost, implement observability and self-healing automation, and drive adoption of generative AI tools and intelligent automation across development teams.

Sr Manager leading a team of Platform Engineers/DataOps analysts to architect, deploy, and manage enterprise-scale Databricks Unity Catalog across dev/test/production environments. Responsibilities span platform governance, fine-grained access control (RBAC/ABAC), workspace automation via Terraform, CI/CD pipeline design (GitHub Actions, Azure DevOps, GitLab CI), FinOps optimization, and incident management for critical data infrastructure.…See more Requires 10+ years in data operations/platform engineering with 5+ years as a lead Databricks administrator, deep expertise in Unity Catalog governance, cloud security (IAM, VPC/VNet, compliance), and proficiency in Python, PySpark, and SQL.

Design and build production ETL/ELT pipelines ingesting data from databases, APIs, applications, and Microsoft 365 sources into Azure Databricks and Power BI. Own code quality, CI/CD automation, testing strategies, monitoring, and data validation across development, test, and production environments in Azure DevOps.…See more Develop transformations and curated data products for analytics and AI applications, including secure document ingestion, metadata indexing, and telemetry. Collaborate with business and technical stakeholders to communicate progress, data limitations, and risks while maintaining runbooks and technical documentation.

Staff Data Engineer at Illumina seeking a senior individual contributor with 10+ years building and scaling data products on Databricks and Snowflake. The role spans end-to-end ownership of data pipelines across Supply Chain, Manufacturing, and Quality domains, requiring advanced Python, SQL, and data modeling skills with hands-on expertise in Spark, Delta Lake, dbt, and distributed systems design.…See more Responsibilities include designing medallion-architecture data products, embedding governance and data quality, adopting AI in analytics workflows, and mentoring a global engineering team. The candidate will set technical standards, lead architecture decisions, and communicate trade-offs across business, AI, and platform stakeholders.

Principal Engineer leading the design, build, and operation of an enterprise data platform serving 50+ source systems across R&D, Commercial, Manufacturing, and other business functions. The role combines hands-on data engineering (PySpark, T-SQL, Python, Data Factory, medallion architecture) with technical leadership of data engineers, vendor management, platform standards enforcement, and cross-functional stakeholder partnership.…See more Responsibilities span CI/CD pipeline design, data quality and reliability instrumentation, semantic layer development, data governance implementation, and delivery of new source-system integrations at scale.

Design, build, and maintain scalable batch and streaming data pipelines on AWS and Databricks, developing reusable data engineering frameworks and standardized patterns for ingestion, transformation, and validation. Leverage AI-based techniques to automate data quality checks, anomaly detection, and pipeline self-healing.…See more Define SLIs/SLOs, participate in on-call rotations, and mentor junior engineers while driving adoption of data frameworks and best practices across analytics, reporting, and AI/ML teams.

Design and build scalable, distributed data platform systems on AWS and Databricks, owning workspace architecture, governance, and lifecycle management. Implement infrastructure as code with Terraform, develop end-to-end data pipelines from ingestion to analytics and ML, and operate Amazon EKS clusters.…See more Define SLIs/SLOs for platform reliability, lead incident response and on-call rotations, implement security and compliance controls, and mentor engineers on platform standards. This senior staff role requires 8+ years building large-scale cloud platforms, 5+ years in Data Platform/DevOps/SRE roles, expert AWS knowledge, and deep hands-on experience with Databricks Lakehouse (Unity Catalog, Delta Lake, MLflow), Terraform, CI/CD pipelines, and Python/Bash scripting.

Staff-level data engineer who designs, develops, and maintains scalable ETL/ELT pipelines using Databricks, PySpark, and Python for enterprise analytics and AI use cases. Builds curated data layers following Lakehouse Medallion architecture, develops Databricks-native applications (notebooks, dashboards, APIs), and optimizes Apache Spark workloads for performance and cost efficiency.…See more Owns production data pipelines end-to-end, ensures data quality and reliability, and acts as a technical leader mentoring teams on data engineering best practices and modern Databricks capabilities.