delta-lake jobs in San Diego

Staff Data Engineer at Qualcomm responsible for designing, building, and maintaining scalable batch and streaming data pipelines on AWS and Databricks. Develop reusable data engineering frameworks, libraries, and templates while implementing AI-driven automation for schema inference, data quality checks, pipeline orchestration, and self-healing mechanisms.…See more Lead reliability and operations efforts by defining SLIs/SLOs, participating in incident response, and driving performance tuning and cost optimization. Mentor junior engineers, collaborate across analytics and platform teams, and drive adoption of standardized data frameworks and best practices.

Lead large-scale enterprise data transformation programs at Qualcomm, managing multi-year initiatives across data modernization, cloud migration, and analytics platforms. Coordinate work across distributed data engineering, architecture, governance, security, and analytics teams while managing multi-million-dollar budgets and vendor relationships.…See more Drive adoption of modern data architectures (data lakehouse, data mesh, medallion patterns) and cloud platforms (Databricks, AWS, Azure, GCP), and establish federated data governance frameworks. Influence executives and technical stakeholders to deliver measurable business outcomes while handling legacy system retirement and enterprise data integration from CRM, ERP, HCM, and supply-chain systems.

Design, build, and operate enterprise-scale Databricks lakehouse platforms for a global semiconductor company. Lead AI-native development practices, machine learning enablement, and AIOps automation while mentoring engineering teams and establishing technical standards.…See more Drive platform modernization, cloud-native practices, and intelligent data/AI infrastructure at scale. Requires 4–5 years of hands-on Databricks experience, strong Python/SQL/Spark skills, MLOps expertise, and proven technical leadership.

Lead the design, build, and operation of enterprise-scale Databricks Lakehouse platforms at Qualcomm, driving AI-native development, MLOps practices, and cloud modernization. This hands-on Staff IT Engineer role requires 4–5 years of Databricks experience and 8–12 years total IT background, with responsibility for platform architecture (Delta Lake, Unity Catalog, Workflows), machine learning enablement (MLflow, Mosaic AI), AIOps automation, and technical mentoring of engineers.…See more You will establish engineering standards, optimize platform performance and cost, implement observability and self-healing automation, and drive adoption of generative AI tools and intelligent automation across development teams.

Senior architect responsible for designing and leading enterprise-scale data platforms on Databricks and AWS, including cloud modernization, data migration, and lakehouse architecture. Requires deep expertise in PySpark, advanced SQL, Delta Lake, and data warehousing patterns, with accountability for technical strategy, solution design, and mentoring engineering teams.…See more Will drive CI/CD practices, data governance frameworks, data quality standards, and technical decision-making across complex data infrastructure initiatives. Expected to engage senior stakeholders, manage client relationships, and champion AI-powered engineering tools for productivity and automation.

Lead a data engineering team responsible for building scalable data pipelines, trusted datasets, and reusable data products powering analytics, experimentation, and AI across Scribd. This role blends technical leadership with hands-on architecture guidance; you'll establish engineering standards, drive design reviews, mentor engineers, and partner cross-functionally to translate business needs into production-grade data solutions.…See more Requires 10+ years in data engineering or data platforms, 3+ years leading engineering teams, deep expertise in dimensional modeling and scalable data architectures, and strong SQL plus Python/Scala skills. You'll work with modern cloud data platforms like Databricks, Snowflake, and BigQuery, and distributed processing frameworks like Spark.

Staff Data Engineer at Illumina seeking a senior individual contributor with 10+ years building and scaling data products on Databricks and Snowflake. The role spans end-to-end ownership of data pipelines across Supply Chain, Manufacturing, and Quality domains, requiring advanced Python, SQL, and data modeling skills with hands-on expertise in Spark, Delta Lake, dbt, and distributed systems design.…See more Responsibilities include designing medallion-architecture data products, embedding governance and data quality, adopting AI in analytics workflows, and mentoring a global engineering team. The candidate will set technical standards, lead architecture decisions, and communicate trade-offs across business, AI, and platform stakeholders.

Design, build, and maintain scalable batch and streaming data pipelines on AWS and Databricks, developing reusable data engineering frameworks and standardized patterns for ingestion, transformation, and validation. Leverage AI-based techniques to automate data quality checks, anomaly detection, and pipeline self-healing.…See more Define SLIs/SLOs, participate in on-call rotations, and mentor junior engineers while driving adoption of data frameworks and best practices across analytics, reporting, and AI/ML teams.

SHEIN is seeking a Senior Data Engineer to build and productionize GenAI/LLM solutions for data engineering workflows, including code assistance, metadata discovery, and knowledge access. The role focuses on developing RAG and agentic workflows, automating incident triage and log analysis, and integrating AI capabilities into internal developer tools and services.…See more You'll own scoped projects from problem definition through production support, requiring 3+ years of hands-on production experience with strong Python/SQL fundamentals, proven GenAI/LLM application development, and practical knowledge of databases, APIs, data pipelines, and distributed systems.

Senior Data Engineer role focused on building and productionizing GenAI/LLM solutions for data engineering workflows. You will develop RAG and agentic workflows, automate incident triage and root-cause analysis, integrate AI capabilities into internal developer tools, and own projects from problem definition through production support.…See more Required: 3+ years of production systems experience, strong Python/SQL, hands-on GenAI/LLM application building, and software engineering fundamentals including testing, CI/CD, and monitoring.

Design and build scalable, distributed data platform systems on AWS and Databricks, owning workspace architecture, governance, and lifecycle management. Implement infrastructure as code with Terraform, develop end-to-end data pipelines from ingestion to analytics and ML, and operate Amazon EKS clusters.…See more Define SLIs/SLOs for platform reliability, lead incident response and on-call rotations, implement security and compliance controls, and mentor engineers on platform standards. This senior staff role requires 8+ years building large-scale cloud platforms, 5+ years in Data Platform/DevOps/SRE roles, expert AWS knowledge, and deep hands-on experience with Databricks Lakehouse (Unity Catalog, Delta Lake, MLflow), Terraform, CI/CD pipelines, and Python/Bash scripting.

As a Data Developer, you will design, develop, and maintain scalable ETL/ELT pipelines and cloud-based data platforms that integrate enterprise systems and support AI/ML initiatives for the U.S. Navy.…See more You will perform data engineering tasks including cleansing, transformation, and optimization of SQL and Apache Spark workloads on AWS, while also developing predictive models and generative AI solutions using Python-based technologies. You will build interactive dashboards and visualizations in Tableau or Qlik, translating technical findings into actionable insights for stakeholders. An active Secret clearance, U.S. citizenship, and a bachelor's degree in a technical field (or 4+ years of relevant professional experience) are required.