databricks jobs in San Diego

This role is a Data Engineering Architect responsible for designing and reviewing enterprise-scale data architectures on Databricks and AWS, leading migration and modernization programs, and resolving complex technical challenges during build and deployment phases. The architect will provide strategic insights to development teams and clients, establish data governance frameworks, monitor system performance, and contribute to proposals and RFPs.…See more Required expertise includes hands-on proficiency in Databricks, PySpark, SQL, AWS, CI/CD pipelines, and data architecture best practices, with proven experience in enterprise-scale programs and strong technical leadership and stakeholder management skills.

Lead commercial data management for a biotech company, overseeing data quality, validation, and governance processes across field and internal reports, analytics systems, and business operations. Design and implement data governance policies, standards, and best practices while managing cross-functional projects in a matrixed environment.…See more Requires 6+ years in pharmaceutical/biotech commercial operations or data management, expert knowledge of commercial biopharma data ecosystems, and proficiency with tools including Excel, Power BI, Tableau, Databricks, SQL, and Python. Ensure compliance with HIPAA and regulatory requirements while mentoring junior team members and communicating data insights to technical and business stakeholders.

Design, build, and maintain secure enterprise data platforms supporting AI/ML workloads, with responsibility for database optimization, CI/CD automation, and DoD compliance. Mentor junior engineers and provide technical leadership on architecture decisions.…See more Implement security controls aligned with NIST 800-53, RMF, and DISA STIG standards throughout the platform lifecycle. Develop ETL/ELT pipelines, support data lakes and lakehouses, and evaluate emerging data technologies for mission fit.

Lead the modernization of legacy SSIS/SSRS data systems into cloud-native Lakehouse pipelines using Databricks, dbt, Fivetran, and Airflow. Design standardized frameworks for ingestion, transformation, and consumption layers while driving CI/CD, observability, and engineering best practices across Azure and GCP.…See more Evaluate emerging technologies (Delta Live Tables, Iceberg, streaming ingestion) through POCs and POVs, and leverage AI-assisted tools (Databricks Assistant, Cursor AI, GitHub Copilot) to accelerate development and reduce technical debt. Provide technical guidance to the team and translate business requirements into scalable data solutions.

Principal-level individual contributor role leading AI/ML architecture and engineering strategy at scale. Requires 13+ years across ML, data engineering, and distributed systems, with 3+ years shipping production Generative/Agentic AI systems.…See more Hands-on expertise in RAG, vector databases, LLM optimization, agentic orchestration, and MLOps/LLMOps practice. Will set technical direction, own architecture decisions, drive platform initiatives, and raise engineering standards across the organization without direct people-management responsibility.

Enterprise Data & Analytics Architect responsible for designing and developing SQL Server databases, data warehouses, and ETL pipelines using SSIS/Azure Data Factory. The role requires 8–10 years of experience in OLAP/OLTP environments, expert SQL Server knowledge, and proficiency in enterprise reporting (SSRS, Power BI).…See more Candidates will oversee data engineering, BI development, and data science initiatives; mentor developers; and maintain compliance with company policies and federal regulations.

Lead a Data Engineering team to establish modern DataOps and CI/CD practices while providing technical mentorship in PySpark, SQL, dbt, and Fivetran. Design and optimize scalable, reliable data infrastructure across cloud and on-premise ecosystems (AWS, GCP, Azure) using Databricks, BigQuery, and Snowflake.…See more Manage end-to-end data sourcing, processing, integration, and delivery projects to meet business objectives and predetermined success criteria. Require 10+ years managing Data Engineering teams and 7+ years working with cloud-native data platforms and advanced SQL/Python/PySpark expertise.

Lead complex contact center routing initiatives end-to-end for Intuit's GBSG business unit, from stakeholder discovery through Amazon Connect configuration, deployment, and production monitoring. Design and execute advanced routing strategies using CUP/CUE platforms, manage multi-region Amazon Connect environments, and analyze routing performance across channels using QlikSense, Splunk, and Databricks.…See more Serve as a routing and platform subject matter expert, provide guidance on design tradeoffs, and manage P0 incidents within SLA. Proactively apply generative AI tools to accelerate analysis, configuration work, and team workflows.

Lead a data engineering team responsible for building scalable data pipelines, trusted datasets, and reusable data products powering analytics, experimentation, and AI across Scribd. This role blends technical leadership with hands-on architecture guidance; you'll establish engineering standards, drive design reviews, mentor engineers, and partner cross-functionally to translate business needs into production-grade data solutions.…See more Requires 10+ years in data engineering or data platforms, 3+ years leading engineering teams, deep expertise in dimensional modeling and scalable data architectures, and strong SQL plus Python/Scala skills. You'll work with modern cloud data platforms like Databricks, Snowflake, and BigQuery, and distributed processing frameworks like Spark.

RIVO is seeking a Senior Data Platform Engineer to architect and build a modern enterprise data platform supporting analytics, machine learning, and near real-time processing. You will design the foundational data warehouse or lakehouse architecture, modernize legacy ETL processes, implement CDC-based ingestion and low-latency pipelines, and establish data quality and governance frameworks.…See more This role requires 8+ years of data warehousing or platform engineering experience, expertise in cloud platforms (Snowflake, Databricks, Azure), SQL and Python proficiency, and the ability to support both analytics and data science workloads at enterprise scale.

Design, build, and maintain Python and SQL-based data transformation workflows using Databricks and PySpark for large-scale processing. Analyze datasets to independently identify actionable insights, translate ambiguous data problems into clear analytic solutions, and ensure data quality and governance across pipelines.…See more Collaborate with stakeholders and cross-functional teams to integrate data outputs into downstream applications, troubleshoot pipeline issues, and document architecture and processes.

Design and build Python and SQL-based data transformation workflows using Databricks and PySpark for large-scale processing. Analyze unstructured datasets to independently identify actionable insights, translate ambiguous data problems into clear analytic solutions, and ensure data quality and governance across pipelines.…See more Troubleshoot pipeline issues, document processes and architecture, and partner with cross-functional teams to integrate data outputs into downstream applications and dashboards.

A Forward Deployed AI Engineer who builds and deploys production AI/ML and Generative AI applications, particularly agentic workflows and RAG systems integrated with enterprise data and business systems. The role requires 6+ years of software/ML/AI engineering experience, advanced Python proficiency, hands-on expertise with LLMs, modern GenAI frameworks (LangChain, LangGraph, LlamaIndex), and cloud platforms (Azure, AWS, GCP).…See more You'll own solutions from prototype through production, working directly with product and business stakeholders to identify where AI creates value, troubleshoot integration issues, and drive continuous improvement.

Join Sony Interactive Entertainment's D2C Data Science team to support PlayStation's payment and subscription experiences through statistical analysis, A/B testing, and machine learning. You will own analyses of payment success, subscription recovery, and payment flow optimization, building models for forecasting, segmentation, and propensity while translating complex data into actionable recommendations for product and engineering teams.…See more The role requires strong SQL and Python skills, solid statistics and experimental design knowledge, and the ability to work independently on well-scoped analyses while communicating findings clearly to both technical and non-technical stakeholders.

Senior Enterprise Data Engineer role requiring 5+ years designing and developing OLTP/OLAP databases, ETL/ELT pipelines using SSIS, Azure Data Factory, or Python, and enterprise reporting solutions with Power BI and SSRS. Must have advanced SQL Server expertise, strong data warehousing and dimensional modeling knowledge, and experience in Agile environments.…See more Preferred qualifications include Azure data platforms (Synapse, Data Lake, Fabric), Databricks, Apache Spark, Azure DevOps, and CI/CD practices.

As an Architecture Consultant 3, you will design and review solution architectures for data engineering projects on AWS and Databricks, translating business requirements into scalable technical solutions. You'll assess technologies and platforms, develop integration strategies across systems, and resolve complex technical challenges during build and deployment phases.…See more You'll provide architectural guidance to development teams, identify system risks, and contribute insights to proposals and RFPs. The role requires expertise in solution architecture design, client engagement, and mentoring junior team members.

Lead a team of Databricks engineers delivering GenAI and LLM solutions to government clients. You'll architect and deploy production-scale data platforms, mentor engineers, and translate business requirements into AI solutions using Databricks' full stack (Lakehouse, Agent Bricks, Model Serving, Genie, Apps).…See more Enforce strong data management, CI/CD, testing, and documentation practices while working across AWS, Azure, and Google Cloud environments.

Senior Forward Deployed Engineer at Deloitte GPS building AI-enabled solutions and agentic platforms on Databricks for enterprise government clients. Requires 7+ years software/data engineering experience, 5+ years deploying GenAI/LLM solutions in production, and 5+ years hands-on Databricks expertise across Lakehouse, Agent Bricks, Model Serving, and Genie.…See more Mentors junior engineers, designs scalable AI patterns with human-in-the-loop controls, delivers production-quality code with strong CI/CD and testing practices, and translates complex business problems into deployable AI solutions.

Senior Software Engineer role at CoStar focused on designing and building production applications using C# and .NET on AWS cloud infrastructure. The position emphasizes leveraging Claude AI for code generation, refactoring, and analysis, while building and operating scalable, cloud-native solutions.…See more Requires strong AWS hands-on experience (EC2, ECS/EKS, Lambda, S3, RDS, DynamoDB, CloudWatch), API and backend service development, and solid software design principles. Preferred qualifications include Databricks experience, microservices/distributed systems knowledge, CI/CD and DevOps practices, and Infrastructure-as-Code tools like Terraform and CloudFormation.

Design, build, and maintain scalable data pipelines and platforms that power high-volume, data-driven applications. Develop and optimize ETL/ELT processes for large, complex datasets using Databricks and Spark, and build backend data services in C# / .NET.…See more Lead technical design decisions, mentor engineers, and leverage AI-assisted coding tools to improve engineering efficiency. Requires 5+ years of data engineering experience, strong expertise with data pipelines and ETL frameworks, and hands-on experience with AWS-based data platforms.

CoStar seeks a Senior Software Engineer to design and build production applications using C# and .NET on AWS, with hands-on experience leveraging Claude for AI-assisted development. The role focuses on building scalable, cloud-native solutions and involves building APIs, backend services, system design, code reviews, and mentoring.…See more Required skills include strong C# and .NET expertise, hands-on AWS experience (EC2, ECS/EKS, Lambda, S3, RDS, DynamoDB, CloudWatch), Git workflows, and software design principles. Preferred qualifications include practical Claude experience, Databricks, microservices, CI/CD pipelines, and infrastructure-as-code tools like Terraform or CloudFormation.

Staff Data Engineer at Illumina seeking a senior individual contributor with 10+ years building and scaling data products on Databricks and Snowflake. The role spans end-to-end ownership of data pipelines across Supply Chain, Manufacturing, and Quality domains, requiring advanced Python, SQL, and data modeling skills with hands-on expertise in Spark, Delta Lake, dbt, and distributed systems design.…See more Responsibilities include designing medallion-architecture data products, embedding governance and data quality, adopting AI in analytics workflows, and mentoring a global engineering team. The candidate will set technical standards, lead architecture decisions, and communicate trade-offs across business, AI, and platform stakeholders.

This Data Engineer II role focuses on data operations and infrastructure, supporting the transition of data systems from development to production. The engineer will administer data lakehouse/warehouse platforms (Snowflake, Databricks, Redshift, BigQuery, Azure Synapse), manage RBAC and user governance, monitor and troubleshoot data pipelines, and contribute to data governance and quality best practices.…See more Key responsibilities include orchestration tool administration (Apache Airflow, dbt), observability framework development, incident management, and hands-on SQL and Python work to maintain and optimize production data systems.

Principal Engineer leading the design, build, and operation of an enterprise data platform serving 50+ source systems across R&D, Commercial, Manufacturing, and other business functions. The role combines hands-on data engineering (PySpark, T-SQL, Python, Data Factory, medallion architecture) with technical leadership of data engineers, vendor management, platform standards enforcement, and cross-functional stakeholder partnership.…See more Responsibilities span CI/CD pipeline design, data quality and reliability instrumentation, semantic layer development, data governance implementation, and delivery of new source-system integrations at scale.

Design, build, and maintain scalable batch and streaming data pipelines on AWS and Databricks, developing reusable data engineering frameworks and standardized patterns for ingestion, transformation, and validation. Leverage AI-based techniques to automate data quality checks, anomaly detection, and pipeline self-healing.…See more Define SLIs/SLOs, participate in on-call rotations, and mentor junior engineers while driving adoption of data frameworks and best practices across analytics, reporting, and AI/ML teams.