spark jobs in San Diego

Lead Forward Deployed Engineer responsible for architecting and delivering LLM-enabled applications (copilots, agentic workflows, assistants) using enterprise AI platforms like Claude, Gemini, or GPT-4o. Drive end-to-end RAG pipeline design, prompt engineering, evaluation frameworks, and data foundations while mentoring junior engineers and enforcing delivery standards across multi-pod engagements.…See more Require 7+ years software/data engineering experience, 1+ year hands-on GenAI/LLM production deployment, and proven ability to lead project workstreams and translate business problems into AI solutions.

Staff-level backend engineer responsible for designing and maintaining statistical capacity models that forecast infrastructure resource needs (CPU, memory, connections, ElasticCache, DynamoDB, AuroraDB) to support traffic demands during peak sales events. Sets technical strategy for the team, collaborates across product and analytics to ensure system sustainability, owns operational excellence including monitoring and on-call processes, and establishes code review and design standards.…See more Requires 8+ years building highly available distributed systems at scale using Python or Kotlin, AWS, MySQL, Spark, and Kubernetes, with proven experience scaling systems for major sales events like Prime Day and Black Friday.

Forward Deployed Engineer who embeds with clients to identify business needs and translate high-value GenAI use cases into production solutions. Build AI-enabled solutions, agentic platforms, and workflows across enterprise AI platforms, delivering production-quality code with strong testing, CI/CD, logging, and documentation practices.…See more Lead working sessions, prototype solutions, and mentor team members while balancing architecture decisions around quality, safety, latency, cost, and model risk. Requires 3+ years software/data engineering experience, 1+ years hands-on GenAI/LLM deployment, and 1+ years with Microsoft AI & Data (Azure AI Foundry).

A forward-deployed engineer who embeds with enterprise clients to identify business needs and build production-grade AI solutions using Palantir platforms (Foundry, AIP, Maven). The role combines solution engineering—developing agentic AI systems, LLM-powered workflows, and scalable patterns with strong testing and CI/CD practices—with client engagement and stakeholder alignment.…See more Requires 3+ years of software/data engineering experience, 1+ years of hands-on GenAI/LLM deployment, and 1+ years working with Palantir; involves 50% travel and mentoring of junior team members.

Principal Engineer Data will develop and execute a data architecture strategy aligned with business goals, building and maintaining data models, metadata standards, and integration frameworks. The role requires leading technical assessments for digitizing core business functions, working with stakeholders to translate requirements into architectural solutions, and ensuring data storage, retrieval, and governance meet enterprise standards.…See more The candidate must have a STEM degree with at least 5 years of experience (or Master's with 3 years), strong expertise in SQL Server, data visualization tools (Tableau, Power BI), and data architecture principles including data integration, governance, and security protocols.

Lead a team of data scientists at Insulet's Analytics Center of Excellence, managing the execution of global analytics and AI initiatives while maintaining hands-on expertise in data science work. You will mentor scientists, guide development of cloud-based analytics pipelines and automated machine learning solutions, and coordinate with cross-functional stakeholders across R&D, clinical, quality, and manufacturing to identify and deliver high-value data science opportunities.…See more The role requires a Master's degree with 5+ years in data science (or PhD with 3+), 2+ years in machine learning/AI, direct team management experience, and proficiency in Python/R, SQL, cloud platforms (Azure/AWS/GCP), and big data technologies like Spark/Databricks. You'll perform statistical analysis, predictive modeling, and communicate insights to executives while owning the data products portfolio and managing annual budgets.

Develop scalable AI engineering patterns and production-quality code for government clients, balancing quality, safety, latency, cost, and model risk. Apply strong practices in testing, CI/CD, logging, versioning, and documentation while designing extensible functionality aligned with senior team members.…See more Requires 4+ years of software/data engineering experience, 1+ year building and deploying GenAI or LLM solutions, and 1+ year with Google Cloud (Gemini API, Vertex AI Agent Builder, Grounding, or Google Workspace integration). Lead project workstreams, translate business problems into AI solutions, and contribute reusable assets including code, prompt libraries, runbooks, and reference implementations.

This role combines Business Intelligence development with Navy ERP (SAP) functional expertise to design and sustain reporting solutions, dashboards, and automated data pipelines for Navy financial operations. The developer will use Databricks, Qlik, Tableau, SQL, Spark, and Python to build analytics tools and data models while applying deep functional knowledge of SAP modules (FM, FI, CO, PS, MM, WM, SD, WFM) and financial business rules.…See more Responsibilities include developing executive dashboards, creating automated data delivery workflows via AWS S3 and email, integrating data from multiple systems (Jupiter, Navy ERP, Oracle), and supporting user acceptance testing and stakeholder training. The role requires hands-on BI development, SAP functional analysis, application development using NetWeaver and ABAP, and collaboration across functional teams to ensure data quality and audit compliance.

The Data Integrator will design and optimize SQL queries, build advanced DAX reports in Power BI, and integrate data from multiple APIs, databases, and files at scale. The role requires 3+ years of data analysis and validation experience with Power BI, DAX, Power Query, Azure Data Flow, and SQL, plus strong communication skills to gather requirements and deliver solutions to end users.…See more The person will maintain these data pipelines and coordinate with management throughout the development lifecycle using modern Azure and data tools.

Design, build, and operate enterprise-scale Databricks lakehouse platforms for a global semiconductor company. Lead AI-native development practices, machine learning enablement, and AIOps automation while mentoring engineering teams and establishing technical standards.…See more Drive platform modernization, cloud-native practices, and intelligent data/AI infrastructure at scale. Requires 4–5 years of hands-on Databricks experience, strong Python/SQL/Spark skills, MLOps expertise, and proven technical leadership.

Lead the design, build, and operation of enterprise-scale Databricks Lakehouse platforms at Qualcomm, driving AI-native development, MLOps practices, and cloud modernization. This hands-on Staff IT Engineer role requires 4–5 years of Databricks experience and 8–12 years total IT background, with responsibility for platform architecture (Delta Lake, Unity Catalog, Workflows), machine learning enablement (MLflow, Mosaic AI), AIOps automation, and technical mentoring of engineers.…See more You will establish engineering standards, optimize platform performance and cost, implement observability and self-healing automation, and drive adoption of generative AI tools and intelligent automation across development teams.

Design, build, and operate production ML systems across the full lifecycle—from data preparation and model training through deployment, monitoring, and continuous improvement. Partner with data scientists, software engineers, and platform teams to turn research prototypes into reliable, scalable services.…See more Work across recommendation, ranking, forecasting, classification, NLP, and generative AI depending on product priorities. Balance model quality, inference latency, scalability, and operational resilience as equally important outcomes.

Staff Data Engineer to architect and build fault-tolerant, scalable big-data platforms supporting high-velocity analytics use cases across Intuit. Responsibilities include designing data normalization, lineage, governance, and discoverability systems; collaborating with product and engineering teams on extensible platform design; and enabling analytics and GenAI/ML initiatives.…See more Requires 8+ years in product/web analytics, advanced SQL and big-data technologies (Redshift, Spark, Hive, BigQuery), Python or Java/Scala, and data pipeline orchestration experience. Role includes code reviews, CI/CD process ownership, mentorship, and translating business strategy into data-driven insights.

Lead a Data Engineering team to establish modern DataOps and CI/CD practices while providing technical mentorship in PySpark, SQL, dbt, and Fivetran. Design and optimize scalable, reliable data infrastructure across cloud and on-premise ecosystems (AWS, GCP, Azure) using Databricks, BigQuery, and Snowflake.…See more Manage end-to-end data sourcing, processing, integration, and delivery projects to meet business objectives and predetermined success criteria. Require 10+ years managing Data Engineering teams and 7+ years working with cloud-native data platforms and advanced SQL/Python/PySpark expertise.

Staff-level backend engineer responsible for setting technical strategy and owning search and recommendation systems at scale. You'll design and launch highly available distributed systems using Python or Kotlin, collaborate across product/design/analytics to balance technical sustainability with business impact, and lead a team through code review standards, mentorship, and operational excellence.…See more The role requires 8+ years of backend systems experience, 2+ years in search/recommendations, and proficiency with AWS, MySQL, Kubernetes, and modern architecture patterns.

Lead a team of Databricks engineers delivering GenAI and LLM solutions to government clients. You'll architect and deploy production-scale data platforms, mentor engineers, and translate business requirements into AI solutions using Databricks' full stack (Lakehouse, Agent Bricks, Model Serving, Genie, Apps).…See more Enforce strong data management, CI/CD, testing, and documentation practices while working across AWS, Azure, and Google Cloud environments.

Staff Data Engineer at Illumina seeking a senior individual contributor with 10+ years building and scaling data products on Databricks and Snowflake. The role spans end-to-end ownership of data pipelines across Supply Chain, Manufacturing, and Quality domains, requiring advanced Python, SQL, and data modeling skills with hands-on expertise in Spark, Delta Lake, dbt, and distributed systems design.…See more Responsibilities include designing medallion-architecture data products, embedding governance and data quality, adopting AI in analytics workflows, and mentoring a global engineering team. The candidate will set technical standards, lead architecture decisions, and communicate trade-offs across business, AI, and platform stakeholders.

Senior engineer at Deloitte GPS delivering production-quality code for GenAI/LLM-powered solutions, with 7+ years in software/data engineering and 1+ years hands-on experience building and deploying Claude-based applications. Responsibilities include designing extensible functionality, leading project workstreams, translating business problems into AI solutions, and contributing reusable assets (code, prompt libraries, runbooks).…See more Strong emphasis on testing, CI/CD, logging, documentation, and mentoring junior team members in fast-paced, ambiguous client delivery environments.

Lead a Palantir Forward Deployed Engineering team for the Government & Public Services practice, guiding architecture of data pipelines powering GenAI use cases. You will contribute production-quality code, enforce strong data management and CI/CD practices, and translate business problems into AI solutions while working with federal/state/local government clients.…See more The role requires 10+ years of software/data engineering experience, 1+ years of hands-on GenAI/LLM deployment, and 1+ years working with Palantir (Foundry, AIP, or Maven), with deep familiarity in AWS, Azure, and/or Google Cloud environments.

Lead a forward-deployed engineering pod (2–5 engineers) architecting and delivering LLM-enabled applications, copilots, and agentic workflows using enterprise AI platforms, particularly Microsoft Azure AI Foundry. Own end-to-end GenAI solution design—from RAG pipeline governance and prompt engineering to production deployment and quality evaluation—while mentoring junior engineers, enforcing delivery standards, and translating business problems into scalable AI systems.…See more Require 7+ years of software/data engineering experience, 1+ years of hands-on GenAI/LLM deployment in production, and proven ability to lead technical workstreams in ambiguous, fast-paced client environments.

A forward-deployed engineer role at Deloitte's Government & Public Services practice focused on designing, building, and deploying GenAI/LLM-powered solutions on Databricks for federal and public-sector clients. The role requires 4+ years of software/data engineering experience, 2+ years hands-on with GenAI/LLM solutions in production, and 2+ years with Databricks platform technologies.…See more Responsibilities include delivering production-quality code with strong CI/CD and documentation practices, designing extensible functionality, leading project workstreams, and translating business problems into AI solutions while traveling approximately 50%.

Senior Forward Deployed Engineer responsible for building and deploying production-grade AI solutions using frontier GenAI platforms (Claude, Gemini, GPT-4o) in client environments. Will lead end-to-end solution delivery, including architecture decisions, demo/POC development, and client enablement while mentoring junior engineers.…See more Requires 5+ years software/data engineering experience, 1+ years hands-on GenAI/LLM development, and demonstrated ability to translate business problems into scalable AI workflows with strong engineering practices (CI/CD, testing, documentation).

A Senior Site Reliability Engineer at SHEIN will own and operate mission-critical, large-scale distributed systems (Kubernetes, Kafka, Elasticsearch, Redis, APISIX, Nginx) running 24/7/365, participating in on-call rotations and driving fast incident response using AI-assisted log analysis and anomaly detection. The role requires strong software engineering expertise in Python or Go, deep Linux and networking knowledge, and hands-on experience with observability platforms (Prometheus, Grafana) and configuration management tools.…See more Responsibilities include designing resilient monitoring and alerting infrastructure, automating operational workflows to eliminate toil, capacity planning, and collaborating with global teams to improve system reliability and performance.

Senior Forward Deployed Engineer responsible for building and deploying GenAI/LLM-powered solutions on AWS, working directly with enterprise clients to translate business needs into production AI systems. The role requires 5+ years of software/data engineering experience, 1+ years hands-on with AWS AI&Data services (Bedrock, Neptune, OpenSearch), and the ability to lead technical workstreams while mentoring team members.…See more You will prototype solutions, develop scalable AI patterns with human-in-the-loop controls, write production-quality code with strong testing and CI/CD practices, and create reusable assets including code libraries and reference implementations. 50% travel required.

Senior/Staff Data Engineer responsible for building production data pipelines, APIs, and BI tools that serve Agoda's contact center operations. Will design and maintain dashboards, metrics, and analytics platforms using SQL, Python, Java, Golang, or TypeScript; work with data warehouses like BigQuery, Snowflake, or Redshift; and orchestrate ETL/ELT workflows using tools such as Airflow or dbt.…See more At Staff level, defines roadmap for key data platforms, mentors junior engineers, and influences cross-team data architecture to scale with business growth.