apache-flink jobs in San Diego

Design, build, and maintain real-time data ingestion pipelines that reliably flow streaming data from multiple sources into a data platform while ensuring quality, observability, and scalability. Monitor data health, develop resilient pipelines with proper error-handling and backpressure strategies, and automate monitoring of real-time feeds to alert on timeliness, volume, and distribution issues.…See more Collaborate with data, platform, and software engineers to optimize pipeline configuration, perform root-cause analysis on data outages, and work with security teams on encryption and data classification. Requires 3+ years in data engineering, data operations, or DevOps roles; proficiency with Python and bash; and hands-on experience with tools like Kafka, NiFi, Spark Streaming, Snowflake, Elasticsearch, Grafana, and Prometheus.

As an IC5 Senior Staff Engineer, you will architect and deliver large-scale distributed data platform components centered on Kafka, Apache Iceberg, and Apache Spark. You will lead complex technical initiatives, design high-performance data ingestion pipelines, and define engineering best practices across the organization.…See more The role demands deep expertise in distributed systems, JVM performance tuning, stream processing, and full-stack Data Lake solutions, with hands-on delivery of production systems and mentorship of engineering teams.

Design, build, and maintain real-time data ingestion pipelines that reliably stream data from diverse sources into a production data platform. You will ensure data quality, observability, and scalability while monitoring pipeline health, building resilient systems with proper error handling and backpressure strategies, and automating monitoring and alerting for timeliness and data issues.…See more Partner with data engineers, platform engineers, and analytics teams to configure pipelines for reliability, perform root cause analysis on outages, and work with security teams on encryption and data governance. Proficiency required in Python and bash scripting, plus hands-on experience with tools like Kafka, NiFi, Flink, Spark, Snowflake, Grafana, and Prometheus.

Design, build, and maintain real-time data ingestion pipelines that reliably stream data from multiple sources into a centralized data platform. You will ensure data quality, observability, and low-latency delivery while collaborating with data engineers, platform engineers, and analytics teams.…See more Responsibilities include building resilient pipelines with error handling, automating monitoring and alerting for data issues, performing root cause analysis on outages, and partnering with security teams on encryption and data governance. You must demonstrate proficiency in Python and bash scripting, familiarity with tools like Kafka, NiFi, Spark Streaming, Snowflake, and Elasticsearch, and 3+ years of experience in data engineering or DevOps roles supporting production pipelines.

SHEIN is seeking a Senior Data Engineer to build and productionize GenAI/LLM solutions for data engineering workflows, including code assistance, metadata discovery, and knowledge access. The role focuses on developing RAG and agentic workflows, automating incident triage and log analysis, and integrating AI capabilities into internal developer tools and services.…See more You'll own scoped projects from problem definition through production support, requiring 3+ years of hands-on production experience with strong Python/SQL fundamentals, proven GenAI/LLM application development, and practical knowledge of databases, APIs, data pipelines, and distributed systems.

Staff Data Engineer role at Credit Karma building petabyte-scale data infrastructure and streaming platforms on GCP. You will design and operate Kafka-based streaming infrastructure, build cloud-native data pipelines using Dataflow/Beam, Flink, and Spark, and create persistence frameworks across Spanner, MySQL, and BigQuery.…See more The role requires 7+ years backend/data systems experience in JVM languages (Scala/Java), expertise with high-throughput distributed systems, and deep knowledge of streaming platforms, Apache Beam, CDC patterns, encryption, and data governance frameworks. You'll also integrate AI/ML and generative AI technologies including LLMs, RAG, semantic search, and knowledge graphs into the data platform.

Senior Data Engineer role focused on building and productionizing GenAI/LLM solutions for data engineering workflows. You will develop RAG and agentic workflows, automate incident triage and root-cause analysis, integrate AI capabilities into internal developer tools, and own projects from problem definition through production support.…See more Required: 3+ years of production systems experience, strong Python/SQL, hands-on GenAI/LLM application building, and software engineering fundamentals including testing, CI/CD, and monitoring.

Staff Software Engineer leading the design, architecture, and operations of PlayStation's Real-Time Analytics Platform (RTAP), a large-scale distributed data system handling high-throughput, low-latency analytics and stream processing. The role involves architecting and evolving streaming and analytics platforms using Apache Flink, Spark, ClickHouse, and Druid; integrating batch and streaming workloads via lakehouse architectures; and embedding AI capabilities to automate engineering workflows and improve platform reliability.…See more You will own critical platform capabilities end-to-end from design through production operations, mentor engineers, establish technical standards, and influence cross-functional technical strategy across PlayStation's data infrastructure.

Staff Software Engineer leading design and evolution of PlayStation's Real-Time Analytics Platform (RTAP), a large-scale distributed data system handling high-throughput, low-latency analytics and stream processing. Responsibilities include architecting stream processing pipelines, optimizing data services using Apache Flink, Spark, ClickHouse, and Druid, integrating batch and streaming workloads with lakehouse architectures, and building AI-powered capabilities for platform automation.…See more The role requires deep expertise in distributed systems, real-time analytics, and mentoring engineering teams while establishing technical standards and best practices across the organization.

Staff-level engineer designing and operating large-scale distributed data platforms powering real-time analytics across PlayStation. You will lead the architecture and evolution of the Real-Time Analytics Platform (RTAP), own critical capabilities end-to-end from design through production operations, and mentor engineers.…See more Key responsibilities include building and optimizing distributed services using Apache Flink, Spark Structured Streaming, ClickHouse, and Apache Druid; integrating batch and streaming workloads with lakehouse architectures; and establishing technical standards for distributed systems, stream processing, and data modeling.

Design, build, and scale ETL/ELT pipelines and cloud data warehouse infrastructure using Python, SQL, and modern orchestration tools like Apache Airflow or Prefect. Own the end-to-end data infrastructure that powers business intelligence and operational reporting, including data quality checks, anomaly detection, and performance optimization.…See more Collaborate with analytics and software engineering teams to define data modeling standards and maintain infrastructure-as-code deployments using Terraform and CI/CD. Requires 3–6 years of data engineering or backend development experience with advanced SQL, Python, and hands-on work with cloud data warehouses like Snowflake or BigQuery.