stream-processing jobs in San Diego

Design, build, and operate the data and state management layer for an AI platform serving cloud, on-prem, edge, and air-gapped environments. Own production relational and key-value stores (PostgreSQL, Redis), vector and graph databases (Qdrant, Milvus, PGVector, Neo4j), object storage (MinIO, OpenEBS), event streaming (Kafka, NATS), and CDC pipelines (Debezium, Flink).…See more Ensure data durability, consistency, recoverability, multi-tenancy, and schema governance across all deployment contexts, serving as the reliability and observability substrate for downstream AI capabilities.

As a Senior Staff Platform Engineer (IC5), you will architect and deliver large-scale distributed systems and data platform components, with emphasis on scalability, operational excellence, and engineering best practices. You'll lead complex technical initiatives, provide technical leadership for high-risk projects, and research/adopt new technologies to enhance platform capabilities.…See more The role requires 12+ years of hands-on experience in distributed systems, data platforms, or large-scale backend infrastructure, with deep expertise in Java, JVM performance tuning, Kubernetes, Apache Kafka/Flink/Spark, and data formats like Parquet and Avro. You'll also be expected to leverage AI-powered tools and reason about how AI changes software and infrastructure engineering.

Sr Staff Software Engineer leading architecture and design across ServiceNow's data fabric platform, including Trino connectors, Kafka streaming, ETL pipelines, and data quality. Requires 12+ years building and operating large-scale distributed systems, deep expertise in query processing, data integration, CDC, streaming pipelines, and Kubernetes-based multi-tenant SaaS.…See more Responsibilities include writing production code, leading cross-team technical reviews, defining performance and scalability strategy, and serving as senior escalation point for complex production issues. Must demonstrate strong hands-on coding in Java, Go, C++, or Scala, with proven track record of leading architecture across multiple teams and influencing without authority.

Senior Software Engineer role focused on building scalable payment infrastructure and banking integrations. Responsibilities include designing and implementing core platform services and APIs, building secure integrations with financial institutions, ensuring high availability and system reliability, and collaborating across teams on fintech solutions.…See more Requires 5+ years of backend systems experience with strong proficiency in Java, Spring, Kubernetes, microservices architecture, and cloud platforms (AWS, GCP, or Azure), plus understanding of payments processing, financial compliance, and security best practices.

As an IC5 Senior Staff Engineer, you will architect and deliver large-scale distributed data platform components centered on Kafka, Apache Iceberg, and Apache Spark. You will lead complex technical initiatives, design high-performance data ingestion pipelines, and define engineering best practices across the organization.…See more The role demands deep expertise in distributed systems, JVM performance tuning, stream processing, and full-stack Data Lake solutions, with hands-on delivery of production systems and mentorship of engineering teams.

Own and deliver small to medium machine learning system components from design through production deployment, including building data pipelines, training and evaluating models, and implementing MLOps monitoring. Write high-quality Python code to translate technical requirements into maintainable solutions, working with frameworks like scikit-learn, TensorFlow, PyTorch, and HuggingFace.…See more Design and deploy ML models as microservices, APIs, batch jobs, or streaming components on AWS, with responsibility for model performance metrics, data drift detection, and retraining triggers. Collaborate across Data Engineers, Software Engineers, Data Scientists, and product stakeholders to deliver project objectives.

Staff Software Engineer leading the design, architecture, and operations of PlayStation's Real-Time Analytics Platform (RTAP), a large-scale distributed data system handling high-throughput, low-latency analytics and stream processing. The role involves architecting and evolving streaming and analytics platforms using Apache Flink, Spark, ClickHouse, and Druid; integrating batch and streaming workloads via lakehouse architectures; and embedding AI capabilities to automate engineering workflows and improve platform reliability.…See more You will own critical platform capabilities end-to-end from design through production operations, mentor engineers, establish technical standards, and influence cross-functional technical strategy across PlayStation's data infrastructure.

Staff Software Engineer leading design and evolution of PlayStation's Real-Time Analytics Platform (RTAP), a large-scale distributed data system handling high-throughput, low-latency analytics and stream processing. Responsibilities include architecting stream processing pipelines, optimizing data services using Apache Flink, Spark, ClickHouse, and Druid, integrating batch and streaming workloads with lakehouse architectures, and building AI-powered capabilities for platform automation.…See more The role requires deep expertise in distributed systems, real-time analytics, and mentoring engineering teams while establishing technical standards and best practices across the organization.

Staff-level engineer designing and operating large-scale distributed data platforms powering real-time analytics across PlayStation. You will lead the architecture and evolution of the Real-Time Analytics Platform (RTAP), own critical capabilities end-to-end from design through production operations, and mentor engineers.…See more Key responsibilities include building and optimizing distributed services using Apache Flink, Spark Structured Streaming, ClickHouse, and Apache Druid; integrating batch and streaming workloads with lakehouse architectures; and establishing technical standards for distributed systems, stream processing, and data modeling.