Big Data Engineer
RemoteSan Diego Metropolitan Area · RemoteLast seen today
Summary
Design, build, and operate large-scale data processing pipelines and analytics platforms using Hadoop and related big-data ecosystems with 6+ years of experience. Ingest, transform, and analyze massive volumes of structured and unstructured data to support enterprise analytics, machine learning, and reporting workloads. Required expertise includes Apache Spark (Scala, Python, or Java), Hive, HDFS, Kafka or Spark Streaming, SQL, workflow orchestration (Airflow or Oozie), and distributed systems fundamentals. Deliver reliable, performant, and cost-effective data platforms in production environments.
Data ScienceDevOps / InfrastructureAWSAzureJavaKubernetesPythonSQLApache AirflowApache AtlasApache FlinkApache KafkaApache SparkAWS EmrAzure HdinsightCI CDCollibraDatabricksDeltaDistributed SystemsHadoopHbaseHdfsHiveHudiIcebergInfrastructure As CodeNoSQLOozieScalaShell ScriptingSpark On K8sSpark StreamingSqoopTrino