Lead Big Data Engineer (Databricks + AWS)

REMOTE GLOBAL | SOFTSERVE

Company

SoftServe delivers engineering and technology services for enterprise clients.

What you will do

• Design, build, and optimize scalable AWS data solutions using Databricks, including Lakehouse architectures with Delta Lake and Unity Catalog.
• Develop batch and real-time data processing using Apache Spark, Flink, Kafka, Amazon MSK, and Kinesis.
• Lead data integration and migration (source-to-target mapping, ingestion, transformation, and data quality assurance).
• Drive data platform architecture, data modeling, and engineering best practices for scalability, reliability, and long-term maintainability.
• Build and manage automated workflows and data pipelines using Databricks Workflows, Apache Airflow, MWAA, Snowflake, and Amazon Redshift.
• Mentor data engineering teams and translate business requirements into technical solutions, roadmaps, and delivery plans.

Requirements

• Proven experience as a Lead Data Engineer delivering scalable data platforms and pipelines.
• Hands-on expertise in batch and real-time processing with Apache Spark, Flink, Kafka, Amazon MSK, or Kinesis.
• Strong AWS and Databricks experience, including Delta Lake, Unity Catalog, Workflows, and Jobs.
• Advanced SQL plus proficiency in Python (preferred), Scala, or Java.
• Experience orchestrating data workflows with Databricks Workflows, Apache Airflow, or MWAA.
• Upper-intermediate or higher English.

Culture & Benefits

• Remote/office work model.
• Collaborative, innovation-driven environment across the full solution lifecycle.
• Opportunity to explore emerging technologies and share knowledge within the engineering community.
• Leadership role with mentoring and team enablement.

Hiring process

• Submit the application form.
• Recruiter follow-up and interview steps.

APPLY