Data engineer / Lamoda

What you will work on:

• Design and build next generation best on market e-commerce data platform;
• Build and maintain reliable and scalable enterprise level distributed transactional data processing systems for scaling the existing business and supporting new business initiatives;
• Optimize jobs to utilize Kafka, Hadoop, Spark resources in the most efficient way;
• Monitor and provide transparency into data quality across systems (accuracy, consistency, completeness, etc);
• Increase accessibility and effectiveness of data (work with analysts, data scientists, and developers to build/deploy tools and datasets that fit their use cases);
• Perform code review;
• Provide mentorship and guidance to junior team members.

Team responsibilities:

• Performance tuning, monitoring, capacity planning of Hadoop cluster and other infrastructure;
• Ingest, validate and process internal & third party data;
• Create, maintain and monitor data flows for consistency, accuracy and lag time;
• Maintain and enhance framework for jobs (Spark Streaming, Spark, MapReduce, aggregate jobs in Hive);
• Train Developers/Analysts on tools to pull data;
• Tool evaluation/selection/implementation;
• Review/Approval - DDL for database, Hive Framework jobs and Spark Streaming to make sure they meet our standards.

We expect:

• Experience in Hadoop stack (YARN, HDFS, HBase, Hive);
• Experience in Spark;
• Knowledge of version control systems (git);
• Experience in Python;
• Strong understanding of RDBMS, SQL;
• Experience in Docker, Kubernetes is a plus;
• Passion for engineering and computer science around data.

We offer:

• Official employment, social package;
• Competitive salary;
• Flexible working hours, it is possible to plan working and personal time, remote employment;
• Challenging and engaging atmosphere in a team of professionals;
• Discounts on the entire range of products;
• Education of our employees: we share knowledge within the company, organizing internal meetings and taking part in external events;
• Attending at the conferences as listeners and speakers (for those who want to prepare professionally for presentations with reports or presentations we have Speakers Club);
• Relocation package for candidates living outside Moscow and Moscow region;
• A comfortable office in Moscow (MCC Khoroshevo / m. Polezhaevskaya) with activities like table tennis, kicker, gym and yoga.

More here: https://hh.ru/vacancy/44189104