B2B

Other

remote

Requirements

Experience: 3 years

Experience indata engineer

Experience inPython

Experience inAirflow

Experience in DevOps- will be a plus

English -Upper Intermediate

Description

On behalf with our customer we are seeking for Data Engineer Team Lead to join our global R&D department.

Our customer is an innovative technology company led by data scientists and engineers devoted to mobile app growth. They focus on solving the key challenge of growth for mobile apps by building Machine Learning and Big Data-driven technology that can both accurately predict what apps a user will like and connect them in a compelling way. We are looking for a data centric quality driven team leader focusing on data process observability. The person is passionate about building high-quality data products and processes as well as supporting production data processes and ad-hoc data requests. As a Data OPS TL, you will be in charge of the quality of service as well as quality of the data and knowledge platform for all data processes. You’ll be coordinating with stakeholders and play a major role in driving the business by promoting the quality and stability of the data performance and lifecycle and giving the Operational groups immediate abilities to affect the daily business outcomes.

Responsibilities:

Process monitoring — managing and monitoring the daily data processes; troubleshooting server and process issues, escalating bugs and documenting data issues.
Ad-hoc operation configuration changes — Be the extension of the operation side into the data process; Using Airflow and python scripting alongside SQL to extract specific client relevant data points and calibrate certain aspects of the process.
Data quality automation — Creating and maintaining data quality tests and validations using python code and testing frameworks.
Metadata store ownership — Creating and maintaining the metadata store; Managing the metadata system which holds meta data of tables, columns, calculations and lineage. Participating in the design and development of the knowledge base metastore and UX. In order to be the pivotal point of contact when needing information on tables, columns and how they are connected. I.e., What is the data source? What is it used for? Why are we calculating this field in this manner?

Requirements:

Over 2 years in a leadership role within a data team.
Over 3 years of hands-on experience as a Data Engineer, with strong proficiency in Python and Airflow.
Solid background in working with both SQL and NoSQL databases and data warehouses, including but not limited to MySQL, Presto, Athena, Couchbase, MemSQL, and MongoDB.
Bachelor’s degree or higher in Computer Science, Mathematics, Physics, Engineering, Statistics, or a related technical discipline.
Highly organized with a proactive mindset.
Strong service orientation and a collaborative approach to problem-solving.

Nice to have skills: