Data Engineer (AWS)

The CHI Software team is not standing still. We love our job and give it one hundred percent of us! Every new project is a challenge that we face successfully. The only thing that can stop us is... Wait, it’s nothing! The number of projects is growing, and with them, our team too.

Now we are looking for Senior Data Engineer.

Project Description:

It is a real-time data processing and analytics solution for a high-traffic web application.Tech stack.AWS: AWS Glue Studio, Redshift, RDS, Airflow, AWS Step Function, Lambda, AWS Kinesis, Athena, Apache Iceberg, AWS Data Brew, S3, OpenSearch, Python, SQL, CI/CD, dbt, Snowflake

Responsibilities:

• Design a scalable and robust AWS cloud architecture;
• Utilize AWS Kinesis for real-time data streaming and aggregation;
• Implement AWS Lambda for serverless data processing, reducing operational costs;
• Configure AWS RDS (Relational Database Service) for structured data storage and AWS DynamoDB for NoSQL requirements;
• Ensure data security and compliance with AWS IAM (Identity and Access Management) and encryption services;
• Develope and deployed data pipelines using AWS Glue for ETL processes;
• Write Python scripts and SQL queries for data transformation and loading;
• Set up continuous integration and continuous deployment (CI/CD) pipelines using AWS CodePipeline and CodeBuild;
• Monitore system performance and data quality using AWS CloudWatch and custom logging solutions;
• Collaborate with other teams to integrate data sources and optimize data flow;
• Achieve a highly scalable real-time data processing system, resulting in a 40% increase in data analysis efficiency and a significant reduction in operational costs.
• Build ETL pipelines from S3 to AWS OpenSearch by AWS Glue

Requirements:

• Proven experience designing scalable and reliable AWS cloud architectures
• Hands-on experience with AWS Kinesis for real-time data streaming and aggregation
• Strong knowledge of AWS Lambda and serverless data processing
• Experience working with AWS RDS (relational databases) and AWS DynamoDB (NoSQL)
• Solid understanding of AWS IAM, data security, and encryption best practices
• Practical experience building and deploying ETL pipelines using AWS Glue
• Experience creating data pipelines from Amazon S3 to AWS OpenSearch using AWS Glue
• Strong proficiency in Python for data processing and automation
• Advanced SQL skills for data transformation and loading
• Experience setting up and maintaining CI/CD pipelines using AWS CodePipeline and CodeBuild
• Experience monitoring system performance and data quality using AWS CloudWatch and custom logging solutions
• Ability to collaborate effectively with cross-functional teams to integrate data sources and optimize data flows
• Experience building highly scalable, real-time data processing systems
• Upper-Intermediate (B2) or higher English level

Our perks

• Work and learn from great minds by joining a community of inspiring colleagues
• Put your passion to work in a purposeful organisation dedicated to creating impact in a region with a lot of untapped potential
• Explore new opportunities to learn and grow every day
• Covered vacation period: 20 business days and 8 days off
• Free English classes.
• Flexible working schedule
• Truly friendly and supporting atmosphere
• Working remotely or in one of our offices