Big Data Engineer

n   Work with a cross-functional team to design, develop, and implement enterprise-level data architectures and solutions using cloud-based big data and AI technologies.

n   Create and maintain data models, supporting ETL and ELT development processes to ensure smooth data flow across various systems.

n   Collaborate with clients to understand their data requirements and deliver tailored solutions that meet their specific needs.

n   Design, build, and maintain data pipelines for processing and analyzing large-scale datasets using distributed computing frameworks.

n   Optimize data storage and processing for high performance, scalability, and reliability.

n   Implement data governance practices to ensure data quality, consistency, and security.

n   Work with relational and non-relational databases, and MPP Data Warehouses, to build and maintain robust data storage solutions.

n   Hands-on experience with complex SQL queries, data manipulation, and data analytics.

n   Stay up-to-date with emerging technologies in big data and AI fields, providing recommendations for improvement and innovation.

Requirements

n   2+ years of hands-on experience in Big Data and AI engineering, including data modeling, ETL / ELT development, and advanced SQL queries.

n   Strong proficiency in working with distributed computing technologies such as Spark, Flink, and Hadoop ecosystem.

n   Experience with both relational and non-relational databases, such as PostgreSQL, HBase, and MongoDB.

n   Solid understanding of cloud-based big data and AI services, with hands-on experience using technologies such as Elasticsearch, Kafka, TensorFlow, PyTorch, Keras, and others.

n   Experience with MPP Data Warehouses and their integration with various data processing tools.

n   Proficient in Python, SQL, and other relevant programming languages for data engineering.

n   Knowledge of data governance practices, data quality, and ensuring compliance with best practices.

n   Ability to troubleshoot and optimize complex data processing tasks in a cloud environment.

n   Strong communication skills, both verbal and written, to effectively collaborate with stakeholders and present technical solutions.

Benefits

n   Cloud vendor certifications (Huawei Cloud or other cloud providers) in Big Data or AI technologies.

n   Experience in delivering projects in an international setting, with flexibility to travel abroad for onsite delivery.

n   Familiarity with advanced analytics, machine learning algorithms, and artificial intelligence concepts.