CONTRACTOR

Data Engineer (Big Data) - SII Belgium

Brussels BE
2022-07-05
2022-07-31

Role Description

Our client is looking for a data egineer to join the team. In their department they are delivering the data warehouse and big data solutions (based on a Cloudera stack) for internal and external customers.
More than 10 data engineering teams are working on these platforms and an important aspect is to ensure these teams have a common way for scheduling and orchestration of ETL jobs. Therefore we have developed a custom application that offers metadata driven scheduling, orchestration and monitoring capabilities. You will be part of the team that is adding features to this tool.
Next to this you will also be involved in other data engineering tasks on the big data platform.
 

Vereisten

Skills:

 You have at least 2 years of working experience in developing data ingestion, data processing and analytical pipelines for big data, relational databases and data warehouse solutions and deploying ETL solutions on premises and/or cloud
 Deep knowledge of coding in Python, PySpark or any other additional programming language as Java and Linux OS (e.g. bash)
 Deep knowledge of working with various kinds of file formats like CSV, XML, JSON, Apache Parquet etc.,
 Deep knowledge of SQL
 Knowledge of version control tools like GitLab and CI/CD processes
 Have a degree in IT/Engineering/Sciences or a strong relevant technical experience
 An added value is if you have experience with Cloudera technology stack (HDFS, Hive, Spark, Atlas, Ranger, Kafka, Nifi, …), preferable both in batch and real-time use cases
 You are fluent in English, both spoken and written. Knowledge of Dutch and/or French is an asset.