Description

INTEGER, a technology consulting company with over a decade of presence in the Portuguese market, is looking for a PySpark Engineer with strong communication skills and a sense of responsibility and teamwork.

MANDATORY SKILLS: ETL pipeline, Python programming, scripting with PySpark/Scala, Docker

ROLE DESCRIPTION/KEY DELIVERABLES
•Senior experience on ETL pipeline development, Python programming, scripting with PySpark/Scala, Docker etc.

• Strong analytical skills who can analyse the data inconsistencies and derive meaningful insights from the large data.

• Good knowledge on Hadoop Architecture and its ecosystem.

• Experience in optimizing Hive SQL queries and Spark Jobs.

• Problem-solving skills including knowledge of data structure and algorithms.

• Experience on agile software development process.

• Spark experience using python on Dataproc/Cloudera is desired.

• Like to collaborate within engineering and beyon

• Fluent in written and spoken English

If you have the amazINg talent and want to be part of the INTEGER team, answer this announcement with your updated CV to [email protected]