Description

Complex problems require the right expertise! Today one of the biggest challenges is data storage and processing, across these three main domains (3Vs): Volume, Velocity and Variety.

At Xpand IT, the Big Data technological area develops and implements architectures and software solutions that represent the state of the art on the capture, ingestions, process, enrich, storage and management of critical data from huge clusters where the 3Vs are always present. In what concerns to technology stack, we take advantage of almost all state-of-the-art frameworks in Big Data ecosystem such as Spark, Kafka, Hive/Impala, Azure Data Services, MongoDB using Java and Scala as programming language to interact with them.

YOUR ROLE

As a Big Data Administrator you’ll assure the Big Data platform and overall landscape (whatever technology it may be) cluster scalability and performance in the whole Big Data ecosystem.

From the day one, you’ll take part in the analysis and definition of large scale distributed systems but also in its management and integration with external systems. You’ll have a special focus on performance tuning and in application development best practices implementation, both in terms of architecture and security. Usually this role also works closely with application development teams to design, develop and implement the best solutions for data consuming services.

Your daily activities will include:

• Management of large scale big-data systems in critical environments
• Install/upgrade and certify big data platforms
• Implement security rules and policies into big data platforms
• Periodically recommend and update the big data platforms ( hot fixes, patches, etc
• Setup best practices for cluster and infrastructure monitoring
• Analyze hardware, software requirements for each project to be implemented
• Design and develop new processes for better maintenance of the environments
• Help the team to troubleshoot the performance issues

// Stacks: Shell Linux (e.g., bash), Cloudera, Confluent, Azure Data Services, MongoDB; MIT Kerberos ou Windows Active Directory

SKILLS YOU NEED TO HAVE

• MSc / BSc in Information Systems and Computer Engineering and/or Computer Science
• Strong knowledge on Linux operating systems
• Good knowledge of Shell Scripting
• Knowledge about High Availability/Distributed systems and its goals and terminology
• Strong communication skills (written and spoken)

// Will be a nice plus if you have:

• Hands on experience in setting up Spark, Kafka, or other Big Data technology

• Curiosity in Big Data technologies (Hadoop stack, Kafka, MongoDB)
• Team player and problem solving skills

Send us your CV to [email protected] with the reference position (XP_BDA).