Latest news

Big Data System Engineer

Job Opportunity Big Data System Engineer Xpand IT

Big Data System Engineer

Complex problems require the right expertise! Today one of the biggest challenges is data storage and processing, across these three main domains (3Vs): Volume, Velocity and Variety.

At Xpand IT, the Big Data technological area develops and implements architectures and software solutions that represent the state of the art on the capture, ingestions, process, enrich, storage and management of critical data from huge clusters where the 3Vs are always present. In what concerns to technology stack, we take advantage of almost all state-of-the-art frameworks in Big Data ecosystem such as Spark, Kafka, Hive/Impala, Azure Data Services, MongoDB using Java and Scala as programming language to interact with them.

YOUR ROLE

As a Big Data System Engineer you’ll have a fundamental role in several phases of the adoption of the Big Data platform, participating in the analysis, definition and sizing of distributed storage and / or computing systems, setup, upgrade, securization and tuning. For these critical systems, a special focus on performance and security is crucial, as well as the implementation of the best service development practices to serve as the basis for monitoring tools. Usually this role also works closely with the development teams in the design, development and installation of application solutions, large-scale data processing and storage.

Your daily activities will include:

  • Setup / upgrade / securitization / tuning of big data platforms on a large scale in critical environments
  • Implement security rules and policies on Big Data platforms
  • Recommend and periodically update Big Data platforms (hot fixes, patches, etc.)
  • Configure good practices for monitoring the infrastructure of Big Data clusters
  • Analyze hardware and software requirements for each project to be implemented
  • Design and develop new processes for better stability and performance maintenance of environments
  • Participate and help solve performance, scalability and security issues

// Stacks: Shell Linux (e.g., bash), Cloudera, Confluent, Azure Data Services, MongoDB; MIT Kerberos ou Windows Active Directory

Big Data System Engineer Confluent
Big Data System Engineer MongoDB
Big Data System Engineer Xpand IT
Big Data System Engineer Xpand IT
Big Data System Engineer Xpand IT
Big Data System Engineer Xpand IT

SKILLS YOU NEED TO HAVE

  • MSc / BSc in Information Systems and Computer Engineering and/or Computer Science
  • Good knowledge on Linux operating systems is valued
  • Good knowledge of Shell Scripting is valued
  • Knowledge of High Availability/Distributed systems and its goals and terminology
  • Strong communication skills (written and spoken)
  • Team playerand problem solving skills
  • Fluent English (written and spoken)

// Will be a nice plus if you have:

  • Hands on experience in setting up Spark, Hive, or other Apache Hadoop
  • Curiosity about Big Data technologies such as Hadoop, Kafka and MongoDB

// Learn more about Big Data area:

Apply here

As a Big Data System Engineer, every day is challenging: between the clusters deployments from scratch and the optimization of distributed systems, it is absolutely crucial to have a deep understanding of the ecosystem’s multiple services, dealing with many different contexts.

Pedro Martins, Big Data System Engineer @Xpand IT
Big Data System Engineer Xpand IT

Videos that tell you the whole story

Xpand IT: You Have to Live it!

Summer Trip

Be an Xpander

Life at Xpand IT

XTech Community

Meetup

Sara GodinhoBig Data System Engineer

Read more in

Want to get amazing Big Data, Business Intelligence, Middleware
Mobile articles & news directly from our experts?
Subscribe to our blogs now.