Latest news

Big Data CloudOps Engineer

Job Opportunity Big Data CloudOps Engineer Xpand IT

Big Data CloudOps Engineer

Complex problems require the right expertise! Today one of the biggest challenges is data storage and processing, across these three main domains (3Vs): Volume, Velocity and Variety.

At Xpand IT, the Big Data technological area develops and implements architectures and software solutions that represent the state of the art on the capture, ingestions, process, enrich, storage and management of critical data from huge clusters where the 3Vs are always present. In what concerns to technology stack, we take advantage of almost all state-of-the-art frameworks in Big Data ecosystem such as Spark, Kafka, Hive/Impala, Azure Data Services, MongoDB using Java and Scala as programming language to interact with them.

YOUR ROLE

As a Big Data CloudOps Engineer you’ll play a fundamental role in several phases of the adoption of Big Data platform in Cloud context (Azure or AWS),  participating in the analysis, definition and sizing of distributed storage and / or computing systems, setup, upgrade, securization and tuning. For these critical systems, a special focus on performance and security is crucial, as well as implementing the best service development practices to serve as the basis for monitoring tools.

Usually this role also works closely with the development teams in the design, development and installation of application solutions, processing and storage of large-scale data.

Your daily activities will include:

  • Setup / upgrade / securization / tuning of platforms on a large scale in critical, ephemeral and / or self-scaling environments on a Cloud provider
  • Implement security rules and policies on Cloud platforms
  • Recommend and periodically update the best practices for using Cloud services
  • Configure good practices for monitoring the infrastructure
  • Analyze sizing requirements and ideal services for each project to be implemented
  • Design and develop new processes for better stability and performance maintenance of environments
  • Develop integration or automation processes for Cloud deployments. Being able to resort to containerized environments on Kubernetes
  • Participate and help solve performance, scalability and security issues

// Stacks: Cloudera, Confluent, Azure Data Services, MongoDB, Kerberos ou Windows Active Directory

Big Data CloudOps Engineer Xpand IT Confluent
Big Data CloudOps Engineer Xpand IT Mongo DB
Big Data CloudOps Engineer Xpand IT Spark
Big Data CloudOps Engineer Xpand IT Azure
Big Data CloudOps Engineer Xpand IT Cloudera
Big Data CloudOps Engineer Xpand IT Databricks

SKILLS YOU NEED TO HAVE

  • MSc / BSc in Informatics and Computers, Information Systems or Computer Science
  • Good knowledge of Linux operating systems are valued
  • Good shell scripting knowledge is valued
  • Knowledge of the objectives and terminology of distributed high availability systems is valued
  • Team player and problem solving skills
  • Good communication skills (written and spoken)
  • Fluent English (written and spoken)

// Will be a nice plus if you have:

  • Hands on experience in setting up Spark, Hive or Apache Hadoop
  • Hands on experience about automation tools like Ansible, Terraform or CloudFormation
  • Curiosity about Big Data technologies such as Hadoop, Kafka, MongoDB
  • Curiosity about Kubernetes and the various existing flavors Openshift, Rancher, AKS or EKS
  • Curiosity about the Cloud services that make up Azure Big Data Stack or AWS Big Data Stack, such as Databricks, Azure Synapse or Amazon S3

// Learn more about Big Data area:

Apply here

As a Big Data System Engineer, every day is challenging: between the clusters deployments from scratch and the optimization of distributed systems, it is absolutely crucial to have a deep understanding of the ecosystem’s multiple services, dealing with many different contexts.

Pedro Martins, Big Data System Engineer @Xpand IT
Big Data ClouOps Engineer Xpand IT

Videos that tell you the whole story

Xpand IT: You Have to Live it!

Summer Trip

Be an Xpander

Life at Xpand IT

XTech Community

Meetup

Sara GodinhoBig Data CloudOps Engineer

Read more in

Want to get amazing Big Data, Business Intelligence, Middleware
Mobile articles & news directly from our experts?
Subscribe to our blogs now.