Hadoop Engineer, Remote

Pentalog
Posted 3 years ago

Our client and partner, the world's largest travel site, operating at scale for almost half a million unique visitors each month, needs to expand their data platform team.

As a data-driven company, this user-generated platform needs a Hadoop Engineer who will share with his teammates the responsibility of maintaining the company’s petabyte-scale data lake and the ecosystem of tools that surround it. This position open at Pentalog requires hands-on operational experience with Big Data technology and securing distributed platforms.

The tech stack is a hybrid open-source Hadoop cloud, house-built distributed ETL tools, and public cloud technologies that help to drive the analytics of the company. A successful engineer will be able to solve complex security problems in the hybrid-cloud environment while working on a team spanning multiple time-zones. 

Responsibilities

  • Write code, as you create today’s solutions for tomorrow’s problems;
  • Use your expertise to build the kinds of tools that expand the modern Big Data ecosystem;
  • Work with a team of talented engineers to identify the shortcomings in Big Data technologies;
  • Apply cloud engineering best practices and develop cloud-first solutions to Big Data challenges.

Job requirements

  • 3+ years of in-depth technical experience with modern Big Data technologies such as Hadoop, Hive, Presto and Spark;
  • Familiarity with the Linux operating system and the Bash shell;
  • General software engineering expertise (Java - preferred, Python and Kotlin);
  • The ability to communicate effectively in an operations environment;
  • Willingness to "get your hands dirty" in a modern Big Data environment
  • Very good level of English.