Data Engineer - Hadoop Platform

Are you driven by data and passionate about building modern data platforms?
See Yourself in the Team
As a member of the Operations Data Engineering team, you will perform day-to-day Hadoop platform performance tuning and deliver support, ensuring optimal system performance and reliability. Building ETL applications and managing large-scale, complex big data applications using cloud-native technologies. Collaborate with stakeholders in the Data and Platform crew to drive our Data Strategy and achieve our vision of becoming the best AI Bank in the world.

We Are Interested in Hearing from People Who:
  • Have hands-on experience in building cloud-native applications in AWS.
  • Bring hands-on experience with Infrastructure as Code using tools such as Terraform and CloudFormation for repeatable AWS deployments.
  • Are comfortable designing CI/CD pipelines with GitHub Actions.
  • Focus on automation and building efficient systems.
  • Are familiar with tools like Hadoop, Spark, Kafka, and other distributed processing frameworks for handling large datasets.
  • Have knowledge of financial markets and/or corporate & institutional lending products (advantageous).

Tech Skills
We use a broad range of tools, languages, and frameworks. While we don't expect you to know them all, experience or a willingness to learn these skillsets (or equivalents) will set you up for success in our team:
  • AWS Certified Data Engineer – Associate
  • Strong SQL experience
  • Understanding of ETL data pipelines using Ab Initio (any other ETL tools)
  • AWS EMR or Glue
  • Integration knowledge of MQs, REST APIs and Kafka
  • Data warehousing with Snowflake
  • Infrastructure as Code using Terraform
  • Design and implement secure CI/CD pipelines using GitHub Actions

With this role, there will be an on-call support component with it sitting in the operations team.

If you're interested in this role, click 'apply now' to forward an up-to-date copy of your CV. If this job isn't quite right for you, but you are looking for a new position, please contact us for a confidential discussion on your career.


LHS 297508

Summary

Job Type
Permanent
Industry
Banking & Financial Services
Location
NSW - Sydney CBD
Specialism
Data & Advanced Analytics
Ref:
2924810

Talk to a consultant

Talk to Luke Moran, the specialist consultant managing this position, located in Sydney City
Level 13, Chifley Tower, 2 Chifley Square

Telephone: 02 9249 2289

Recommended Courses

Introduction to Power BI21 minutes
A Useful Guide To Work Life Balance210 minutes

Similar jobs to Data Engineer - Hadoop Platform

  • Senior Data Engineer

    Senior Data Engineer
    NSW - Sydney CBD

  • Staff Data Engineer

    Are you looking for your next Staff Data Engineering Permanent role within Banking?
    NSW - Sydney CBD

  • REQ244686_ Staff Software Engineer AWS

    Permanent opportunity with one of the leading banking clients in Sydney Location
    NSW - Sydney CBD

  • Staff Software Engineer

    Drive cloud-based data and analytics innovation to combat financial crime using AWS and Quantexa.
    NSW - Sydney CBD

  • Senior Cloud Engineer

    Join an innovative team to work with cutting-edge technology in a collaborative environment.
    NSW - Sydney CBD