In the data team at InDebted, we are developing machine learning models and other data driven approaches to accelerate our disruption of the traditional debt collection industry. By utilising the rich datasets we have accumulated from communicating with our customers via digital-first channels and other innovations, we are able to efficiently deliver significantly better results compared to our peers. Be part of this exciting journey as a data engineer working with our data scientists, software engineers and product managers as we develop our data pipeline infrastructure as part of a greenfield project.
As a Data Engineer at InDebted, you will be sought after for your broad technical skills developing and implementing data pipelines (amongst other data infrastructure) to support the implementation of machine learning models, business intelligence analytics, and other data projects.
Data Tech Stack
Data Tech Stack: Python, Go, Event Sourcing;AWS, AWS Serverless (Lambda Functions), Sagemaker, S3, Glue, Athena, ECR, SQS, SNS, Dynamo, Terraform, DataDog, GitHub, Trello
What will you accomplish?
- Implement data pipelines to extract data from multiple sources.
- Implement data pipelines to transform data using stream, micro-batch or batch methods in preparation for consumption by ML model training, ML model inference, business intelligence analytic tools and self-service platforms.
- Deploy data pipelines following best practice DevOps principles and incorporating automated testing.
- Build and manage robust cloud architectures to provide efficient processing and transformation of data.
- Be a team player, foster a speed-oriented culture in our data team and help to cultivate a fast-growing team.
- Work autonomously and make decisions collaboratively, within a team that will support and challenge you as you grow and develop.
What do we need from you?
- 3+ years of experience developing highly scalable and resilient data pipelines.
- Well-versed in multiple programming languages and paradigms, preferably in two or more of Python, Go, SQL, Scala, Java.
- Experience with orchestration tools (such as Apache Airflow, Luigi) and data processing frameworks (such as Apache Beam or Spark).
- Experience utilising Terraform to orchestrate infrastructure (Infrastructure as Code).
- Strong ability to implement, maintain and manage relational databases, cloud-based data warehouses, non-relational databases and storage processes for unstructured data.
- Familiarity with data integration tools (such as dbt, Stitch, Fivetran, Talend, Informatica, Matillion)
- Having an understanding of best practices in the storage and processing of data, as well as computer science fundamentals, software best practices, automated testing, networking protocols and distributed systems.
- Excellent verbal and written communication skills with the ability to understand and explain complex concepts to technical and non-technical audiences.
Where will this role be based?
InDebted is a remote-first company. Work from anywhere you like, as long as you have a good internet connection and at least a 4-hour overlap with Australian Eastern Standard Time (UTC+10) during business hours.
Diversity and Inclusion
At InDebted, we respect and celebrate the unique attributes, characteristics, and perspectives that make each person who they are. We also believe that bringing diverse individuals together allows us to build better products and a better overall company. InDebted is an Equal Opportunity Employer.
InDebted is a remote-first company. Work from anywhere you like, as long as you have a good internet connection and at least a 4-hour overlap with Australian Eastern Standard Time (UTC+10) during business hours.
Diversity and Inclusion
At InDebted, we respect and celebrate the unique attributes, characteristics, and perspectives that make each person who they are. We also believe that bringing diverse individuals together allows us to build better products and a better overall company. InDebted is an Equal Opportunity Employer.
Data source
We Work Remotely