As a Data Engineer, you will help us establish and maintain the quality and integrity of our data by understanding the context of the data, creating automated data quality checks, and driving actionable data insights. You will collaborate with business users, data engineers, and data scientists to bring meaning to the data and help ensure data pipelines are measuring metrics consistently in an accurate way based on agreed definitions.
Requirements
- Experience addressing issues of data integrity;proven ability to mine and validate data across structured and unstructured data sets.
- Moderate to advanced SQL experience with the ability to seamlessly query multiple datasets.
- Problem solver with the ability to make sound complex decisions in a fast-paced, technical environment.
- Experience writing automated test cases and data validation processes.
- Attention to detail and ability to persist/follow up on issues that require input from multiple people to be resolved.
- Strong documentation skills with the ability to log and concisely express technical data issues.
- Knowledge of AWS data ecosystem (Athena, Redshift, SageMaker, FindMatches ML).
- Python, R, or other programming skills are a big plus.