AWS Data Engineer
JobWatch powered by BigChange plans, manages, schedules &tracks your mobile workforce! BigChange is an established force in mobile resource management, offering a host of solutions that have proven to transform the way companies manage their mobile workforces and mobile operations. By challenging convention and pushing for constant innovation, BigChange is pioneering new, intuitive and simple to use technology that delivers industry leading levels of return on investment.
Brochure: https://www.bigchangeapps.com/view-brochure/
Website: https://www.bigchangeapps.com
This role arises from BigChange rapid business growth. JobWatch is a multi-tenanted, SaaS and mobile applications provide transaction processing for operational data from all customers’business and resource location minute-by-minute. Data is comprehensive, user extensible and includes customers’prospects, customers, customer correspondence, pricing, sales opportunities, orders, jobs, risk assessments, job worksheets, engineers/drivers, vehicles, stock and equipment, locations, and financial transactions. This role enables data to be assembled in a Data Lake in AWS so that business insights are available for customers and BigChange from exploring this data.
Your Role:
- Design, build, maintain, test and deploy data pipelines and architectures in AWS and potentially Azure;
- Ensure the scalability of the data engineering architecture to meet future business demand;
- Extract, transform and load data from multiple source systems such as SQL Server, MongoDB, Flat Files etc;
- Performance tuning of data pipelines;
- Ensures BigChange customers' and BigChanges' personally identifiable information (PII) and any other sensitive data is secured within the BigChange Data Lake and Data Warehouse systems;
- Provide advice and support to other colleagues on system functionality;
- Be aware of the latest releases of relevant technologies;
- Manages, documents and improves data engineering processes and system records;
- Actively attends and contributes in Agile sprint ceremonies/meetings.
You will need:
- 3+ years' experience in a similar data engineering role i.e. building data pipelines for either a Data Warehouse and/or a Data Lake
- 2+ years' experience using Python
- 2+ years' experience of using AWS cloud computing services
- 1+ years' experience using Linux or Unix
- Strong experience using SQL for querying data in any relational database
- Writes clear and concise technical documentation including run-books, play-books, specifications, software component designs and test cases with due regard for the knowledge and skill levels of the target readers
We would also like:
- In depth experience of any of the following AWS services will be a significant advantage: Lambda, Glue, Athena, Step Functions, Cloudformation, DMS, EMR, EC2, IAM, VPC, Cloudwatch, SQS, S3
- Strong experience using SQL for querying data in SQL Server
- In depth knowledge of utilizing Change Data Capture processes, particularly on SQL Server
- In depth knowledge of utilizing Replication processes, particularly on SQL Server
- Experience of "Big Data" technologies e.g. Spark, Hadoop, AWS EMR, Data bricks etc.
- Experience of Azure Cloud Computing Services
- Built continuous integration / deployment pipelines in any technology.
What we offer:
- Salary: Competitive
- Expenses paid for all overnight stays, subsistence and mileage whilst on business.
- 33 days holiday, plus bank holidays, plus ‘BigChange Birthday’.
- Pension plan (NEST)
- Gym membership assistance £20 gross pay monthly (for 12 months).
- Free massage in the office
- Free Yoga Classes
- Cash Health Plan
- “Motivational Mondays”–inspiring talks monthly from extraordinary people.
- Local fruit delivered weekly to the office.
- Being part of a supportive team with the ability to learn new skills and grow within the company.
- Experience cutting edge technology and be part of a company that is shaping the future.
All rewards are agreed between each employee and the company based on their role/offer letter and may differ from the standard
Location of work: Remote
Important Information:
Any offer is subject to the successful candidate obtaining a BS7858 Certification during their probationary period. BigChange reserves the right to with draw any offer if an employee is not able to obtain a BS7858 Certificate. The probationary period will be extended if all HR information is not complete by the end of the initial probationary period.
BigChange are an equal opportunities employer and welcome applications from all suitably qualified persons regardless of their race, sex, disability, religion/belief, sexual orientation or age.