MAIN JOB PURPOSE
As Big Data Engineer in our Cyber Security team you will help build and deliver our Big Data Threat Intelligence platform. You will have a focus on data processing, data security and deliver the systems that process huge volumes of data from various sources.
You will be working closely with our Data Scientists, Researchers and Product Development team helping with the development of the data processing pipelines that feed our Threat Intelligence Platform and our Customer Reporting Dashboards. You will be a self motivated learner, willing and able to integrate new technologies and ways of working.
JOB SUMMARY
- Development of Nominet’s Threat Intelligence Platform
- Maintenance and design of data pipelines
- Integration of new security data sources
- Implementation of continuous integration and continuous delivery pipelines
KEY REQUIREMENTS
- Experience in building large-scale data processing systems and large-scale data infrastructures
- Experience within cloud platforms especially AWS
- Good understanding of distributed computing principles
- Able to work with the latest (NoSQL) database technologies such as Cassandra and Elasticsearch
- Commercial or open source experience in software engineering (object-oriented design, coding and testing patterns) ·
- Extensive knowledge in programming languages such as Java and Python
- Experience building data processing systems with Spark and Kafka using Java or Python
Data source
Stack Overflow