Head Resourcing is currently recruiting for a Data Engineering Manager for our client based in Edinburgh. Our client is a global technology company that uses location-based data and analytics to deliver digital content to audiences on the move, they work with some of the world's best known companies and have offices across the globe (they have c. 200 employees currently). They are looking to recruit a creative and pragmatic Data Engineering Manager to join their Data Platform team; this team owns the big data infrastructure and processing that powers the entire business. The team has responsibility across devops, data engineering, data analysis, and data science - receiving over 350gb of data an hour and responding to nearly half a million decision requests each second. The team utilises as wide range of technologies such as Kafka, Spark, BigQuery, Druid, ScyllaDB, and Postgres - they are constantly looking at how things can be done better and want you to drive the next level of scale and intelligence. As an Engineering Manager you will be technically fluent, a hands-on leader who facilitates communication between business, customers, and internal teams. You will have a passion for recruiting, nurturing, motivating, and getting the most out of amazing people, while delivering products that go above and beyond customer expectation.
Responsibilities:
- Build, lead, and manage a cross-functional engineering team to design, deliver, support, and iterate data products and infrastructure
- Drive a culture of lean development and ensure that the team you're responsible for operates efficiently and effectively
- Empower and mentor team members to improve their craft, deliver high-quality products, and develop themselves as individuals
- Identify and take advantage of new and innovative opportunities for commercial and technical growth
- Ensure that products and services are built and operate to a high standard and anticipate the future
Essential:
- Engineering experience solving complex, large-scale data challenges
- Strong decision-making skills and an ability to effectively prioritise against business outcomes
- Hands-on implementation and architectural familiarity with streaming data, relational and non-relational databases, orchestration frameworks, and distributed processing technologies (e.g. Spark)
- Familiarity with remote management and working
- Advanced knowledge of cloud based services (AWS, GCP)
- Excellent working understanding of server-side Linux
- Experience integrating and consuming data from third party API's
Desirable:
- Advanced knowledge and experience with tools like Google BigQuery and Spark to solve data-centric problems
- Experience with data processing pipelines using Airflow
- Exposure to Docker and Kubernetes
- Understanding and ability to innovate, apply, and optimise complex algorithms and statistical techniques to large data structures
