A Senior Data Engineer with experience in Databricks and Spark is urgently sought after by a global market data and analytics provider to help drive engineering on a large-scale data transformation programme.
You will join a team who are about to embark on the build of a new enterprise data platform, which is being developed as part of a plan to consolidate/merge the backend of 5 separate client-facing data products into one and provide a uniform singular offering.
Your role will be twofold in nature, undertaking both the design and development of this new platform, as well as working to mentor and upskill the wider engineering teams in the usage of Databricks.
The team are highly agile, fully setup on AWS cloud and real proponents of modern development best practises i.e., TDD/BDD and E2E CI/CD pipelines.
Candidates must have:
- A solid background in Data Engineering using Python
- Proven experience in designing and building Big Data Solutions at scale using Spark and Databricks
- Excellent experience in systems processing very large data volumes at near real-time speed.
- Ideally, commercial exposure to both AWS and Terraform, and an understanding of Infrastructure as Code
- An Agile mindset, being a keen proponent of TDD/BDD and DevOps (CICD)
- Ideally, a career within either Finance or Professional Services (or Blue-chip)
This is an excellent opportunity to join a leading organisation going through a large-scale technical transformation and help drive their adoption of Big Data practises.
This contract will be offered for an initial 6 months, extension likely, and you will be required to work via umbrella company.
Cornwallis Elt is an Employment Agency & Employment Business and has been listed 3 times in The Sunday Times Virgin Fast Track 100 of the UK`s fastest growing private companies, as well as in the Recruitment International Top 250, Top 50 in IT and the Recruiter Fast 50 & Hot 100 reports.
