Data Engineer (Azure) – Spark, Data Lakes, Databricks – Hybrid
Permanent @Garniche Jobs UK posted 3 weeks ago in IT ShortlistJob Detail
-
Job ID 330594
Job Description
Data Engineer – (Azure) x2, Hybrid, East London, £70,000-£100,000, 10% Bonus – DataLakes, PySpark, SQL, Azure, Python, Spark, Databricks, Agile
Role Overview
We are looking for a data engineer responsible for the design, development, and maintenance of applications. You will be working alongside other engineers and developers working on different layers of the infrastructure. Therefore, a commitment to collaborative problem-solving, sophisticated design, and the creation of quality products is essential.
Skills and Experience
Demonstrable design & development experience and experience with big data technologies like Spark/
Proficient in Python, PySpark
Hands-on experience with some of the following technologies:
Azure – Data Lake Projects
SQL
ETL/ELT
Databricks
Spring/Guice or any other DI framework,
RESTful Web Services.
Proficient in querying and manipulating data from various DB (relational and big data).
Experience in writing effective and maintainable unit and integration tests for ingestion pipelines.
Experience in using static analysis and code quality tools and building CI/CD pipelines.If you are interested and meet the above requirements please send your latest CV and call (phone number removed)