Skip to content
Six Figure Jobs
GIS Recruiter Ltd logo

GIS Recruiter Ltd

Databricks Data Engineer

  • Bishopsgate, United Kingdom
  • Onsite
  • Posted 5d ago
  • Full-time
  • 5+ yrs experience

Salary

£75K - £95K

Listed in the posting

Pays more than 15% of 41 data engineer listings in GBP.

This job
£75k–£95k
Typical
£98k
Most pay
£90k–£105k
Apply on Adzuna

Skills

Azure
Databricks
Delta Lake
ELT
ETL
Medallion Architecture
Pyspark
Python
Spark
SQL

Nice to have

Full job description

Salary: £75,000 - 95,000 per year

Requirements:
  • 5+ years of hands-on Databricks experience, including expert-level knowledge of Azure Databricks.
  • Advanced PySpark, Spark SQL, and Python programming skills.
  • Strong experience developing complex ETL/ELT pipelines and implementing Medallion Architecture (Bronze, Silver, and Gold).
  • Deep understanding of Delta Lake, Delta Tables, and Delta Live Tables / Lakeflow Declarative Pipelines.
  • Strong knowledge of Apache Spark architecture, optimisation, and performance tuning.
  • Hands-on experience with Unity Catalog, access controls, and data governance.
  • Experience implementing incremental data loads, CDC, and SCD Type 1/Type 2.
  • Strong SQL development and query optimisation skills.
  • Experience integrating Azure Data Factory with Databricks and knowledge of Azure Data Lake Storage Gen2.
  • Experience with CI/CD pipelines, Git, Azure DevOps, and automated deployments.
  • Experience implementing data quality, validation, and reconciliation frameworks.
  • Previous experience delivering data engineering projects in UK Government or other public sector organisations, such as Central Government, Local Government, NHS, DWP, HMRC, or the Home Office.
  • Understanding of public sector data governance, security, and information management.
  • Experience working with sensitive and confidential datasets.
  • Knowledge of UK GDPR, the Data Protection Act 2018, and relevant government data security standards.
  • Experience working in Agile delivery teams and government digital transformation programmes.
  • Eligibility and willingness to undergo SC clearance, government security vetting, and background checks.
  • Previous UK Government or Central Government project experience is highly desirable.
Responsibilities:
  • Design and develop scalable Databricks data pipelines.
  • Build and maintain enterprise-grade Lakehouse platforms.
  • Optimise Spark workloads and Databricks clusters.
  • Implement robust data quality and governance controls.
  • Collaborate with architects, analysts, and business stakeholders.
  • Support production deployments and troubleshoot complex data issues.
  • Ensure solutions meet public sector security and compliance requirements.
Technologies:
  • Azure
  • CI/CD
  • Cloud
  • Databricks
  • DevOps
  • ETL
  • Git
  • Support
  • Python
  • PySpark
  • SQL
  • Security
  • Spark
  • Unity
  • GameDev

More:

We are seeking an experienced Senior Databricks Data Engineer to deliver enterprise-scale data engineering solutions using Azure Databricks. The role involves building complex data platforms within UK public sector environments, working with large datasets, sensitive information, and secure cloud-based data architectures.

last updated 46 week of 2026

Apply on Adzuna

Opens GIS Recruiter Ltd’s listing on Adzuna in a new tab.

Similar six-figure roles

Browse all Data Engineer jobs →
DistroKid logo
DistroKid

Senior Data Engineer

£185k - £200k
Salary shown
UKRemoteSenior

We are seeking a Senior Data Engineer to join our team at DistroKid. The successful candidate will play a crucial role i...

Posted 4d ago
Ravio logo
Ravio

Data engineer

£100k - £120k
Salary shown
UKRemote

We’re looking for a Lead Data Engineer to lead a small team of data engineers within our Engineering team. You’ll remain...

PythondbtAgentic AIAIAutomation+7 more
Posted 6d ago
MRJ Recruitment logo
MRJ Recruitment

Lead Data Engineer

£95k - £105k
Salary shown
Manchester, UKHybridLead

Leading the technical delivery of a major data warehouse migration from on-premise SQL Server to an AWS-based lakehouse....

Posted 7d ago

Get jobs like this one by email

Data Engineer jobs · UK

Only new matching jobs, nothing else. Confirm by email; unsubscribe in one click.

Explore more

Salary

£75K - £95K

Apply