Senior Data Engineer (6-month contract).

Job description

gravity9 is a boutique IT consulting company headquartered in the UK with offices in the US, Canada, and Poland. Our team has deep experience in engineering, experience design and product management. We enjoy a challenge and pride ourselves on working with our clients on their most complex problems, finding elegant and flexible solutions that help them transform their businesses.

Role Description

We are seeking a highly motivated Senior Data Engineer to join our client's Product Development team within their Digital Practice. In this role, you will play a key part in client digital transformation journeys by delivering solutions that support decision-making, autonomous execution, and integrated operations. You will focus on productizing deep subject-matter expertise through robust data platforms and AI-enabled solutions built on Microsoft Azure. Operating within a flat, connected organization that values innovation and an entrepreneurial mindset, you will challenge existing ideas to take business outcomes to higher levels. The ideal candidate is a confident, passionate, and outcome-oriented leader with exceptional communication skills who excels at setting direction, mentoring team members, and leading directly by doing.

A successful candidate would:

·       Work in Agile teams to deliver the committed backlog against acceptance criteria

·       Design, build and operate ingestion and orchestration pipelines in Azure Data Factory

·       Develop ETL/ELT in Azure Databricks using a medallion architecture to deliver trusted data for analytics, reporting and AI

·       Lead code reviews and set engineering standards; take part in estimating, planning, testing, deployments and documentation

·       Shape data architecture decisions, including metric definitions, semantic models and data that serves AI agents

·       Apply data governance, security and access controls in line with client classification and residency requirements

·       Use AI-assisted development tools responsibly to accelerate delivery without compromising quality or security

·       Communicate progress, open items and issues to stakeholders, including management

·       Stay abreast of relevant technology developments

Job requirements

Experience:

·       7+ years’ experience in data engineering or software development, including 3+ years hands-on with Azure Databricks or Spark and actively mentoring junior team members. Proven capabilities in:

o   Python and PySpark (expert level) and SQL

o   Azure Data Factory, Azure Databricks (Delta Lake, Workflows, Unity Catalog), Azure Data Lake Storage and Azure SQL

o   Data modelling (dimensional and lakehouse) and integration via APIs, OData, JDBC/ODBC and files

o   Azure AI services (e.g. Azure AI Foundry, Azure OpenAI) and preparing data for AI agents

·       Secure CI/CD and infrastructure-as-code (e.g. Terraform, Bicep, Databricks Asset Bundles) across DEV, UAT and PROD using Azure DevOps or GitHub, with sound Git branching practices

·       AI-assisted development, with sound judgement in reviewing generated code

·       Data quality, testing, security and privacy in regulated or restricted-data environments

·       Designing end-to-end data solutions and recommending new frameworks and technologies

·       Agile and Kanban delivery in mixed-discipline teams, building consensus across subject matter experts, engineers, data scientists and designers

·       A self-starter who is organized and needs minimal supervision

·       Bachelor’s degree in computer science, Engineering or a related field, or equivalent education and experience

Nice to have:

·       Experience in integrating enterprise systems (e.g. ERP, project management or document management platforms)

·       Data management and governance concepts (e.g. data quality, cataloguing, lineage, master data, stewardship) using any supporting toolset

Explore the possibilities of application modernization for your organization. Reach out to us to learn more about how we can help you navigate legacy systems; turning legacy complexity into digital agility.