This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Data Engineering Lead based in the United States.
We are seeking an experienced Data Engineering Lead to drive the design, development, and delivery of scalable cloud-based data solutions. In this role, you will leverage AWS native services, Python, and PySpark to build robust data pipelines and modernize enterprise data platforms. You will play a key role in shaping data architecture, improving data integration, and developing high-performance data products. Working across complex enterprise environments, you will establish engineering standards and ensure the reliability, quality, and scalability of data solutions. You will also provide technical leadership, mentor engineering teams, and collaborate with architecture, technology, and business stakeholders. This is an opportunity to make a meaningful impact in a flexible, remote-first environment while contributing to large-scale data transformation initiatives.
Accountabilities
-
Lead the architecture, design, and development of scalable data pipelines using AWS Glue, EMR, S3, Python, and PySpark.
-
Drive enterprise data integration initiatives and the implementation of ETL/ELT workflows.
-
Lead the development of reliable, scalable, and reusable data products.
-
Define and maintain engineering standards, best practices, and development guidelines.
-
Conduct technical design reviews and code reviews to ensure solution quality and consistency.
-
Optimize data pipelines for performance, data quality, processing efficiency, and scalability.
-
Mentor and guide Data Engineers, supporting their technical development and delivery.
-
Collaborate with architecture, technology, and business teams to align data engineering solutions with enterprise objectives.
-
Strong experience in AWS data engineering, particularly Amazon S3, AWS Glue, and Amazon EMR.
-
Proficiency in Python, PySpark, SQL, and ETL/ELT development.
-
Expertise in designing, developing, and optimizing large-scale data pipelines.
-
Solid experience with enterprise data integration and complex data engineering environments.
-
Proven ability to provide technical leadership, establish engineering standards, and conduct code and design reviews.
-
Experience mentoring and guiding data engineering teams.
-
Experience with Amazon Redshift, Athena, and DynamoDB.
-
Knowledge of Apache Iceberg and modern data lake or lakehouse architectures.
-
Familiarity with domain-oriented data products and authoritative data sources.
-
Exposure to large-scale enterprise data platform modernization initiatives.
-
Permanent remote working model, with flexibility to work from anywhere within the eligible employment location.
-
Flexible working arrangements adapted to individual, customer, and business needs.
-
Potential for part-time schedules and working hours outside the traditional 9-to-5 framework.
-
Welcome Back Program supporting individuals returning to work after an extended absence due to health or family circumstances.
-
Inclusive and diverse working environment that values different perspectives and individual contributions.
-
Opportunities to make a meaningful impact, develop your expertise, and grow professionally.
As a Data Engineering Lead, you will oversee technical delivery and guide the development of enterprise-grade data engineering solutions. Your responsibilities will include:
Requirements
The ideal candidate combines strong hands-on AWS data engineering expertise with proven technical leadership experience. You should demonstrate:
Preferred Qualifications: