Responsibilities
- Design and develop scalable and reliable data platforms and/or data pipelines for analytical and operational use cases
- Design and implement data processing workflows using distributed processing frameworks such as Apache Spark/pySpark (Databricks)
- Contribute to the evolution of the organization’s cloud data platform using Azure technologies (e.g., Azure Data Factory, Synapse, Microsoft Fabrik, Databricks, Event Hub, Azure Data Lake Storage)
- Strong background in data modelling to support analytics, reporting, and downstream data consumption
- Integrate and process data from multiple internal and external sources while ensuring data consistency, quality, and reliability
- Take ownership of data pipeline reliability, monitoring, and performance, ensuring scalable and maintainable data processing workflows
- Collaborate with data analysts, data scientists, and engineering teams to translate business and analytical requirements into robust data solutions
- Drive improvements in data engineering practices, including pipeline design standards, testing approaches, and operational reliability
- Participate in evaluating and introducing new tools and technologies within the data engineering ecosystem
Required Qualifications
- Strong hands-on experience (4+ years) in data engineering or building data processing platforms
- Strong SQL skills and solid experience with Python for building and maintaining data pipelines
- Experience working with distributed data processing frameworks such as Apache Spark/pySpark (Databricks experience is a strong advantage)
- Practical experience designing and implementing data pipelines in cloud environments, preferably within the Azure ecosystem
- Experience designing data pipelines and data models for production-scale analytical systems
- Strong understanding of data engineering concepts including ETL/ELT design, dimensional modeling, and data warehousing principles
- Experience working with modern data lake architectures and data formats such as Parquet and JSON
- Experience with workflow orchestration tools (e.g., Airflow)
- Experience with database development and performance optimization
- Ability to design scalable and maintainable data solutions with minimal supervision
- Strong collaboration and communication skills
- Professional working proficiency in English
Nice to Have
- Experience with modern cloud data warehouses such as Snowflake, BigQuery, or Redshift
- Experience with streaming data technologies (e.g., Event Hub, Kafka, Spark Streaming)
- Experience with Delta Lake or other lakehouse technologies
- Familiarity with CI/CD practices and Git-based development workflows
- Experience working in agile development environments