Overview:
As a Data Engineer, you will play a critical role in transforming raw data into valuable insights that drive our business decisions. You will design, develop, and maintain data pipelines and infrastructure across hybrid cloud environments, while also building robust reporting and visualization solutions.
Responsibilities:
- Data Pipeline Development: Design, build, and maintain scalable data pipelines to extract, transform, and load (ETL) data from various sources (e.g., databases, APIs, files) into data warehouses or data lakes.
- Hybrid Cloud Infrastructure: Manage and optimize data infrastructure across hybrid cloud environments, leveraging cloud-native services and on-premises resources.
- Data Quality: Ensure data quality through implementation of data validation, cleansing, and standardization processes.
- Reporting and Visualization: Develop interactive reports and dashboards using tools like Power BI, Tableau, or Looker to provide actionable insights to stakeholders.
- Data Governance: Adhere to data governance policies and procedures, including data security, privacy, and compliance regulations.
- Data Modeling: Design and implement data models (e.g., dimensional, normalized) to optimize data storage and retrieval.
- Automation: Automate data pipelines and processes using scripting languages (e.g., Python, SQL) and automation tools.
- Collaboration: Work closely with data analysts, scientists, and business users to understand their requirements and deliver relevant data solutions.
Requirements
- Experience: Proven experience as a Data Engineer or similar role with a focus on data pipelines, cloud infrastructure, and reporting.
- Technical Skills: Strong understanding of data engineering concepts, tools, and technologies (e.g., SQL, Python, ETL tools, cloud platforms).
- Cloud Platforms: Experience with major cloud platforms (e.g., AWS, Azure, GCP) and their data-related services (e.g., data warehouses such as BigQuery/Redshift, data lakes, data pipelines).
- Data Modeling: Proficiency in data modeling techniques (e.g., dimensional, normalized) and data warehouse design.
- Reporting and Visualization: Expertise in using reporting and visualization tools (e.g., Looker, Power BI, Tableau) to create interactive dashboards.
- Problem-Solving: Ability to troubleshoot complex data-related issues and find innovative solutions.
- Communication: Excellent communication skills to collaborate effectively with cross-functional teams.
- Certifications: Preferred certifications (e.g., AWS Certified Data Engineer, Azure Certified Data Engineer, GCP Certified Professional Data Engineer).
Additional Skills (Preferred):
- Experience with data warehousing and data lake technologies (e.g., Google BigQuery, Amazon Redshift, Snowflake, Databricks)
- Knowledge of data analytics and machine learning concepts
- Familiarity with data governance and compliance frameworks (e.g., HIPAA, GDPR, CCPA)