We are seeking a highly skilled and experienced Data Engineer to join our dynamic team. As a Data Engineer, you will play a critical role in designing, developing, and maintaining our data infrastructure on Azure Data Services. Your primary focus will be on building scalable and efficient data pipelines, ensuring data quality and integrity, and optimizing data workflows for various data processing needs.
Responsibilities:
Design, develop, and deploy end-to-end data pipelines on Azure Data Services, including Azure Data Factory, Azure Databricks, Azure Synapse Analytics, and Azure Data Lake Storage.
Collaborate with cross-functional teams to understand data requirements and design efficient data models, ensuring optimal performance and scalability.
Extract, transform, and load (ETL) data from various sources, including structured and unstructured data.
Implement data integration processes and data ingestion mechanisms to collect, process, and store large volumes of data.
Ensure data quality and consistency by implementing data validation, cleansing, and transformation techniques.
Monitor and troubleshoot data pipelines, identifying and resolving data-related issues and performance bottlenecks.
Collaborate with data scientists and analysts to provide them with the necessary data infrastructure and tools for analysis and reporting.
Stay updated with the latest advancements and best practices in Azure Data Services, contributing to the continuous improvement of our data architecture and processes.
Qualifications:
Minimum of 5 years of hands-on experience in data engineering, with a focus on Azure Data Services.
Strong proficiency in building and optimizing data pipelines using Azure Data Factory, Azure Databricks, Azure Synapse Analytics, and Azure Data Lake Storage.
In-depth knowledge of data modeling, data warehousing concepts, and data integration techniques.
Solid programming skills in languages such as Python, SQL, and Spark.
Experience with data governance, data security, and compliance frameworks.
Proven ability to collaborate effectively with cross-functional teams and communicate complex technical concepts to non-technical stakeholders.
Strong problem-solving and analytical skills with a keen attention to detail.
Azure certifications related to data engineering or data analytics are highly desirable.
If you are a passionate and driven Data Engineer with expertise in Azure Data Services, we invite you to join our team and contribute to the success of our data-driven initiatives. Apply now and be part of a dynamic organization at the forefront of innovation in data engineering.