Azure Data Engineer
- - - - - - - - - - - -
KEY EXPECTED SKILLS :
Data Pipeline Development:
- Design and develop ETL processes using Azure Data Factory to move data from various sources to Azure Data Lake Store and SQL Database.
- Implement and manage data pipelines using Azure Databricks.
- Having understanding to Utilize Azure DevOps for CI/CD pipeline setup and management.
Data Storage and Management:
- Manage and optimize data storage solutions on Azure Data Lake Store.
- Ensure data integrity and quality through efficient data governance practices.
Data Integration:
- Integrate data from multiple sources to create unified datasets for analytics and reporting.
- Implement data transformation and cleansing processes to ensure data accuracy.
Data Visualization:
- Develop interactive and visually appealing dashboards and reports using Power BI.
- Collaborate with stakeholders to understand their reporting needs and deliver insightful data visualizations.
Performance Optimization:
- Monitor and optimize the performance of data pipelines and storage solutions.
- Troubleshoot and resolve issues related to data processing and integration.
Collaboration and Communication:
- Work closely with Business analysts and stakeholders to understand data requirements and deliver solutions that meet their needs.
- Document data workflows, processes, and best practices.
Qualifications:
- Bachelor's degree in Computer Science, Information Technology, or a related field.
- 3-5 years of experience as a Data Engineer or in a similar role.
- Strong expertise in Azure Databricks, Azure Data Factory, Azure Data Lake Store, and Power BI.
- Proficiency in SQL and experience with data modeling.
- Knowledge of Python for data processing in Databricks.
- Basic knowledge of Azure DevOps and CI/CD processes
- Familiarity with data governance and data security best practices.
- Familiarity with Agile and Scrum Practices.
- Excellent problem-solving skills and attention to detail.
- Strong communication and collaboration skills.