Description
We are seeking an experienced Azure DevOps Engineer to join our data platform team. In this role, you will be pivotal in designing, building, and maintaining robust CI/CD pipelines for our Azure-based data solutions. You will leverage your expertise in Azure DevOps, Azure Databricks, and Azure Data Factory to ensure the secure, reliable, and scalable deployment of our data infrastructure. Your responsibilities will include developing CI/CD strategies using Databricks Asset Bundles, implementing selective deployment for ADF, and creating automation scripts with PowerShell and Python. The ideal candidate will possess strong problem-solving skills to resolve complex pipeline issues and a commitment to implementing security best practices. You will collaborate closely with data engineers and architects to enhance our deployment processes, document standards, and drive innovation in our data operations.
Requirements
1. 5+ years of experience in a DevOps role with a focus on CI/CD.
2. Proven, hands-on experience designing and managing CI/CD pipelines using Azure DevOps.
3. Strong, practical knowledge of Azure Databricks and Azure Data Factory CI/CD processes.
4. Demonstrated experience with Databricks Asset Bundles for managing Databricks deployments.
5. Proficiency in automation scripting using both PowerShell and Python.
6. Experience implementing selective deployment strategies for Azure Data Factory pipelines.
7. Strong troubleshooting and problem-solving skills for complex pipeline issues.
8. Solid understanding of Git version control and branching strategies.
Desirable
1. Experience with containerization technologies like Docker and Kubernetes.
2. Knowledge of cloud security principles and DevSecOps practices.
3. Experience working in large-scale, enterprise cloud environments.
4. Excellent communication and collaboration skills for working with cross-functional teams.
5. Azure certifications such as AZ-400: Designing and Implementing Microsoft DevOps Solutions.