Job Summary
We are seeking a talented Data Engineer to join our innovative team. As a key player in our data management efforts, you will be responsible for designing and implementing robust data pipelines and ETL processes. Your expertise in Python and cloud-based solutions will be crucial as we migrate our existing data warehouse from MySQL 1.0 to Snowflake and enhance our data infrastructure. This role offers the opportunity to work in a dynamic environment where your contributions will directly impact our ability to deliver exceptional service in the Specialty insurance sector.
Job Responsibilities
- Design, develop, and maintain ETL processes to ensure efficient data flow and transformation.
- Migrate existing data from legacy systems (MySQL warehouse 1.0) to Snowflake, ensuring data integrity and quality throughout the process.
- Transform custom code in Power BI into dbt models for streamlined analytics and reporting.
- Collaborate with cross-functional teams to gather business requirements and translate them into scalable data solutions.
- Implement data governance and security best practices to protect sensitive information and ensure compliance with relevant regulations.
- Optimize data models and workflows for performance and scalability, ensuring that data is accessible and usable for analytics.
- Utilize tools like Apache Airflow for orchestration and automation of data pipelines, ensuring timely and reliable data delivery.
- Provide support for data-related issues and assist in troubleshooting production problems, ensuring minimal downtime and disruption.
- Stay updated with industry trends and emerging technologies to continuously improve our data engineering practices.
Basic Qualifications
- 5+ years of experience in data engineering, with a strong track record of designing and managing complex data systems.
- Proficiency in Python for data manipulation and ETL processes, with a solid understanding of libraries such as Pandas and NumPy.
- Good working knowledge of Power BI for data visualization and reporting.
- Experience with SQL and NoSQL databases, including Snowflake, Amazon Redshift, Databricks, and other cloud data solutions.
- Strong understanding of data modeling, ETL processes, and data warehousing concepts, with hands-on experience in building data pipelines.
- Familiarity with cloud computing solutions such as AWS, Azure, or Google Cloud Platform (GCP).
- Excellent communication skills and the ability to work collaboratively in a team environment.