Lead Data Engineer

Job Description

Technical skills:

• Should have 6+ years of hands-on experience in Data engineering
• Excellent command over ETL tools covering Snowflake, Azure Data Lake, Databricks and Azure Data Factory
• Good command over writing Python Scripts for custom build data ingestion
• Proficient in SQL scripts (Stored procedure, Views, Functions, Joins)
• Experience with Snowflake
• Experience with DBT (for ETL transformation logic)
• Experience with FIVETRAN (for data ingestion)
• Good understanding of data warehouse fundamentals and principles, database modelling knowledge and experience Experience utilizing data pipelines and connectors across different source systems
• Experience with cloud data integration tools and services related to Fivetran, DBT or similar data ingestion and data transformation tools
• Experience working Git version control, preferably GitHub
• Professional IT related education or equivalent (university) education (e.g., B.E/BTech/MCA/MSC) in a relevant subject, preferably in Computer Science is a preferred
• Experience working as ETL/ELT developer or a Data engineer

Requirements for the role:

• Analyse and organize raw data, providing datasets and sources for business reporting requirements
• Implement data quality rules to enhance data reliability Understand business needs and objectives
• Conduct complex data analysis and data modelling, interpreting trends and patterns
• Collaborate with data scientists and Reporting engineers to help resolve the reporting and Analytics needs
• Provide expertise and leadership in the development of new products, services, and processes from a Data Engineering technical perspective.
• Work independently adhering to high technical standards, taking ownership of projects ensuring they conform to departmental standards before going to QA.
• Work effectively within an agile, deadline-driven production process to complete project development with rapid, iterative cycles.
• Liaise with Team Leaders, Product and Project Managers to ensure projects are taken from creation to completion - on time and on budget.
• Provide accurate estimates for existing and new projects alike.