Integrating data from diverse sources into the enterprise data lake.
Implementing solutions for handling and analyzing unstructured data.
Enforcing data governance policies and ensuring compliance with data privacy regulations and implementing security measures to protect sensitive data.
Collaborating with cross-functional teams to understand business requirements and align data solutions with organizational goals.
Providing technical leadership and mentorship to junior data engineers.
Planning and implementing scalable data infrastructure to accommodate growing data volumes.
Evaluating and adopting new technologies to enhance data engineering capabilities.
Implementing monitoring solutions to track the performance of data systems.
Proactively identifying areas for optimization and implementing improvements.
Creating and maintaining comprehensive documentation for data architectures and processes.
Identifying opportunities for process improvements and implementing best practices.
Staying informed about emerging trends and technologies in the data engineering field.
Designing and implementing complex ELT/ETL processes for large-scale data integration, optimizing ELT/ETL workflows for performance, scalability, and resource efficiency
Working with distributed computing frameworks and big data technologies (e.g., Hadoop, Spark) for processing large datasets.
Designing, implementing, and managing data warehouses for efficient data storage and retrieval.
Creating and maintaining data models to support business requirements.
Implementing and optimizing database schemas for performance and scalability.
Conducting performance analysis and implementing optimizations for database queries and data processing.
Identifying and resolving performance bottlenecks in data pipelines.
Facilitating knowledge sharing within the team and across departments.
Requirements
Degree in Computer Science, Data Engineering or equivalent
A min. of 3-5 years experience (banking & insurance experience is a plus)
Proficiency in SAS, Python, SQL
Hands-on experience with Hadoop, Spark or similar technologies for large-scale data processing
Understanding of data quality checks, monitoring, and testing procedures
Min. 3 years experience in analytics related roles (banking experience is a plus)
Leadership
Agile development and DevOps operating practices
Listening and communication/storytelling
Critical and evaluative thinking
Proactive approach with ability to work independently
Business
View of what data is needed to be collected, analyzed across business solutions