Senior Data Engineer
About this position
CJ MORE is a leading Thai retailer aiming to differentiate itself through innovative business strategies and strong store brands. The Senior Data Engineer will play a crucial role in designing and implementing data solutions to support the company's growth.
Responsibilities
• Design and implement data pipeline on data lake/data lakehouse concept
• Design and implement self-service tools in data platform (orchestration tool and data quality tool)
• Design and implement data validation process to improve data quality in data pipeline
• Monitor data platform cluster (Kubernetes)
• Design and implement data observability process to monitor data pipeline
• Design and implement data lineage process to tracking data movement
• Design and implement data models to support analysts
• Coaching other team members
Requirements
• 4+ years of experience in design and implementation data pipeline on data lake/data lakehouse concept
• Experience in Apache spark
• Experience in Apache airflow
• Manage multiple projects
• Understand data quality/data validation concept
• Strong programming skills (SQL, Python, Scala)
• Strong in using Docker, Kubernetes
• Experience in coding standard and testing
• Have a growth mindset, and willing to learn new things or share knowledge with others
• Understand data lineage concept is a plus
• Understand data observability concept is a plus
• Understand data mesh concept is a plus
• Data analytics engineer experience is a plus