Data Engineer (Senior - Manager)
Detail posisi
Kewajiban
• Designing data warehouse using star schema, snowflake, and data vault.
• Assemble large, complex data sets that meet functional / non-functional business requirements.
• Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, infrastructure, etc.
• Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources.
• Develop tools for processing in “big data” platforms.
• Work closely with stakeholders/ team member such as Data Scientist/ Developer/ Business domains.
• Identify and troubleshoot any issue in data pipelines.
• Analyzing and translating business needs into long-term solution data models.
• Evaluating existing data systems.
• Working with the development team to create conceptual data models and data flows.
• Developing best practices for data coding to ensure consistency within the system.
• Reviewing modifications of existing systems for cross-compatibility.
• Implementing data strategies and developing physical data models.
Kualifikasi
• Bachelor’s degree or higher in Computer Engineering, Computer Science, or IT related fields.
• Experience in Cloud Platform (AZure, AWS, GCP).
• Knowledge of database tools (Sqoop, Apache, Nifi, Hive).
• Experience working with non-relational/relational databases (SQL, MySQL, NoSQL, Hadoop, Hdfs, Spark etc.).
• Experience in data warehouse.
• Experience building and improving ‘big data’ data pipelines, architectures and data sets is preferred.
• Able to build processes supporting data transformation, data structures, metadata, dependency and workload management.