---

About this position

Senior Data Engineer will be responsible for creating and delivering the strategic enterprise data platform roadmaps for data engineering solutions and services. This will involve building and maintaining data pipelines, integrations, transformations, performance tuning, data governance, and setting monitoring tools.

Responsibilities

• Responsible for overall data architecture design, and implementation of data platform including Data Lake, Data Lakehouse in the multi-cloud environments.
• Design and develop the E2E data pipeline: data transformation, ingestion, processing, and surfacing of data for large-scale applications.
• Analyze source data and perform data ingestion in both batch and real-time patterns via various methods; for example: file transfer, API, Data Streaming using Kafka and Spark Streaming.
• Define and implement the data pipelines for real-time and batch processing across layers in the data platform.
• Participate in cluster performance tuning to continually improve the quality of data pipeline and overall data platform.
• Apply software engineering rigor and best practices to data and ML services, including CI/CD, automation, testing, and Infrastructure as Code.
• Design data organization structures and taxonomies from multiple data sources for optimal use and enabling data quality and governance.
• Work collaboratively with the data governance team to maintain data integrity and data availability across data domains.
• Work with sale team to support presale activities related to TechX data platform and data engineering solutions.
• Establish strong collaboration with internal and external stakeholders to ensure project delivery.
• Ensure appropriate strategy/solution is proposed to customer and monitor actions and results against plans.
• Provide demonstrations, presentations, training, and consultation for potential clients.

Requirements

• Bachelor's degree or equivalent experience in Computer Engineering, Computer Science, Information Technology, or related fields.
• 5+ of Experience in Data Engineer, Data Architecture, Data Modeler, or Data Management with large-scale data in the distributed data platform and overseeing complex data projects in an enterprise environment.
• Excellent proficiency in programing languages (Python, PySpark, SQL, R) and experience with business intelligence solutions (Tableau, PowerBI or Advanced Excel).
• Full stack experience in

Benefits