Data Engineer
About this position
Responsibilities
• Develop, construct, test, and maintain architectures such as databases and large-scale processing systems.
• Implement complex data processing pipelines as part of a balanced team according to business requirements.
• Utilize big data tools to improve data reliability, efficiency, and quality.
• Collaborate with data scientists and analysts to assist with data-related technical issues and support their data infrastructure needs.
• Ensure architecture will support the requirements of the business and data scientists.
Requirements
• Bachelor or Master Degree in Computer Science, Information Technology, Data Engineering.
• 3 above years of experience in a data engineering role, with expertise in building and optimizing ‘big data’ data pipelines, architectures, and data sets.
• Strong analytic skills related to working with unstructured datasets.
• Experience with big data tools: Hadoop, Spark, Kafka, etc.
• Experience with data pipeline and workflow management tools: Azkaban, Luigi, Airflow, etc.
• Proficiency in scripting languages: Python, Java, Scala.