Data Engineer
Employee | Tech | Professional | Philippines | Taguig | 2024-04-19 | REQ-10066790
Data Engineer [for Project Domino; preferably senior 5 years and up experience]
Responsibilities
- Design, architect and maintain secure data platforms such as setting up data storage solutions like SQL databases
- Implement and maintain optimized workflows, ETL / ELT processes, and pipelines; deliver flexibility for data consumption by designing batch and streaming data use cases
- Ensure support for traceability and reproducibility through enabling of data lineage features for ingested data
- Work with colleagues such as Data Scientists, Data Analysts, software and data engineers, and architects on implementing solutions involving data on distributed systems and platforms
- Create highly available and scalable capabilities that deliver data related tooling and manage data platforms and pipelines for stakeholders
Qualifications
- Proficiency and experience in software engineering programming languages such as Python. Experience in Golang is preferred
- Expertise in SQL / RDBMS databases and proficiency in setting up these resources. Additional experience in NoSQL databases and setting up these is preferred
- Extensive experience in design and implementation of data handling processes and pipelines (ingestion, transformation, modelling, and storage) using Python, Apache Airflow, Spark
- Expertise in cloud platforms and technologies (Azure, AWS, GCP) is a plus but not required
- Experience with streaming platform setup and consumption such as Kafka, Flink
- Experience with consumption from other storages for streaming such as S3, Spark
- Experience in containers and container orchestration platforms such as Docker and Kubernetes
- Experience in CI / CD and Devops, scripting languages such as shell and automation
- Preferably a degree in the related fields of Computer Science
- Strong problem solving and solutions engineering mindset
- Experience in working and coordinating remotely