Data Engineer

Description

Summary: A Data Engineer designs, builds, and maintains data pipelines and platforms to reliably collect, transform, and deliver data for analytics and machine learning, focusing on quality, scalability, performance, and governance. Highlights: 1. Design and build robust production data pipelines 2. Focus on data quality, scalability, and operational excellence 3. Utilize cloud data stacks and modern modeling approaches **Data Engineer — Job Description** **Role summary** A Data Engineer builds and maintains data pipelines and platforms that collect, transform, and deliver data reliably for analytics and machine learning. The role focuses on data quality, scalability, performance, governance, and operational excellence. **Key responsibilities** * Design, build, and maintain data ingestion pipelines (batch and streaming). * Create data models and transformation layers (ETL/ELT) for analytics use. * Develop and optimize SQL transformations and data processing jobs. * Build reliable data workflows with orchestration (scheduling, dependencies, retries). * Implement data quality checks, validation, and reconciliation processes. * Maintain data warehouses/lakehouse structures and performance (partitioning, indexing, incremental loads). * Manage/operate pipelines in cloud or on\-prem environments; support production operations. * Support BI and analytics teams with data access patterns, documentation, and troubleshooting. * Implement data governance practices (lineage, access controls, documentation, retention where applicable). * Ensure security best practices for sensitive data (encryption, least privilege, auditing). * Set up monitoring/alerting for pipeline failures and data anomalies. * Document pipelines and maintain runbooks; improve processes over time. **Required qualifications** * Strong experience designing and building production data pipelines. * Proficiency in SQL and at least one programming language (Python/Scala/Java preferred). * Experience with ETL/ELT and data modeling (dimensional modeling and/or modern modeling approaches). * Hands\-on experience with workflow orchestration tools (e.g., Airflow, Dagster, Prefect). * Experience with at least one cloud data stack or equivalent technologies. * Knowledge of data quality techniques and pipeline troubleshooting/incident handling. * Understanding of data security and access control concepts. **Preferred qualifications** * Experience with cloud services (AWS/Azure/GCP) and data platforms (warehouse/lakehouse). * Experience with streaming (Kafka/Kinesis/PubSub/Flink/Spark Structured Streaming). * Familiarity with modern data governance tooling (catalog/lineage). * Experience with performance tuning for large datasets. * Exposure to ML/feature engineering pipelines (feature stores, training data sets). * Certifications for relevant cloud/data technologies (if applicable). **Skills and competencies** * Strong analytical thinking and problem\-solving. * Ability to design for scalability, reliability, and maintainability. * Clean engineering practices (testing, version control, CI/CD). * Clear communication with analytics, engineering, and stakeholders. * Ownership of data quality and operational readiness. Pay: RO799\.450 \- RO1,362\.340 per month Work Location: In person