Data Engineer
Overview
The Data Engineer is responsible for building, supporting, and optimising scalable, repeatable, and secure data pipelines and analytical platforms. The role focuses on enabling business insights through the integration, transformation, and management of large-scale batch and real-time data solutions.
Requirements
- 5+ years’ experience in Data Engineering or Software Engineering roles
- 2+ years’ experience working with Big Data technologies and platforms
- 2+ years’ experience with ETL processes and tooling
- 2+ years’ experience working with AWS cloud technologies
- Strong experience with Python for data manipulation and processing
- Hands-on experience with AWS services including EMR, EC2, and S3
- Experience with PySpark, Spark, Hadoop, or distributed data processing frameworks
- Strong understanding of data modelling, data structures, and scalable data architecture design
- Experience designing highly scalable distributed systems using open-source technologies
- Strong understanding of object-oriented design, coding standards, and testing practices
- Experience working with large-scale data infrastructure and analytical platforms
- Strong analytical, troubleshooting, and problem-solving skills
- Bachelor’s degree in Computer Science, Computer Engineering, Information Technology, or equivalent practical experience
Responsibilities
- Design, build, and maintain scalable data analytics frameworks and data platforms
- Translate complex functional and technical requirements into scalable architecture and high-performing data solutions
- Develop and support batch and real-time data processing solutions using modern big data technologies
- Create and maintain secure and reliable data pipelines between on-premise systems and AWS cloud environments
- Develop and maintain ETL processes and data transformation workflows using Talend or similar ETL tools
- Manipulate, process, and analyse data using Python and related technologies
- Develop and support Big Data and Business Intelligence solutions including automated testing and deployment practices
- Process large-scale datasets using Hadoop paradigms and AWS EMR services
- Support production data feeds and resolve operational issues through break-fix support activities
- Contribute to database development, optimisation, and operational management
- Participate in software development, DevOps, and data operations activities
- Ensure alignment with policies, standards, procedures, business continuity, and disaster recovery practices
- Research and evaluate emerging technologies, tools, and frameworks relevant to data engineering and analytics
- Collaborate with business and technical stakeholders to understand data requirements and deliver effective solutions
How to apply
Verify before you apply
SpanSam summarises opportunities for easier discovery. Always confirm the closing date, eligibility requirements and submission instructions on the original source before sending personal information or documents.
About this listing: SpanSam is an opportunity discovery service and is not the hiring employer unless explicitly stated. Application decisions and source-listing changes are controlled by the employer or institution.