Cloud Data Engineer

1 day ago Be among the first 25 applicants We are recognized for our talents as an Excellent Company to Work For. Join a team that values people, invests in continuous development, and lives technology with passion! We are looking for a Cloud Data Engineer with proven experience in data engineering in a cloud environment to join a project in the financial sector. Hybrid project - Porto Requirements: At least 3 years of experience in developing and implementing data pipelines on public cloud platforms; At least 1 year of experience in architecture, design, development, and support of data solutions, demonstrating the ability to design scalable and efficient structures; Solid knowledge of database architecture, ETL processes, business intelligence, big data, machine learning, and advanced analytics; Practical experience with the Spark ecosystem, with proficiency in Hadoop considered a plus; Knowledge of pipeline orchestration tools, such as Airflow, Step Functions, or AWS Workflow, ensuring the automation and monitoring of data processes; Mastery of Lakehouse architecture implementation, with a focus on Iceberg, with knowledge of Hudi and Delta, as well as optimization techniques (Compaction, Z-Order, etc.); Proficiency in Python and SQL, applying best practices in development, automation, and continuous integration (CI/CD); Experience with containers (Docker) and infrastructure as code, using tools such as AWS CloudFormation and Terraform to automate the creation and maintenance of cloud resources; Have in-depth knowledge of key AWS services, including Lambda, Glue, Crawlers, Step Functions, DynamoDB, and AWS Batch; Fluency in English. Responsibilities: Advise on strategy, architecture, and data implementation in a cloud environment; Design and develop data pipelines in AWS (and other public clouds when applicable); Implement and optimize Lakehouse architectures (preferably Iceberg, with Hudi and Delta being an added value); Build, automate, and maintain ETL processes, workflows, and orchestration (Airflow, Step Functions, AWS Workflow); Ensure reliability, automation, security, and maintenance of all delivered solutions; Support incident resolution, root cause analysis, and preventive actions; Share best practices and knowledge with internal teams; Collaborate with Delivery Managers, Architects, PMs, Developers, and Business Analysts in an agile and collaborative environment. We offer: Health insurance; Free online training through the Udemy platform; On-site and remote training at Olisipo's Learning Center; Free certifications (after passing the exam); Discounts at Olisipo Partners (in the areas of health and well-being, fitness, travel, among others); Free psychological consultations; Possibility of a salary advance, without commissions. Apply and get to know about this project! Seniority level Mid-Senior level Employment type Full-time Job function Information Technology Industries IT Services and IT Consulting Referrals increase your chances of interviewing at Olisipo by 2x Get notified about new Data Engineer jobs in Porto, Portugal . We're unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI. #J-18808-Ljbffr