DevOps Engineer - AI Model Evaluator
$85 por horaMercor
Lisboa
- Trabalho remoto
About the job
Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .
Position: DevOps / SRE / Cloud Engineer (Coding Agent Experience)
Type: Contract
Compensation: $85/hour
Location: Remote
Role Responsibilities
- Use frontier AI coding agents to complete and evaluate complex infrastructure engineering tasks.
- Review model-generated implementations involving cloud platforms , Kubernetes , CI/CD systems , observability , and infrastructure automation .
- Identify bugs, edge cases, reliability issues, and failure modes.
- Compare outputs from multiple frontier models and assess their strengths and weaknesses.
- Apply professional engineering judgment to realistic infrastructure engineering scenarios.
- Work independently and asynchronously to meet sprint-based project deadlines.
Qualifications
Must-Have
- 2+ years of professional DevOps , SRE , or Cloud Engineering experience.
- Experience with AWS , Azure , GCP , Kubernetes , Terraform , CI/CD pipelines , or observability tooling .
- Regular use of AI coding agents such as Cursor, Claude Code, Codex, Windsurf, Gemini CLI, or similar tools.
- Ability to evaluate model-generated infrastructure and reliability engineering solutions.
Preferred
- Experience supporting production-scale systems.
Compensation & Legal
- $400 per accepted task .
- Compensation is tied to accepted work.
Application Process (Takes 20–30 mins to complete)
- Upload resume
- AI interview based on your resume
- Submit form
Resources & Support
- For details about the interview process and platform information, please check:
- For any help or support, reach out to: Mostrar e-mail
PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
Vaga publicada 5 horas atrás
Empregos semelhantes que podem ser interessantes para vocêCom base na vaga DevOps Engineer - AI Model Evaluator em Lisboa
$85 por hora
...technical talent with leading AI research labs.... ...Dorsey . Position: DevOps / SRE / Cloud Engineer (Coding Agent Experience)... ...coding agents to complete and evaluate complex infrastructure engineering... ...engineering tasks. Review model-generated implementations...Trabalho remoto$40 por hora
...specialists with project-based AI opportunities for leading... ...companies, focused on testing, evaluating, and improving AI systems. Participation... ...coding agents - how well a model handles real-world developer... ...data labeling Not prompt engineering Not writing code from...$18 a $42 por hora
...technical talent with leading AI research labs. Headquartered in... ...Role Responsibilities Evaluate AI-generated music across... ...deadlines while improving AI model performance . Collaborate... ...experience as a music producer, audio engineer, or mixing engineer. ~...Trabalho remoto- ...English (B2). Have experience in AI data annotation, LLM evaluation, RLHF, SFT, Quality Assurance, Content... ...Generative AI technologies, Large Language Models (LLMs), and annotation platforms.... ...and consistent labels for language model outputs. Assess content for quality...
$60 a $70 por hora
...creative and technical talent with leading AI research labs. Headquartered in San... ...Remote Role Responsibilities Evaluate AI-generated responses for safety, factual... ...Provide structured feedback to improve model alignment and safety performance. Collaborate...Trabalho remoto$14 por hora
...Portugal Portuguese AI Product Evaluator The Project Productive Playhouse is building a talent... ...user interaction with the various AI models. Open to freelancers based outside... ...insights and data will directly inform model development and optimization. Exact...€1,100 por mês
...Lisbon, PortugalRole: German speaking AI Language Evaluation SpecialistSalary: salary package 1100 euro... ...and safety of next-generation language models. This is a fantastic opportunity for... ...generated responsesLabel and annotate language model outputs according to detailed...- ...German AI Content Evaluator | Lisbon Interested in Generative AI and Large Language Models? Join a project evaluating AI-generated content, datasets and written responses for quality and accuracy. ~€1,100 gross × 14 payments ~€7.63 daily meal allowance...
- ...results. As AI becomes central... ...Integrate models with web applications... ...collaborative engineering practices.... ...validation, guardrails, model routing and... ...for retrieval, evaluation and application... ...The AI Engineer will design, develop... ...understanding of DevOps practices and...
LBC - Leadership Business Consulting
Lisboa25 dias atrás $40 por hora
...Mindrift está à procura de um/a Freelance Agent Evaluation Engineer para colaboração em projetos de avaliação de agentes de AI aplicados a tarefas de desenvolvimento de... ...critérios de avaliação para testar a capacidade de modelos de AI em resolver desafios realistas de...- ...French AI Content Evaluator | Lisbon Interested in Generative AI and Large Language Models? Join an international team evaluating AI-generated content, datasets and written responses for quality and accuracy. ~€930 gross × 14 payments ~€7.63 daily meal allowance...
- ...We're looking for a DevOps Engineer to help develop and maintain infrastructure built on GitOps principles. The focus is on automating CI/CD processes, managing Kubernetes clusters, and implementing delivery workflows based on ArgoCD and FluxCD. You'll work closely with...
Axiros Portugal
Lisboa12 dias atrás $90 por hora
...technical talent with leading AI research labs. Headquartered in... .... Position: Frontend Engineer Type: Contract... ...visual fidelity in detail: box model and spacing, typography (family... ..., preference-labeling, model-evaluation, or structured code-review work...Trabalho remoto- ..., Cloud Migration, and DevOps Modernization. Notable... ...Operations Management using AI/ML and Automation and... ...an early-career AI/ML Engineer who is excited to grow... ...production, and support the evaluation and monitoring loops... ...code and AI/model code. ~ Hands-on experience...
- ...career growth! We are looking for a Senior AI Engineer to integrate a hybrid project in Lisbon... ...and Deep Learning techniques; - Evaluate and validate the performance of PoCs, considering... ...teams; - Optimize and fine-tune AI models and algorithms to meet performance and...
Olisipo
Lisboa7 dias atrás €65,000 a €75,000 por año
...researchers, Riskified’s AI-powered fraud and... ...a Senior AI Engineer to join a talented... ...establish guardrails and evaluation frameworks that... ...— spanning models, tools, APIs, and... ...value and ensure model reliability. Cultivate... ...ownership, some DevOps experience, and the...- ...goods, finance, and investment. Who You Are You are a DevOps engineer with software development background who likes working on everything... ...with Sensor Tower. We may use artificial intelligence (AI) tools to support parts of the hiring process, such as...
- ...Degree in Computer Engineering, Artificial Intelligence... ...experience developing AI-based solutions; Hands... ...Mistral, or equivalent models; Ability to design effective... ...and optimize model performance for production... ...Understanding of AI quality evaluation, latency optimization,...
ITSector - Sistemas de Informação, SA
Lisboa19 dias atrás - ...personalizados e soluções end-to-end. Procuramos um Platform Engineer | DevOps para um projeto em Lisboa ou Porto (hibrido).... ...e dashboards com Power BI, BIRT ou JasperReports, incluindo modelação de dados; - Experiência na construção de workflows de automação...
Dellent
Lisboa14 dias atrás - ...remote We are looking for a Senior AI Engineer to design and build production-grade AI... ...and other AI capabilities ~ Design evaluation frameworks and test cases to measure the... ...Build reusable components for prompting, model interaction, evaluation, retrieval, and...
- ...criar tecnologia com impacto. DEVOPS ENGINEER (IA) O que estamos à... ...agentes autónomos e Large Language Models (LLMs). O que vais fazer... ...LLMs e frameworks de Agentic AI. Definir métricas de... ..., fine-tuning e avaliação de modelos. Experiência na otimização...
- ...specialists providing services in Information Technology and Engineering services. As part of the Netherlands-based organization... ...us at . Role Overview: We are looking for a DevOps/Platform Engineer to join our teams and support one of our clients in building...
Neotalent Conclusion
Lisboa11 dias atrás - ...Milestone está à procura de um(a) colega com experiência como DevOps Engineer para integrar um cliente na área das Telecomunicações. O que... ...de equipa, sentido crítico e capacidade de iniciativa; Modelo de trabalho: Híbrido; Disponibilidade para uma prevenção mensal...
Milestone Consulting
Lisboa27 dias atrás - ...and French is a plus but not mandatory; - Availability to work 2 times per week in Lisbon; - Bachelor's Degree in Computer Engineering or similar. Our agapian incentives are: - Health insurance; - Life and personal accident insurance; - Free training...
- ...); me.happy(true); } We are an IT Consultancy focused on Engineering projects associated with information Technologies. We operate... ...#daretochange Dare to Have: Minimum 2 years of experience in DevOps; Background in software development is valued, but not...
- ...Engenharia Informática, Tecnologias de Informação ou área similar; Experiência profissional mínima de 3 anos em funções de DevOps ou Cloud Engineering ou área similar; Experiência com AWS ou Azure e ambientes containerizados com Kubernetes; Conhecimentos de...
- ...- IT Team Augmentation — helping clients scale and strengthen their tech teams. The Project We are seeking a DevOps Engineer to work on a project for a Tech Company. You will have the following responsibilities: Support and evolution of...
- ...#daretochange. Dare to Have: Experiência mínima de 2 anos em DevOps; Background em desenvolvimento de software é valorizado,... ..., pacote de dados e kit de boas-vindas; Diversity | Na Bee Engineering valorizamos a diversidade, defendemos a equidade e empenhamo-...
€33,550 a €34,000 por año
...ser tão humana quanto técnica. Na Olisipo, tens os dois lados. Vem conhecer a nossa forma de trabalhar! Procuramos um Senior DevOps Engineer para integrar um projeto híbrido em Lisboa. Intervalo salarial: 33 550,00€ - 34 000,00€ bruto anual. Perfil: -...Olisipo
Lisboa12 dias atrás- ...technologies. We’re looking for a passionate and skilled DevOps Engineer who thrives in a deploy-as-code environment to drive the development... ...and career. Work-Life Balance: Flexible hybrid work model with a focus on personal and family needs. Global...
Deseja receber mais vagas?
Assine e receba vagas semelhantes a DevOps Engineer - AI Model Evaluator. Seja o primeiro a se candidatar!
