---
title: "ML Inference Engineer – Terminal-Bench Task Designer (vLLM / SGLang / TensorRT-LLM) at Toptal – All Job Board"
---

# ML Inference Engineer – Terminal-Bench Task Designer (vLLM / SGLang / TensorRT-LLM) at Toptal

Written by

[admin](https://alljobboard.com/author/admin/)

in

- [Anywhere in the World (Remote)](https://maps.google.com/maps?q=Anywhere%20in%20the%20World%20%28Remote%29&zoom=14&size=512x512&maptype=roadmap&sensor=false)
- Posted 1 week ago

![Toptal](https://alljobboard.com/wp-content/plugins/wp-job-manager/assets/images/company.png)

 **Toptal**

Overview

We are building a benchmark that tests how well AI agents handle real expert work in a terminal. We are hiring ML inference and serving experts to turn real, paid tasks from their jobs into expert-level Terminal-Bench challenges. Each task includes the environment, a working solution, and an automated check that proves the result is correct.

\*\*\*THIS IS A TASK BASED COMPENSATION MODEL. PLEASE CHECK THE APPLICATION QUESTIONS FOR MORE DETAILS.\*\*\*

Responsibilities

• Design expert-level Terminal-Bench tasks that turn real, paid ML inference work from your own experience into challenges an AI agent must solve end to end.

• Define the files, systems and environments each task runs on, such as model serving with streaming, so the challenge is realistic, reproducible and command-line solvable.

• Write a working reference solution for every task and verify it runs end to end, sized for about 60 to 75 minutes of expert effort.

• Build automated checks that objectively prove a result is correct, including cases where ML outputs go wrong silently, such as numerical drift or metric bugs.

• Package tasks with Docker and Git so they are reproducible, and document the systems, tools, data and rules involved so reviewers can validate difficulty.

• Iterate on tasks based on reviewer feedback and agent test runs, adjusting difficulty and fixing ambiguities so each task stays expert-level, fair and objectively gradable.

Skills/Experience

• 3+ years doing ML inference or serving work hands-on (not only studying, selling or managing it), or a PhD plus active research in the subfield

• Did this work in the last 3 years and can name the systems, tools, data and rules used

• Trained, served or evaluated models at real scale (multi-GPU or production serving), not only notebooks or API calls

• Hands-on with PyTorch or JAX internals and serving stacks such as vLLM, SGLang or TensorRT-LLM

• Can explain how ML results go wrong without errors (numerical drift, metric bugs) and how to catch them

• Technical fluency in Python and the command line, and with files, CSV/JSON, APIs or databases

• Availability to start by early November 2026 and complete tasks of about 60 to 75 minutes each

Nice-to-have

• AI lab or LLM infrastructure experience

• Contributions to PyTorch, vLLM or SGLang

• GPU profiling experience

• Docker and Git proficiency

• Experience writing automated tests

Process

• Task vetting instead of an interview: complete a vetting task so the client can assess your task design

• Paid per task; each task takes about 60 to 75 minutes

• Work is delivered task by task, with reviewer feedback on submissions

• Expected start in early November 2026

Please mention the word \*\*ASTONISHED\*\* and tag RMjAwMTo0MWQwOjMwMzplYjJjOjo= when applying to show you read the job post completely (#RMjAwMTo0MWQwOjMwMzplYjJjOjo=). This is a beta feature to avoid spam applicants. Companies can search these words to find applicants that read this and see they’re human.

**Source:** this listing was published by [Remote OK](https://remoteok.com/). [View the original posting and apply](https://remoteOK.com/remote-jobs/remote-ml-inference-engineer-terminal-bench-task-designer-vllm-sglang-tensorrt-llm-toptal-1137689) — applications are handled there, not on this site.

To apply for this job please visit [remoteOK.com](https://remoteOK.com/remote-jobs/remote-ml-inference-engineer-terminal-bench-task-designer-vllm-sglang-tensorrt-llm-toptal-1137689).

## More posts

- ### [Holiday Party Networking: How to Job Hunt Without Making It Awkward](https://alljobboard.com/holiday-party-networking-job-search/)

[November 13, 2026](https://alljobboard.com/holiday-party-networking-job-search/)
- ### [How to Negotiate a Start Date After the Holidays (With Scripts)](https://alljobboard.com/negotiate-start-date-after-holidays/)

[November 13, 2026](https://alljobboard.com/negotiate-start-date-after-holidays/)
- ### [Should You Quit Before or After Your Year-End Bonus?](https://alljobboard.com/quit-before-or-after-year-end-bonus/)

[November 13, 2026](https://alljobboard.com/quit-before-or-after-year-end-bonus/)
- ### [How to List a Seasonal Job on Your Resume (With Examples)](https://alljobboard.com/seasonal-job-on-resume/)

[November 13, 2026](https://alljobboard.com/seasonal-job-on-resume/)

```json
{
    "@context": "http://schema.org/",
    "@type": "JobPosting",
    "datePosted": "2026-11-06T18:18:44+00:00",
    "validThrough": "2026-12-05T23:59:59+00:00",
    "title": "ML Inference Engineer – Terminal-Bench Task Designer (vLLM / SGLang / TensorRT-LLM) at Toptal",
    "description": "<p>Overview</p>\n<p>We are building a benchmark that tests how well AI agents handle real expert work in a terminal. We are hiring ML inference and serving experts to turn real, paid tasks from their jobs into expert-level Terminal-Bench challenges. Each task includes the environment, a working solution, and an automated check that proves the result is correct.</p>\n<p>***THIS IS A TASK BASED COMPENSATION MODEL. PLEASE CHECK THE APPLICATION QUESTIONS FOR MORE DETAILS.***</p>\n<p>Responsibilities</p>\n<p>• Design expert-level Terminal-Bench tasks that turn real, paid ML inference work from your own experience into challenges an AI agent must solve end to end.</p>\n<p>• Define the files, systems and environments each task runs on, such as model serving with streaming, so the challenge is realistic, reproducible and command-line solvable.</p>\n<p>• Write a working reference solution for every task and verify it runs end to end, sized for about 60 to 75 minutes of expert effort.</p>\n<p>• Build automated checks that objectively prove a result is correct, including cases where ML outputs go wrong silently, such as numerical drift or metric bugs.</p>\n<p>• Package tasks with Docker and Git so they are reproducible, and document the systems, tools, data and rules involved so reviewers can validate difficulty.</p>\n<p>• Iterate on tasks based on reviewer feedback and agent test runs, adjusting difficulty and fixing ambiguities so each task stays expert-level, fair and objectively gradable.</p>\n<p>Skills/Experience</p>\n<p>• 3+ years doing ML inference or serving work hands-on (not only studying, selling or managing it), or a PhD plus active research in the subfield</p>\n<p>• Did this work in the last 3 years and can name the systems, tools, data and rules used</p>\n<p>• Trained, served or evaluated models at real scale (multi-GPU or production serving), not only notebooks or API calls</p>\n<p>• Hands-on with PyTorch or JAX internals and serving stacks such as vLLM, SGLang or TensorRT-LLM</p>\n<p>• Can explain how ML results go wrong without errors (numerical drift, metric bugs) and how to catch them</p>\n<p>• Technical fluency in Python and the command line, and with files, CSV/JSON, APIs or databases</p>\n<p>• Availability to start by early November 2026 and complete tasks of about 60 to 75 minutes each</p>\n<p>Nice-to-have</p>\n<p>• AI lab or LLM infrastructure experience</p>\n<p>• Contributions to PyTorch, vLLM or SGLang</p>\n<p>• GPU profiling experience</p>\n<p>• Docker and Git proficiency</p>\n<p>• Experience writing automated tests</p>\n<p>Process</p>\n<p>• Task vetting instead of an interview: complete a vetting task so the client can assess your task design</p>\n<p>• Paid per task; each task takes about 60 to 75 minutes</p>\n<p>• Work is delivered task by task, with reviewer feedback on submissions</p>\n<p>• Expected start in early November 2026</p>\n<p>Please mention the word **ASTONISHED** and tag RMjAwMTo0MWQwOjMwMzplYjJjOjo= when applying to show you read the job post completely (#RMjAwMTo0MWQwOjMwMzplYjJjOjo=). This is a beta feature to avoid spam applicants. Companies can search these words to find applicants that read this and see they&#8217;re human.</p>\n<p class=\"job-source\"><strong>Source:</strong> this listing was published by <a href=\"https://remoteok.com/\" rel=\"dofollow\">Remote OK</a>. <a href=\"https://remoteOK.com/remote-jobs/remote-ml-inference-engineer-terminal-bench-task-designer-vllm-sglang-tensorrt-llm-toptal-1137689\" rel=\"dofollow\">View the original posting and apply</a> &mdash; applications are handled there, not on this site.</p>\n",
    "hiringOrganization": {
        "@type": "Organization",
        "name": "Toptal"
    },
    "identifier": {
        "@type": "PropertyValue",
        "name": "Toptal",
        "value": "https://alljobboard.com/job/ml-inference-engineer-a%c2%80%c2%93-terminal-bench-task-designer-vllm-sglang-tensorrt-llm-at-toptal/"
    },
    "jobLocation": {
        "@type": "Place",
        "address": "Anywhere in the World"
    },
    "jobLocationType": "TELECOMMUTE",
    "directApply": true
}
```