Machine Learning Engineer

You build the systems that put models into production — the engineering half of AI that the boom actually runs on. Compensation is among tech's best. The honest part: it's mostly rigorous software engineering plus data plumbing, and the bar to entry is the highest on this site.

Median pay (US)

~$135k / yr — big tech far higher

Typical range

$95k–$200k+

Degree required?

Usually — or a strong SWE record

What the job actually is

ML engineers take models — trained in-house or, increasingly, called from providers like Anthropic and OpenAI — and make them work reliably in production: data pipelines, training and evaluation infrastructure, deployment, monitoring, cost and latency budgets. The role sits between data science (which explores and prototypes) and platform engineering (which runs systems at scale), and in the LLM era it's absorbed a new job: building evals, retrieval systems, and agent scaffolding around foundation models. Less whiteboard math than the branding suggests; more distributed systems, data quality, and 'why did the model do that in prod.'

Is it actually you?

You'll probably love it if

You're a solid software engineer who wants the ML layer, not an escape from engineering
Debugging a system whose failures are statistical, not deterministic, sounds fascinating
You like owning things end to end — data in, predictions out, pager included
Evaluation rigor appeals to you: 'it seems better' isn't good enough
You want to work where the industry's energy and budgets currently are

Maybe not, if

You're hoping to skip programming fundamentals and 'get into AI' directly — this role is engineering first
You want pure research — that's a different job with a different (usually PhD) gate
Ambiguity bothers you: models fail in ways no stack trace explains
You'd resent the unglamorous majority of the work being data pipelines

The real day-to-day (no hype)

The proven door is software engineering. Most ML engineers were backend or data engineers first. The realistic path: strong SWE fundamentals, then ML coursework plus real projects, then an ML-adjacent team at your current company. 'ML engineer, no engineering experience' postings are essentially fiction.
The LLM era split the role. Classical MLE (training and serving your own models — recommender systems, fraud, forecasting) still runs commerce. The fast-growing branch builds on foundation models: retrieval, evals, fine-tuning, agents. The second has a lower math bar and a higher product bar — and is where much of the hiring is.
Evaluation is the actual moat. Anyone can call a model API. Engineers who can build honest evals — measure quality, catch regressions, tie model behavior to business outcomes — are what separates AI products that work from demos that stall. It's the least glamorous, most hireable skill in the field.
Comp is high and so is the churn. The field reinvents its tooling roughly every eighteen months, titles are inconsistent (MLE, AI engineer, MLOps mean different things per company), and interview processes are demanding — often full SWE loops plus ML system design. The reward: some of the strongest compensation and mobility in software.

How people break in — or switch in

From software engineering: move toward the data — take on pipeline, experimentation, or model-serving work where you are, ship one production ML feature, and let that project headline your résumé. From data science: close the engineering gap (testing, infra, deployment) — it's the most common failed interview. From outside tech: be honest that this is a two-step journey through a software or data engineering role first. In all cases, one deployed project with real users and an eval story beats certificates.

Every path in runs through demonstrated engineering. The winning artifact is the same from any origin: a production system with a model in it, and a crisp story about how you measured whether it worked.

Your application, already half-written

Here's a question every Machine Learning Engineer application asks, answered the way pirch would — in a real voice, grounded in real experience:

“Tell us about a time a model performed well offline but failed in production.”

Our churn model looked great in offline eval — AUC up eight points over the baseline — but two weeks after deploy, retention outreach conversion actually dropped. I dug in and found a feature leak: one input, 'days since last support ticket,' was computed differently in the serving path than in training, effectively telling the offline model the future. The deeper failure was process, not code — nothing in our pipeline verified train/serve consistency. I fixed the feature, then built the guardrail: a daily job comparing feature distributions between training snapshots and live serving, alerting on drift. It caught two more skews in the next quarter before they shipped. What I took from it: offline metrics are a hypothesis, and the real deliverable is the harness that tells you when production disagrees.

pirch's co-pilot writes answers like this for your background and the exact job — try it free →

pirch finds the ML roles that are actually you

Tell pirch who you are — your engineering background, the ML work you've actually shipped, classical or LLM-side — and it hunts down real, verified-open machine learning roles that fit the whole you, with a tailored cover letter already written.

start your free hunt

first hunt free · we never auto-apply · you stay in control

Common questions

How much do machine learning engineers make?

US median around $135k, with typical offers $95k–$200k and top-of-market companies well beyond that once equity is counted. LLM-focused roles at well-funded companies are currently the hottest segment.

Do I need a degree to become an ML engineer?

Most MLEs have a CS or related degree, and some teams still filter on it — but a strong software engineering record plus deployed ML projects beats a credential without one. The non-negotiable isn't the diploma; it's provable engineering depth.

What's the difference between a data scientist and an ML engineer?

Data scientists explore data and prototype models to answer questions; ML engineers build the systems that run models in production reliably. The boundary blurs at smaller companies, but the MLE role is engineering-weighted — deployment, monitoring, scale — where data science is analysis-weighted.

Is it too late to get into machine learning?

No — but the entry point moved. Competition for 'train models from scratch' roles is fierce, while demand for engineers who can build reliable products on top of foundation models (retrieval, evals, agents) outstrips supply. Enter through engineering strength and current-era skills rather than chasing the 2018 job description.