Hiring-Manager Round

How would you design a multi-task system that handles varying domains?

By Interview Clozo · Published 27 October 2026 · Last verified 26 October 2026 · 10 min read

A multi-task system is one machine-learning stack, usually a shared backbone with small per-domain parts, trained across several domains, and it is the answer to "how would you design a multi-task system that handles varying domains?" Share and balance first, measure each domain, then contain interference with adapters. The rubric line that separates strong answers is naming negative transfer, when one domain's gain hurts another, and saying how you contain it.

In this article

Before you start
Why interviewers ask this
Multi-task design from first principles
The Staff-level answer
The rubric
Three variants of this question
Common mistakes
Follow-up questions
Sources

Before you start

This page reads the question as a machine-learning design question, not as a job scheduler or task queue. It is a technical probe in the hiring manager round.

Why interviewers ask this

The question tests whether you design for the messy part of production: several kinds of input competing for one model. A hiring manager hears how you think about shared cost, interference between teams' needs, and ownership when quality drops in one area. They also listen for your own part in such a design, which is what the question about your exact contributions to past projects probes.

A weak answer says "train one big model on everything." A strong answer separates sharing from isolating, says how domains are weighted, and says how each domain is measured so a regression is visible. A Staff-level answer adds a rule for when to stop sharing and split the model.

Multi-task design from first principles

Task head: A task head is a small output layer or module attached to a shared backbone that turns its general representation into the output one task or domain needs, such as a class label or a score.

In plain words

Think of one kitchen cooking for five restaurants. The ovens and knives are shared, so the cost is low. Each restaurant still has its own recipe card. The analogy breaks when one restaurant's rush starves the others of oven time; a model has no such manager unless you build one.

A backbone is the large shared part of a model that turns raw input into a general representation; the heads and adapters sit on top of it. In one precise sentence: a multi-task system shares most parameters across domains, adds a few domain-specific parameters, and controls how training effort is divided.

The main risk is negative transfer, where one domain loses quality while others train beside it, and adapters are the usual way to contain it. The design has two stages. Stage 1 trains the backbone and the heads together on all domains, with balanced sampling and per-domain measurement; Steps 1 to 3 belong here. Stage 2 freezes that backbone and adds adapters where a domain suffers or a new domain arrives; Step 4 belongs here.

Four moves cover the design: share, balance, measure, isolate.CC BY 4.0 — source: How would you design a multi-task system that handles varying domains?

Step-by-step description

One backbone, one head per domain.
Temperature sampling lifts small domains.
Evaluate each domain on its own.
Adapters contain interference.

A shared backbone feeds a task head for each domain.
Temperature sampling gives small domains a larger share.
Each domain is evaluated on its own held-out set.
Adapters contain negative transfer between domains.

Hard parameter sharing means one backbone, often a transformer encoder, is trained on all domains at once and each domain or task gets its own task head. Input: text, images or records from any domain. Output: a domain-specific prediction. Ruder (2017), an overview of multi-task learning in deep neural networks, introduces the two most common methods; the other, soft parameter sharing, gives each task its own model and keeps their parameters close with a penalty.

Sharing pays when domains are related, because the backbone learns from more data than any single domain has. It also means one model to deploy, monitor and patch. State this reason aloud in the interview: you share because the cost and the data are shared. You now have the baseline design, and its weakness is that every domain pulls the same shared weights, so one domain's gain can come at another's expense.

Step 2: Balance how often each domain is sampled

If you sample in proportion to data size, the largest domain dominates and small ones barely train. Temperature-scaled sampling is a way of choosing training examples that raises each domain's share to the power one over a temperature, so a higher temperature flattens the mix; it is a common practice when one model trains on many datasets or languages. In an illustrative mix of four domains with 1,000,000, 200,000, 20,000 and 5,000 examples, the smallest gets 0.41% of samples proportionally, 4.26% at temperature 2 and 11.51% at temperature 4.¹

Sampling matters in stage 1, where all domains share weights; frozen per-domain adapters do not compete with each other. Flattening is not free: it repeats small-domain examples more often, which can overfit them. Treat the temperature as a setting tuned on per-domain held-out results.

Step 3: Evaluate each domain on its own and route

Each domain is evaluated on its own held-out set, never only through a pooled average, because a pooled number can hide a drop in a small domain. Report one metric per domain on that set. Compare each domain against a single-domain baseline; if the shared model loses to it by more than your tolerance, that domain moves to Step 4: an adapter, a larger adapter, or a split from the shared backbone.

At serving time, route each request to its domain's head or adapter using a domain label or a lightweight classifier, and send unclear inputs to a safe default. Monitor per-domain quality after launch, since traffic mix drifts.

Step 4: Contain interference between domains

When one domain's updates hurt another, that is negative transfer, and stage 2 contains it. Once Step 3 shows a domain losing quality, one common containment is to freeze the shared backbone and train an adapter per domain. Houlsby et al. (2019) report that adapter modules add few trainable parameters per task (3.6% on the General Language Understanding Evaluation (GLUE) benchmark), leave the original network fixed, and let new tasks be added without revisiting old ones. They transferred BERT (Bidirectional Encoder Representations from Transformers) to 26 text classification tasks and, on GLUE, came within 0.4% of full fine-tuning.

Storage shows the gain. Take an assumed 110 million parameter backbone with adapters at 3.6% of its size (Houlsby's figure on BERT-large, so an illustrative scaling): full copies for 50 domains total 5,500 million parameters, and a backbone plus adapters totals 308 million.¹ Other options include mixture-of-experts routing, which sends each input to a few expert sub-networks, or gradient surgery, which edits conflicting gradient directions between tasks; name them, then pick one for your constraint.

In this illustrative case adapters keep storage near one backbone while full copies grow with every domain.CC BY 4.0 — source: How would you design a multi-task system that handles varying domains?

Text description of the diagram

Full copies for 50 domains total 5,500 million parameters in this illustrative case.
A backbone plus adapters totals 308 million parameters for 50 domains.

Knowing it is half the job. Saying it is the other half.

Interview Clozo is an AI interview copilot for real interview rounds: it hears the question and puts a strong answer on your screen, and you say it in your own words. See the features or how it works.