Domain-flavoured implementation tasks in ML coding interviews, step by step

In this article

1. Before you start
2. Why interviewers ask this
3. Domain-flavoured implementation tasks from first principles
4. Worked example
5. Trade-offs and edge cases
6. How to answer it in an interview
7. Common mistakes
8. Follow-up questions
9. Sources

Before you start

This page is one lesson in the PyTorch and machine learning (ML) coding round guide. You do not need any robotics or chemistry background; each domain idea is explained from zero below.

Why interviewers ask this

A wrong answer to a domain task usually runs without an error, so the interviewer learns more from your choices than from your syntax. Candidates report prompts of this kind for autonomous-driving and drug-discovery roles: group LiDAR points into objects, convert a point cloud to a voxel grid, lift camera pixels with depth into a shared 3D space, test whether a shape lies in a sensor's view, and compute the distance between two molecules. These prompts are reported by candidates, not confirmed by any company.

A weak answer reaches for a library call and cannot say what its parameters mean. A strong answer states the input and output first, names the one trap of the task, picks a data structure that scales, and checks the result on a tiny example. A Staff-level answer adds cost: what happens with many points, and what changes when the data arrives as a stream. Explaining your reasoning aloud as you code, as in the 45-minute notebook round, is part of the signal.

Domain-flavoured implementation tasks from first principles

Voxel grid

A voxel grid is a regular 3D grid of equal cubes, called voxels, laid over a point cloud so that each point falls into exactly one cube. Only the occupied cubes are stored, and each can be reduced to one representative point, which makes the cloud smaller and evenly spaced.

DBSCAN

DBSCAN, short for density-based spatial clustering of applications with noise, is a clustering method that grows a cluster outward from any point with enough neighbours inside a set distance. It needs that distance and a minimum point count, not a number of clusters, and it labels isolated points as noise.

RMSD

RMSD, short for root mean square deviation, is one number for how far apart two paired sets of points are. Square each point's distance to its partner, average over the points, then take the square root. It is only meaningful when both sets are already in the same frame.

In plain words

Imagine dropping thousands of pins on a paper map. The grid squares of the map are the voxels. You keep one flag per square that holds pins. Clustering means finding piles of pins that touch each other. A lonely pin is noise. Comparing two molecules means laying one tracing over the other and measuring the gaps. The analogy breaks in height. A map is flat, while a voxel grid also stacks upward. The road is a floor of pins that touches every pile.

In one precise sentence: these tasks turn raw sensor numbers into a small structure, such as voxels, labels, a 3D position, a yes or no, or a distance. Each step has one input, one change and one output.

Step 1: Bucket points into voxels

A voxel downsample sorts every point into a cube and keeps one point per occupied cube. Open3D describes it as bucketing points into voxels, then replacing each occupied voxel by the average of the points inside it. The tutorial calls it a common pre-processing step.

Input: an N-by-3 array and a voxel size. Change: each point gets an integer cube index, floor of its coordinate divided by the voxel size, per axis. Output: one averaged point per occupied cube.

import numpy as np

def voxel_downsample(points, voxel_size):
    idx = np.floor(points / voxel_size).astype(np.int64)
    _, inverse, counts = np.unique(idx, axis=0, return_inverse=True, return_counts=True)
    inverse = inverse.reshape(-1)
    sums = np.zeros((counts.size, 3))
    np.add.at(sums, inverse, points)   # add each point to its voxel's sum
    return sums / counts[:, None]      # sum divided by count = centroid

Store only occupied cubes. A dense 3D array of empty cells is the usual wrong turn, and the worked example shows its size. A dictionary keyed by the index triple does the same job in expected linear time; np.unique sorts, so it costs more.

Step 2: Remove the ground plane before you cluster

Ground removal finds the road surface and deletes its points, so objects stand alone. Without it, a dense scan has ground points touching each other and touching the bottom of every object, so a density-based cluster grows across the whole floor and merges cars, signs and people into one blob. That is the mistake in the hook, and it is reasoning from how density clustering works, not a measured rate.

Random sample consensus (RANSAC) is a fitting method that repeatedly picks three random points, builds the plane through them and counts how many points lie close to it, then keeps the plane with the most such points. Open3D's plane segmentation takes a distance_threshold, the largest distance from the plane at which a point still counts as an inlier, and returns the plane and the inlier indices.

Input: points. Change: a point whose distance to the plane is under the threshold is ground. Output: the points that remain.

Step 3: Cluster the remaining points by density

Clustering gives every remaining point a label, so each label is one object. Open3D's DBSCAN takes eps, the neighbour distance, and min_points, the smallest count that forms a cluster; its output labels mark noise as minus one. The scikit-learn guide lists DBSCAN for non-flat geometry, uneven cluster sizes and outlier removal, which describes a street scene: objects of every size and stray points.

Input: points, eps, min_points. Change: a point with enough neighbours inside eps is a core point, and clusters grow from core points through their neighbours. Output: one label per point.

A tree structure that splits space along one axis at a time answers neighbour queries without comparing every pair. Voxelising first helps in the same way. The worked example counts the savings. You now have the LiDAR half of the family: voxelise, remove the ground, then cluster.

Step 4: Project pixels and depth into one 3D frame

Back-projection lifts a pixel with a depth value into a 3D point. The intrinsic matrix is the camera's own constants: focal lengths $f_x$ and $f_y$, and the pixel centre $c_x$ and $c_y$. The pinhole model, described in the OpenCV camera calibration documentation, gives $u=f_{x}X/Z+c_x$ and $v=f_{y}Y/Z+c_y$, so inverting it with a known depth Z gives:

Input: a pixel (u, v), its depth Z and the intrinsics. Change: the two formulas above, in the camera's own frame. Output: a 3D point in camera coordinates.

The extrinsic transform is the rotation R and translation t that carry camera coordinates into a shared world frame: $p_{world}=R{p}_{camera}+t$. A unified space needs one such transform per camera. Skipping it is the trap: two cameras then disagree about where the origin is.

Step 5: Test whether a point is inside the field of view

A field-of-view test asks whether a 3D point lies in front of a sensor and inside its viewing pyramid. Work in the sensor's own frame, where Z points forward. The point is visible when Z is positive, the ratio X over Z is no larger in size than the tangent of half the horizontal angle, and Y over Z is no larger than the tangent of half the vertical angle.

Input: a point in sensor coordinates and the two angles. Change: two ratio comparisons. Output: yes or no.

In the field of view is not the same as visible. Add a range limit, because a sensor has a far limit. For occlusion, compare the shape's depth with the nearest return along the same ray, for example in a depth image or z-buffer of the frame.

For a stream, run the test per point or per detected shape as it arrives, and keep a small buffer of recent points rather than the whole history. Checking one point does not prove that a stop sign, an octagon with a pole, is visible: for a shape you test its corners or its centre, and say which rule you chose, such as "centre inside and most corners inside".

Step 6: Compare two molecules with RMSD after alignment

Comparing atomic coordinates needs alignment first. Raw RMSD between two coordinate sets mixes the shape difference with where each molecule happens to sit and how it is turned, so two identical molecules in different poses score badly.

The Kabsch algorithm is a method that finds the rotation minimising RMSD between two paired point sets; Kabsch (1976) published it. The Kabsch algorithm article describes three steps: translate both sets so their centroids coincide, compute a covariance matrix, then compute the optimal rotation, usually from a singular value decomposition (SVD).

Input: two N-by-3 arrays with atoms in the same order. Change: centre both, build the covariance matrix, take the SVD, form the rotation, apply it. Output: the RMSD of the aligned sets. Atom order matters: the pairing is given, not searched for.

Check for a reflection. If the determinant of V Uᵀ is −1, the SVD has produced a reflection, not a rotation; flip the sign of the last column (multiply by diag(1, 1, d) with d = sign(det(V Uᵀ))) before applying it. The Kabsch algorithm article describes this correction.

Only the first three steps form a pipeline. Back-projection, the field-of-view test and RMSD alignment are separate tasks that do not feed one another.

1 · Voxelise2 · Remove ground3 · Cluster4 · Back-project5 · Field of view6 · AlignReplay

Step-by-step description

1. Bucket points into cubes, one point per cube.
2. Fit the road plane and drop its points.
3. Density clustering labels each object.
4. Standalone: pixel plus depth becomes 3D.
5. Standalone: compare ratios with angles.
6. Standalone: centre and rotate before RMSD.

• Voxelise, remove the ground, then cluster.
• Ground points connect objects, so they are removed before clustering.
• Back-projection, the field-of-view test and RMSD alignment are separate tasks that do not feed one another.

Worked example

The point cloud below is illustrative: ten points, and a voxel size of 0.5. Points 1 to 4 sit near (0.2, 0.2, 0.1), points 5 to 8 near (2.1, 2.1, 0.25), and two stray points lie at (5, 5, 1) and (-3, 1, 0.5).¹

Voxel indices are the floor of each coordinate divided by 0.5. The first group lands in voxel (0, 0, 0) and the second in (4, 4, 0); the strays land in (10, 10, 2) and (-6, 2, 1). Ten points become 4 occupied voxels, with centroids (0.175, 0.175, 0.125) and (2.1, 2.075, 0.25) for the two groups.¹ Density clustering with eps 0.3 and min_points 3 (illustrative) labels the points 0, 0, 0, 0, 1, 1, 1, 1, -1, -1: two objects and two noise points. The widest gap inside a cluster is 0.2449, under eps.¹

For the ground, take the plane z = 0 with a threshold of 0.05 (illustrative). A road return at height 0.02 is 0.02 from the plane, so it is removed; the first object point is 0.1 away, so it stays, and all ten toy points are above the threshold.¹ The number of random draws is $k=\frac{\log (1-p)}{\log (1-w^3)}$, where w is the share of points on the plane and p the wanted confidence. With w = 0.5 and p = 0.99 this gives 34.5, rounded up to 35 draws, so that there is a 99 percent chance that at least one draw picks three ground points.¹

For the camera, take $f_x=f_y=500$, centre (320, 240), pixel (420, 340) and depth 2 (all illustrative). Then X = 100 × 2 / 500 = 0.4 and Y = 0.4, so the camera-frame point is (0.4, 0.4, 2.0). With R as the identity and t = (1.0, 0, 0.5), the world point is (1.4, 0.4, 2.5).¹

For visibility, take a camera with a 90-degree horizontal and a 60-degree vertical view. The limits are tan 45° = 1.0 and tan 30° = 0.5774. The projected point has both ratios equal to 0.2, so it is inside the view; a point at (3, 0, 2) has $X/Z=1.5$, which is outside.¹

For RMSD, take atoms A at (0,0,0), (1,0,0), (0,1,0) and B at (1,0,0), (1,1,0), (0,0,0): the same triangle turned 90 degrees and shifted. Every atom is 1 apart from its partner, so raw RMSD is 1.0. Centring alone gives 0.9428, and Kabsch alignment gives 0.0, which is correct because the two are the same shape.¹

Text description of the diagram

• Brute-force pairs for 100,000 points are 4,999,950,000.
• Brute-force pairs for 20,000 voxels are 199,990,000.

A dense grid of 100,000,000 cells at one byte each is 100 megabytes before any point is stored; if only 50,000 cells are occupied (illustrative), that is 0.05 percent of the grid.¹ Sparse storage keeps just the occupied cells.

Trade-offs and edge cases

The voxel size trades detail for speed. A large voxel merges nearby objects into one point; a small one keeps noise. Two points a hair apart can also fall into different voxels when they straddle a boundary, so voxel-averaged clustering can split an object that raw clustering would keep whole. Choose the size from the smallest object you must keep, and say so.

The eps value trades merging for splitting. A small one breaks a large object into pieces; a large one joins neighbouring objects. LiDAR returns also thin out with distance, so one eps that suits near objects can split far ones; scale eps with range or cluster in range bands. A centroid method that needs the number of clusters is a poor fit when that number is unknown; scikit-learn's overview lists the DBSCAN parameter as a neighbourhood size.

For streams, process each frame on arrival, keep a hash map of voxel sums and counts, and expire old voxels. State what you drop. Reduce data early and use bulk array operations rather than Python loops, the habit behind multi-head attention without nn. Sensors also return points behind the camera, with Z at or below zero, which the visibility test must reject first. The same discipline of stating shapes and edge cases appears in numerically stable softmax and building generative pre-trained transformer (GPT) from scratch. Each task has one classic trap, and the table below names it; name the trap before you code.

Text description of the diagram

• Each task has one classic trap.
• Name the trap before you code.

Reported practice differs on whether a library call is acceptable in such rounds; ask before you import a clustering routine.

How to answer it in an interview

The spoken structure: say the pipeline order, name the trap, give the data structure, then the knob you would tune.

Common mistakes

• "I will cluster the raw cloud." (wrong) Ground points connect objects. (right) "I remove the ground plane first, then cluster."
• "I allocate a 3D array for the voxel grid." (wrong) Most cells are empty. (right) "I store only occupied voxels in a hash map."
• "I pick the number of clusters first." (wrong) Density clustering takes a distance and a minimum count. (right) "I tune the neighbour distance and the minimum count, and read noise as label minus one."
• "RMSD is a one-line formula." (wrong) Without alignment it measures pose too. (right) "I centre both sets and rotate with the Kabsch algorithm first."
• "The SVD gives me the rotation." (wrong) If the determinant is −1 it is a reflection. (right) "I check the determinant and flip the last column when it is negative."

Follow-up questions

How would you cluster LiDAR points into distinct objects?

Voxelise, remove the ground plane, then run density-based clustering with a neighbour distance and a minimum point count. Points that join no cluster get the noise label.

How do you check whether a shape is visible in a field of view from a sensor stream?

Transform each point to the sensor frame, require positive depth, and compare X over Z and Y over Z with the tangents of the half angles. For a shape, apply a stated rule to its corners or centre. Inside the frustum is not yet visible: add a range limit, and test occlusion by comparing the shape's depth with the nearest return along the same ray.

How do you convert a 3D point cloud into a voxel grid?

Divide each coordinate by the voxel size, take the floor to get an integer index, and group points by index in a hash map. Keep one centroid per occupied voxel instead of a dense array.

How do you project 2D camera coordinates and depth into one 3D space?

Back-project with the intrinsics to get camera-frame points, then apply each camera's rotation and translation to reach one world frame. Both steps are needed before points from two cameras can be compared.

What if the atom order is unknown?

Then the pairing must be found before Kabsch alignment, for example by matching atom types or by a matching algorithm; Kabsch assumes atom i in one set pairs with atom i in the other.

Why is RMSD only meaningful after alignment?

Raw RMSD adds the pose difference to the shape difference, so identical molecules in different poses score a nonzero value. Aligning with the Kabsch algorithm removes the pose.

Sources

1. Point Cloud: voxel downsampling, DBSCAN clustering, plane segmentation (Open3D tutorial, current release) — Open3D documentation (current release docs, read 2026-10-26; page carries no date), 26 Oct 2026
2. Camera Calibration and 3D Reconstruction (pinhole camera model) — OpenCV documentation (4.x, read 2026-10-26; page carries no date), 26 Oct 2026
3. A solution for the best rotation to relate two sets of vectors — Acta Crystallographica A32, 922-923 (Kabsch, 1976), 1 Sep 1976
4. 2.3 Clustering (overview of methods) — scikit-learn user guide (accessed 2026-10-26; page undated), 26 Oct 2026
5. Kabsch algorithm — Wikipedia (accessed 2026-10-26; pointer to Kabsch 1976), 26 Oct 2026