How do you implement 2D convolution from scratch?

In this article

1. Before you start
2. Why interviewers ask this
3. 2D convolution from first principles
4. Worked example
5. Trade-offs and edge cases
6. How to answer it in an interview
7. Common mistakes
8. Follow-up questions
9. Sources

Before you start

This page is part of the machine learning (ML) coding from scratch guide, which collects the functions interviewers ask you to write without a library.

Why interviewers ask this

Writing a convolution from scratch is a standard machine learning coding question because it tests three things at once: array indexing, the output-size arithmetic, and whether you know what a convolutional layer really computes. It appears as "implement 2D convolution on a 4 by 4 input with a 3 by 3 filter and stride 1", then "now add padding and stride", then "how would you speed it up".

The coding interviews hub places this question among the others. A weak answer writes four nested loops and gets the output size wrong by one. A strong answer states the formula first, writes the loop, checks it against a library on a small case, and can explain the flip and the vectorised version.

2D convolution from first principles

In plain words

Kernel

A kernel is a small grid of weights, often three by three, that is slid over an input and combined with the numbers beneath it at each position; in a neural network its entries are learned.

Padding

Padding is a border of extra values, usually zeros, added around the input so the kernel can be centred on edge pixels and the output keeps its size.

Cross-correlation

Cross-correlation is the sliding multiply-and-sum operation that deep learning libraries call convolution; true mathematical convolution is the same operation with the kernel rotated by 180 degrees.

Picture a magnifying glass with a stencil on it. Move it over a photo one step at a time. At each stop, read the nine numbers under the stencil. Weight them by the stencil's values and add them up. Write that one number on a new sheet. The new sheet is the output; a feature map is exactly that sheet. The analogy breaks at the edges. A real stencil can hang off the photo, but a kernel cannot, which is why padding exists.

In one precise sentence, each output cell is the kernel times one window of the input, multiplied element by element and summed. The window for cell $(i,j)$ starts at row $i·s$ and column $j·s$.

Step 1: Work out the output size first

The output has $\text{floor}((n+2p-k)/s)+1$ rows and columns for an input of size $n$, kernel size $k$, padding $p$ and stride $s$. The PyTorch Conv2d reference gives the same relation with a dilation term that is 1 by default, and the Stanford CS231n notes write it as $(W-F+2P)/S+1$ and point out the result must be a whole number for the filter to fit. Dumoulin and Visin (2016) derive it case by case.

For a 4 by 4 input with a 3 by 3 kernel, padding 0 and stride 1, the output is $(4-3+0)/1+1=2$ on each side, so a 2 by 2 output¹. This number sets the loop bounds, and it is the number to say first in an interview.

Step 2: Pad the input

Padding adds a border of zeros of width $p$ on all four sides, so the padded input has $n+2p$ rows and columns. The reason is the edges: without padding every output shrinks by $k-1$ and edge pixels are used in fewer windows than centre pixels. In NumPy the call is np.pad(x, p), which pads with zeros by default.

Gradient descent can learn the kernel values, as the gradient descent page explains, so the code below only has to apply a kernel. In our worked example the padding is 0, so the input stays 4 by 4. With padding 1 and a 3 by 3 kernel at stride 1 the output is 4 by 4 again, the size of the input¹. Setting padding to keep the size is what PyTorch's padding='same' does, and its reference notes that mode supports no stride other than 1.

The input is the image and the padding width. The output is a larger array with a border of zeros.

Step 3: Multiply and sum, without flipping the kernel

Each output cell is the sum of an element-wise product between the kernel and one window of the padded input, with no flip of the kernel. This is cross-correlation, and the PyTorch reference describes Conv2d as computing "the valid 2D cross-correlation operator". Textbook convolution would rotate the kernel by 180 degrees first.

The first window of the example input is the top-left 3 by 3 block. Multiplying it element by element with the kernel gives the nine products 2, 0, 3, 0, 1, 4, 3, 0 and 2, so the first window multiplied by the kernel sums to 15¹. With the kernel rotated by 180 degrees, the four output cells become 11, 12, 10 and 11 instead of 15, 16, 6 and 15, so the whole output changes¹. For learned kernels the flip does not matter, because training would just learn the flipped weights, but for a fixed kernel or a test against a library it does.

1 · Output size2 · Pad3 · Multiply and sum4 · SlideReplay

Step-by-step description

1. The formula sets the loop bounds.
2. Padding zero leaves the input as it is.
3. One window gives one output cell.
4. Repeat at every position.

• The output of a 4 by 4 input and a 3 by 3 kernel has size 2 by 2.
• The first window multiplied by the kernel sums to 15.
• The full output is 15, 16, 6 and 15.

Step 4: Slide by the stride and fill the output

The kernel moves by the stride along each row, then down to the next row, and the loop writes one output cell per stop. With stride 1 the example visits four windows, so the full output is 15, 16, 6 and 15¹. With padding 1 and stride 2 the same input gives a 2 by 2 output of 7, 10, 10 and 15¹.

def conv2d_loops(x, k, stride=1, padding=0):
    xp = np.pad(x, padding)
    kh, kw = k.shape
    oh = (x.shape[0] + 2 * padding - kh) // stride + 1
    ow = (x.shape[1] + 2 * padding - kw) // stride + 1
    out = np.zeros((oh, ow))
    for i in range(oh):
        for j in range(ow):
            patch = xp[i * stride:i * stride + kh, j * stride:j * stride + kw]
            out[i, j] = np.sum(patch * k)
    return out

The function computes the output size, pads, then for each output cell cuts the window that starts at the cell's index times the stride, multiplies it by the kernel and sums. Integer division with // is the floor in the formula. Read it once more with the example in mind, and check that the slice bounds never run past the padded input.

Worked example

The input is the 4 by 4 array with rows 1 2 3 0, 0 1 2 3, 3 0 1 2 and 2 3 0 1, and the kernel is 2 0 1, 0 1 2 and 1 0 2, both illustrative¹. The function above, run in Python with NumPy, returns the 2 by 2 output below, and three other checks agree with it.

The result of every check came from running assets/data/conv2d.py¹. A unit test of this kind, a small case compared with a trusted library, is what you should offer in an interview.

Text description of the diagram

• Without the flip the four output cells are 15, 16, 6 and 15.
• With the kernel rotated by 180 degrees they are 11, 12, 10 and 11.

The output-size formula was also checked against PyTorch for seven cases and agreed on all of them¹.

Text description of the diagram

• A 4 input with a 3 kernel, stride 1 and padding 0 gives output 2.
• A 5 input with a 3 kernel, stride 2 and padding 1 gives output 3.

The multiply-add count is the number of output cells times the kernel area times the channels; the parameter count is the kernel area times input channels times filters, plus one bias per filter¹. The cost grows with the output area; the Stanford CS231n notes say that in VGGNet most of the memory and compute time is used in the early convolution layers.

Trade-offs and edge cases

The loop version is easy to check and slow in Python, so the usual follow-up is to vectorise it. NumPy's sliding_window_view (reference), added in version 1.20.0, returns every window as a view without copying, and an einsum multiplies each window by the kernel.

from numpy.lib.stride_tricks import sliding_window_view

def conv2d_fast(x, k, stride=1, padding=0):
    xp = np.pad(x, padding)
    windows = sliding_window_view(xp, k.shape)[::stride, ::stride]
    return np.einsum('ijkl,kl->ij', windows, k)

The windows come out with shape (rows, columns, kernel height, kernel width), the slice [::stride, ::stride] applies the stride, and the einsum sums the products over the last two axes. Its output equals the loop version on the example, with and without padding and stride¹. The NumPy reference warns the approach scales with the input size times the window size and can be much slower than specialised methods.

Libraries do not run our loop; they pick among several algorithms. One is the matrix-multiply form the CS231n notes describe, called im2col: each window is stretched into a column, each filter into a row, and the product is reshaped into the output. The price is memory, because input values are replicated in many columns, and the gain is that optimised matrix-multiply libraries do the work. For our example, im2col gives a 9 by 4 matrix, one column per window, and the same 2 by 2 output¹. Others are convolution by fast Fourier transform (FFT) and Winograd's minimal filtering, which Lavin and Gray (2015) show is fast for small 3 by 3 filters.

Edge cases to name: a stride that does not divide the span evenly drops the last partial window, because of the floor; a kernel larger than the padded input has no valid position; and multi-channel inputs sum the products over all channels for each filter, which the next code block does.

def conv2d(x, w, b=None, stride=1, padding=0):
    # x: (C_in, H, W)   w: (C_out, C_in, kh, kw)   b: (C_out,)
    xp = np.pad(x, ((0, 0), (padding, padding), (padding, padding)))
    c_out, c_in, kh, kw = w.shape
    oh = (x.shape[1] + 2 * padding - kh) // stride + 1
    ow = (x.shape[2] + 2 * padding - kw) // stride + 1
    out = np.zeros((c_out, oh, ow))
    for f in range(c_out):
        for i in range(oh):
            for j in range(ow):
                patch = xp[:, i * stride:i * stride + kh, j * stride:j * stride + kw]
                out[f, i, j] = np.sum(patch * w[f]) + (0 if b is None else b[f])
    return out

Each filter has one kernel per input channel; the window now spans every channel, and one bias is added per filter. On a 2-channel 5 by 5 input with 3 filters, padding 1 and stride 1, it returns a 3 by 5 by 5 array that equals PyTorch's result, and the layer has 57 parameters (3 filters times 2 channels times 9 weights, plus 3 biases)¹. The deep learning guide shows where the layer sits in a network, and the activation functions page covers what usually follows it.

How to answer it in an interview

The order to answer in is: output size, loop, the flip remark, a test, then the speed-up.

Common mistakes

• "The output is the same size as the input" (wrong). Without padding it shrinks by the kernel size minus one (right: use the formula, and pad by one for a three-wide kernel at stride 1).
• "I forgot the plus one" (wrong). The output size is the floor of the division plus one, so a 4 input with a 3 kernel gives 2, not 1 (right).
• "Convolution layers flip the kernel" (wrong). PyTorch documents cross-correlation, with no flip (right: say it, and mention that a learned kernel makes the difference immaterial).
• "Multiply the window by the kernel with a matrix product" (wrong). The product is element by element, then a sum (right).
• "Stride 2 halves the output exactly" (wrong). The floor and the padding decide it: 5 with kernel 3, stride 2 and padding 1 gives 3, not 2.5 (right).

Follow-up questions

Implement 2D convolution with a 4x4 input, 3x3 filter, stride = 1?

The output is 2 by 2, because $(4-3)/1+1=2$, and each cell is one window times the kernel summed. With the example input and kernel above the answer is 15, 16, 6 and 15.

Code conv2d from scratch (with padding and stride)?

Pad with np.pad, compute the output size from the formula, and index windows with i * stride and j * stride, as in the loop version above. Test it against a library on a small case.

Implement a convolutional filter from scratch in Python?

Use the multi-channel conv2d above: it takes an input of shape (channels, height, width), a weight array of shape (filters, channels, kernel height, kernel width) and a bias per filter, and loops over filters and output positions. A single fixed filter, such as an edge detector, is the one-channel conv2d_loops case.

Implement a CNN from scratch?

A convolutional neural network (CNN) is a stack of the same pieces: convolution, a rectified linear unit (ReLU) activation, pooling to shrink the grid, flatten, and a linear layer that outputs class scores. The deep learning guide covers the stack; the convolution you wrote here is its core.

Is a convolution layer really a convolution?

It is cross-correlation, because libraries do not flip the kernel; the PyTorch reference says so. For a learned kernel it makes no difference, since training would learn the flipped weights.

How do you speed up the loop version?

Use sliding windows with an einsum, or im2col with one matrix multiplication. Both agree with the loop on the example. Libraries choose among algorithms such as im2col matrix multiplication, fast Fourier transform (FFT) and Winograd; the im2col cost is extra memory for the replicated values.

Sources

1. torch.nn.Conv2d, PyTorch 2.14 documentation — PyTorch (page shows version only; read 2026-10-24), 24 Oct 2026
2. A guide to convolution arithmetic for deep learning (arXiv 1603.07285v2) — arXiv (Dumoulin and Visin), 11 Jan 2018
3. Convolutional Neural Networks for Visual Recognition: Convolutional Networks (undated page, read 2026-10-24) — Stanford CS231n, 24 Oct 2026
4. Fast Algorithms for Convolutional Neural Networks (arXiv 1509.09308v2) — arXiv (Lavin and Gray), 10 Nov 2015
5. numpy.lib.stride_tricks.sliding_window_view, NumPy reference (added in 1.20.0) — NumPy (page shows version only; read 2026-10-24), 24 Oct 2026