Deep Learning Fundamentals

What are the different types of neural networks?

By Interview Clozo · Published 27 October 2026 · Last verified 26 October 2026 · 12 min read

The different types of neural networks are the multilayer perceptron (MLP), the convolutional neural network (CNN), the recurrent neural network (RNN), the transformer and the generative family; each type wires in an assumption about the shape of its data. Reciting the list is a weak answer. A natural follow-up asks why you would not use a plain dense network on an image, and one weight count in the second step answers it.

In this article

Before you start
Why interviewers ask this
Neural network types from first principles
The Staff-level answer
The rubric
Three variants of this question
Common mistakes
Follow-up questions
Sources

Before you start

This page is one step of the deep learning learning path.

Why interviewers ask this

A question this broad is a cheap way to see whether you organize what you know. The answer is reported in interviews at AI labs and large tech companies as an opener to a longer conversation, and the follow-ups decide how it goes.

A weak answer lists names: "CNN, RNN, transformer, generative adversarial network (GAN)." A strong answer attaches each name to the data it suits and the assumption that makes it cheap: images have local patterns, text has order, and so on. A Staff-level answer adds what each type costs and where the textbook pairing is wrong, for example that transformers also process images and that convolutions also process audio. The page below gives you that structure, then the sentence to say.

Neural network types from first principles

Neural network architecture: A neural network architecture is the wiring plan of a network: which layers it has, which units connect to which, and which weights are shared. Two networks with different architectures can learn from the same data but make different assumptions about it.
Inductive bias: An inductive bias is an assumption built into a model's structure that favors some solutions over others before any training happens. A convolutional layer, for instance, assumes that a useful pattern in one part of an image is useful elsewhere.
Parameter sharing: Parameter sharing is the reuse of the same weights at many positions, such as every location in an image or every step in a sequence. It cuts the number of weights and carries the assumption that the same pattern can appear anywhere.

In plain words

Think of tools in a workshop. A hammer, a saw and a drill all work on wood, yet each one assumes something about the job. Neural network types are like that: one family of math, with different wiring for different kinds of data. The analogy breaks in one place. A workshop tool is fixed, but a network's weights are learned from examples, and the wiring only decides what it is able to learn cheaply.

In one precise sentence, each type of neural network is an architecture whose inductive bias matches a data shape: a fixed vector, a grid, a sequence, a set of positions that all relate to each other, or a distribution to sample from. Other shapes exist too; graph networks, for example, take nodes and edges as input. The five steps below cover the types an interviewer expects.

Step 1: Multilayer perceptrons connect every input to every unit

A multilayer perceptron is a network of stacked dense layers in which every unit in one layer connects to every unit in the next, and information flows one way from input to output. It assumes nothing about the structure of its input, so it is the default for tabular rows and other fixed-size vectors.

Input: one vector. Output: one vector. Each layer computes weights times inputs plus a bias, then applies an activation function. The weight count of one layer is the number of inputs times the number of outputs, plus one bias per output. That count is exact, which makes the MLP the easy place to start counting.

The cost of "no assumptions" shows up on images, which is the next step. You now know the baseline every other type is measured against.

Step 2: Convolutional networks reuse one small filter across the image

A convolutional neural network is a network whose layers slide small filters across a grid of values, such as image pixels, so the same pattern detector is applied at every position. It assumes that patterns are local and can appear anywhere.

Why not a dense layer? The Stanford CS231n notes show that one fully connected neuron looking at a 32 by 32 image with three color channels has 3,072 weights, and a 200 by 200 image gives 120,000 per neuron. In our count, a dense layer of 100 units on the 3,072 inputs has 307,300 parameters. A convolutional layer of 12 filters, each 5 by 5 by 3, has only 912, and with a padding of 2 it keeps the 32 by 32 output size.¹ The same output volume of 32 by 32 by 12 from a dense layer would need 37,761,024 parameters.¹

A convolutional layer produces the same output volume as a dense layer with about 41,000 times fewer parameters, because it shares weights.
Layer on a 32 by 32 by 3 image	Parameters
Dense layer, 100 units	307,300
Dense layer producing a 32 by 32 by 12 volume	37,761,024
Convolutional layer, 12 filters of 5 by 5 by 3	912

The sizes follow the formula in the CS231n notes: output size is (W − F + 2P)/S + 1, where W is the input width, F the filter size, P the padding and S the stride. With W = 32, F = 5, P = 2 and S = 1, the output is 32.¹ Convolutions are not only for images: they also run on one-dimensional signals such as audio. See convolutional networks for the full layer-by-layer picture.

Step 3: Recurrent networks and LSTMs carry a memory along a sequence

A recurrent neural network is a network that reads a sequence one step at a time and passes a hidden state from each step to the next, using the same weights at every step. It assumes that order matters and that the rule for updating the memory is the same at every position.

Input: a sequence. Output: a hidden state per step. Because the weights are shared across steps, the parameter count does not grow with sequence length. With an input size of 64 and a hidden size of 128 (illustrative), a plain recurrent layer has 24,704 parameters, whether the sequence has ten steps or ten thousand.¹

The weakness is training. The error signal must travel back through every step, and it can shrink or blow up on the way; see exploding gradients. A long short-term memory (LSTM) cell adds gates that control what the memory keeps and forgets, which eases the problem. It mitigates the problem rather than removing it. With four weight blocks, an LSTM layer of the same sizes has 98,816 parameters.¹ The LSTM page explains the cell. Steps also run one after another, which limits parallel work.

Step 4: Transformers let every position read every other position

A transformer is a network built around attention, a mechanism in which each position in a sequence computes a weighted mix of every other position. Vaswani et al. (2017) proposed it as an architecture "based solely on attention mechanisms", dispensing with recurrence and convolutions, and reported it as more parallelizable and faster to train.

Input: a sequence of vectors. Output: a sequence of vectors of the same length. Because no step waits for the previous one, positions are processed together, which suits modern accelerators. The price is that self-attention scores every pair of positions, so n tokens give n squared scores: 16,384 at 128 tokens, 1,048,576 at 1,024 and 4,194,304 at 2,048. At 2,048 tokens there are 4,194,304 scores, four times the count at 1,024 tokens.¹ Residual connections and normalization make deep stacks trainable; see residual connections.

Attention scores every pair of positions, so doubling the sequence length quadruples the score count.CC BY 4.0 — source: What are the different types of neural networks?

Text description of the diagram

Self-attention scores every pair of positions, so n tokens give n squared scores.
At 2,048 tokens there are 4,194,304 scores, four times the count at 1,024 tokens.

Step 5: Generative networks learn to produce data

A generative network is a network trained to produce new examples that look like its training data, rather than to label an input. It does so by learning the probability distribution of the data, or a way to sample from it.

Three families come up. An autoencoder squeezes an input through a narrow bottleneck and rebuilds it; a variational autoencoder makes the bottleneck a distribution you can sample from. A generative adversarial network (GAN) trains a generator against a discriminator, a classifier that tries to tell real from generated; the GAN as a whole is generative, and only its discriminator is a classifier. A diffusion model learns to reverse a gradual noising process. These families are defined by the training goal, not by a layer type, so they are built from the networks above. Multimodal models combine several of them.

Each type of neural network fits one shape of data, and each pays for it with one weak spot.CC BY 4.0 — source: What are the different types of neural networks?

Step-by-step description

Every input feeds every unit.
One small filter is reused across the grid.
The same weights run at each step.
Every position reads every other position.
The network learns to produce data.

A multilayer perceptron connects every input to every unit.
A convolutional network slides one small filter across a grid.
A transformer lets every position read every other position.

Knowing it is half the job. Saying it is the other half.

Interview Clozo is an AI interview copilot for real interview rounds: it hears the question and puts a strong answer on your screen, and you say it in your own words. See the features or how it works.