Deep learning and neural networks

Neurons, weights, bias and activation

An artificial neuron takes several inputs, multiplies each by a weight (how important that input is), adds them together with a bias (which shifts the result), then applies an activation function:

output = activation(x1.w1 + x2.w2 + ... + b)

Common activation functions:

• Step: 1 if the sum is above a threshold, otherwise 0.
• Sigmoid: 1 / (1 + e^-z), squashing any value into the range 0 to 1.
• ReLU (rectified linear unit): max(0, z): negatives become 0.

The activation function adds non-linearity, so networks can learn complex patterns.

Layers

• Input layer: one neuron per input feature (for example, one per pixel).
• Hidden layers: transform the data; each layer's outputs are the next layer's inputs.
• Output layer: the result, such as a probability for each class.

Deep learning uses neural networks with many hidden layers. Early layers learn simple features (edges), later layers combine them into complex ones (eyes, faces).

Forward and back propagation

• Forward propagation: inputs flow forward through the layers, each neuron computing its weighted sum and activation, until the output layer produces a prediction.
• The prediction is compared with the correct answer using a loss (error) function.
• Back propagation: the error is sent backwards through the network to work out how much each weight and bias contributed to it; each is then adjusted slightly to reduce the error (gradient descent).
• This repeats over many examples and passes (epochs) until the error is small: the network has been trained.

Benefits and drawbacks

Benefits: learns patterns from data without hand-written rules; excellent at images, speech, language and complex predictions; improves with more data.

Drawbacks: needs huge amounts of (often labelled) training data and a lot of computing power and energy (usually GPUs); a 'black box', so decisions are hard to explain; can learn bias present in its training data; can overfit (memorise training data and do badly on new data); errors can be confidently wrong.

Worked example

A neuron has inputs [1.0, 0.5, -1.0], weights [0.4, 0.6, 0.2] and bias 0.1. Find its output with ReLU and with sigmoid.

1. Weighted sum z = 1.0 x 0.4 + 0.5 x 0.6 + (-1.0) x 0.2 + 0.1 = 0.6
2. ReLU: max(0, 0.6) = 0.6
3. Sigmoid: 1 / (1 + e^-0.6) = 0.646

Answer: ReLU 0.6; sigmoid 0.646

Check yourself

1. A neuron has inputs 2 and 3, weights 0.5 and -1, and bias 1. What is the weighted sum before activation?
   Show answer

   -1. 2 x 0.5 + 3 x (-1) + 1 = 1 - 3 + 1 = -1.

2. What does ReLU output for that weighted sum of -1?
   Show answer

   0. ReLU(z) = max(0, z) = max(0, -1) = 0.

3. A step activation outputs 1 if the weighted sum is at least 0.5, otherwise 0. Inputs 1 and 1 have weights 0.3 and 0.4, bias 0. What is the output?
   Show answer

   1. Sum = 0.7, which is at least 0.5, so the output is 1.

4. What is the sigmoid of 0?
   Show answer

   0.5. 1 / (1 + e⁰) = 1 / 2 = 0.5.

5. What happens during back propagation?
   • The error is passed backwards to adjust the weights and biases to reduce it
   • Inputs flow forwards to produce an output
   • New layers are added to the network
   • The training data is deleted
   Show answer

   The error is passed backwards to adjust the weights and biases to reduce it. This is how the network learns.

6. What makes a neural network 'deep'?
   • It has a very large input layer only
   • It uses only step activation functions
   • It has many hidden layers
   • It is trained for a long time
   Show answer

   It has many hidden layers. Depth refers to the number of layers.

7. What is the role of the bias in a neuron?
   • It shifts the weighted sum so the neuron can activate even when inputs are small or zero
   • It sets the number of layers
   • It stores the training data
   • It removes negative inputs
   Show answer

   It shifts the weighted sum so the neuron can activate even when inputs are small or zero. It is a learnable offset, like the intercept of a line.

8. Which is a drawback of deep learning?
   • It cannot recognise images
   • Its decisions can be hard to explain, and it can learn bias from training data
   • It needs no training data
   • It always runs faster than simple programs
   Show answer

   Its decisions can be hard to explain, and it can learn bias from training data. Black-box behaviour and data bias are major concerns.

Practise with instant feedback Print this lesson

Sources

• Pearson Edexcel International Advanced Subsidiary and Advanced Level in Computer Science specification (2026), Issue 1
• Pearson Edexcel IAL Computer Science sample assessment materials (Unit 3 paper, resource booklet and mark scheme)

Last updated 8 November 2026. Spotted a mistake? Tell us.