ANNs, CNNs and model selection
Sketch neural-network structures, explain CNN choices and compare generative-image model families for Visionary Studios.
What you need to be able to do
- outline ANN uses for classification, regression and pattern recognition;
- sketch a perceptron with input, weights, bias, activation function and output;
- sketch an MLP with input, hidden and output layers;
- describe CNN input, convolution, activation, pooling, fully connected and output layers;
- describe effects of number of layers, kernel size, stride, activation function selection and the loss function;
- explain model selection using problem, complexity, desired outcomes and data characteristics.
Rapid Recall Deck
Attempt each card first. If a term is new, flip and study it, then answer it again without looking. Mark only secure recall as “Know it”.
Perceptron and multi-layer perceptron
x1 --(w1)--\\\nx2 --(w2)--- > weighted sum + bias → activation → output\nx3 --(w3)--/
Perceptron
Explain the role of each labelled component.
Inputs supply features; weights scale their influence; bias shifts the combined value; the activation function transforms it; the result becomes the neuron output.
MLP
An MLP contains an input layer → one or more hidden layers → output layer. Each hidden neuron receives values from the previous layer, forms a weighted combination and applies an activation function. Explain how stacking these transformations can represent a more complex pattern than one simple perceptron.
One perceptron forms one relatively simple weighted decision. Hidden layers let many neurons form different intermediate combinations, and later layers combine those intermediate features again. With nonlinear activation functions, the network can therefore represent more complex nonlinear relationships before producing the output.
CNN architecture
input → convolutional layers → activation functions → pooling layers\n → fully connected layers → output layer
Explain the CNN pipeline for an image classifier
For an image classifier, explain what each stage contributes: input image → convolutional layer(s) → activation function(s) → pooling layer(s) → fully connected layer(s) → output layer. Focus on the role of each stage rather than mathematical convolution formulas.
Convolutional layers learn local spatial features, activation functions introduce nonlinear transformations, pooling compacts spatial representations, and fully connected/output layers combine learned features into a final classification or other output.
Architecture changes processing
Kernel size + stride
Describe what changes when either is increased.
Larger kernels inspect wider local regions. Larger stride moves farther between positions, reducing output spatial size and computation while potentially skipping fine detail.
Layers + activation + loss function
Why can these alter performance?
Depth changes capacity and training cost; activation choice changes nonlinear signal behaviour; the loss function defines which errors training is optimized to reduce. Poor choices can reduce generalization or optimize the wrong behaviour.
Model selection is contextual
Three deployments
Explain why a small tabular dataset, a large image dataset and a real-time edge device may favour different models.
Algorithms exploit different data structures and require different amounts of data and computation. A complex neural model may suit large images but be unnecessary for small tabular data or too expensive for an edge device. Selection should compare performance and deployment constraints against the actual objective.
Diffusion mechanism
Explain diffusion generation and its main trade-off
Visionary Studios uses a diffusion model that starts from random noise. Explain how noise injection, iterative denoising, a CNN neural denoiser and the DDPM framework relate to image generation, then explain why repeating the denoising step creates a computational-efficiency trade-off.
Random noise is refined over repeated denoising steps using a CNN trained on image patterns. DDPM formalizes gradual noise addition and removal. Repetition can produce high-quality photorealistic images but creates substantial computation, latency and resource demand.
VAE and flow-based model: broad understanding
Variational autoencoder (VAE) + latent space
Why might Visionary Studios value this component?
A VAE learns a compressed organized latent space where related images can be represented near one another, supporting flexible generation, modification and artistic exploration.
Flow-based model
What distinctive property matters?
The case study describes a reversible exact mapping between noise and images, supporting precision, reproducibility and transparency or traceability.
Combine strengths only when the trade-off makes sense
Compare model families
| Family | Strength | Trade-off |
|---|---|---|
| Diffusion | Photorealistic quality | Iterative compute |
| GAN | Sharp high-quality output | Instability / mode collapse |
| VAE | Latent-space flexibility | Must be judged against output needs |
| Flow-based | Reversible / reproducible mapping | Must be judged against practical needs |
Explain why a hybrid model can be useful but not automatically better.
A hybrid can combine complementary strengths in quality, stability, control, consistency or transparency, but also increases architecture, training and deployment complexity. Visionary Studios should compare actual outcomes rather than count model components.
Consistency and architecture
Character consistency + style adherence
Distinguish them.
Character consistency preserves recurring appearance across images; embedding-based approaches can help. Style adherence means conditional outputs reliably follow the intended style or theme specified by prompts or other conditions.
Paper 1 mini-set
1. Outline a perceptron. 2. Describe pooling. 3. Explain why model selection depends on data characteristics.
Perceptron: weighted inputs plus bias → activation → output. Pooling reduces or aggregates spatial feature information. Different data distributions, scale, noise and feature structures cause models to generalize differently.
Finish the learning cycle
Case study / exam preparation
Finish the diffusion/GAN/VAE/flow/hybrid comparison matrix using official evaluation factors.
Paper 1 — short practice
Redraw a perceptron, MLP and CNN and explain kernel size, stride, activation and loss function.
Case-study checkpoint
Keep VAE and flow-based models broad; focus deeply on diffusion, GAN and hybrid evaluation.