Seminar: Neural Networks and GANs

From Classification to Image Generation

Agenda

1. Introduction to Neural Networks
   • Multilayer Perceptron (MLP)
   • Convolutional Neural Networks (CNN)
2. Generative Adversarial Networks (GANs)
   • Basic GAN
   • DCGAN (Deep Convolutional GAN)
3. Comparison and Applications
4. Hands-on Exercises

1. Introduction to Neural Networks

Why Are They Important?

• Classification: Identify patterns in data (e.g., handwritten digits).
• Generation: Create new data (e.g., images, text, music).
• Applications:
  • Image recognition (CNN).
  • Voice synthesis (GANs, VAEs).
  • Machine translation (Transformers).

1.1 Multilayer Perceptron (MLP)

Architecture

graph TD
    A[Input: 784 pixels] --> B[Hidden Layer 1: 128 neurons]
    B --> C[Hidden Layer 2: 64 neurons]
    C --> D[Output: 10 classes]

• Fully Connected: Each neuron is connected to all neurons in the previous layer.
• Activation Functions: ReLU (hidden layers), LogSoftmax (output).
• Limitation: Does not capture spatial relationships in images.

1.1 MLP: Results on MNIST

• Conclusion: MLP works, but CNN is more efficient for images.

1.2 Convolutional Neural Networks (CNN)

Architecture

graph TD
    A[Input: 1x28x28] --> B[Conv2d: 32 filters 3x3]
    B --> C[MaxPool: 2x2]
    C --> D[Conv2d: 64 filters 3x3]
    D --> E[MaxPool: 2x2]
    E --> F[Flatten]
    F --> G[FC: 128 neurons]
    G --> H[Output: 10 classes]

• Advantages:
  • Captures local patterns (edges, textures).
  • Fewer parameters than MLP (weight sharing).

1.2 CNN: Why Does It Work Better?

• Convolution:
  • Filters detect features (e.g., vertical edges).
  • Example: A filter may activate for a vertical edge.
• Pooling:
  • Reduces dimensionality (e.g., MaxPool2d with kernel_size=2).
  • Invariance to small translations.
• Result: 98-99% accuracy on MNIST with fewer parameters.

2. Generative Adversarial Networks (GANs)

What Are They?

• Generator (G): Creates fake images from noise.
• Discriminator (D): Distinguishes between real and fake images.
• Training: Zero-sum game (G tries to fool D, D tries not to be fooled).

2.1 Basic GAN (Fully Connected)

Architecture

graph LR
    A[Noise: 100 dim] --> B[Generator: MLP]
    B --> C[Fake Image: 28x28]
    D[Real Image: 28x28] --> E[Discriminator: MLP]
    C --> E
    E --> F[Real or Fake?]

• Generator:
  • Linear layers + LeakyReLU + Tanh (output in [-1, 1]).
• Discriminator:
  • Linear layers + LeakyReLU + Sigmoid (output in [0, 1]).

2.1 Basic GAN: Problems

• Blurry images: Does not capture spatial relationships.
• Instability:
  • Mode collapse (G always generates the same image).
  • Vanishing gradients.
• Solution: Use DCGAN (convolutional layers).

2.2 DCGAN (Deep Convolutional GAN)

Architecture

graph LR
    A[Noise: 100 dim] --> B[Reshape: 100x1x1]
    B --> C[ConvTranspose2d: 256x7x7]
    C --> D[ConvTranspose2d: 128x14x14]
    D --> E[ConvTranspose2d: 64x28x28]
    E --> F[ConvTranspose2d: 1x28x28]
    F --> G[Fake Image: 28x28]
    H[Real Image: 28x28] --> I[Conv2d: 64x14x14]
    I --> J[Conv2d: 128x7x7]
    J --> K[Conv2d: 256x4x4]
    K --> L[Conv2d: 1]
    L --> M[Real or Fake?]

• Generator:
  • ConvTranspose2d to upscale noise into an image.
  • BatchNorm + ReLU (except last layer: Tanh).
• Discriminator:
  • Conv2d + LeakyReLU + BatchNorm (except first layer).

2.2 DCGAN: Keys to Success

1. Transposed Convolutions:
   • Upscale noise into an image (e.g., 100x1x1 → 256x7x7).
2. Batch Normalization:
   • Stabilizes training.
3. Activation Functions:
   • Generator: ReLU (hidden layers) + Tanh (output).
   • Discriminator: LeakyReLU (all layers).
4. Optimizer:
   • Adam with lr=0.0002 and beta1=0.5.

2.2 DCGAN: Comparison with Basic GAN

2.2 DCGAN: Example of Generated Images

• Epoch 1: Random noise.
• Epoch 5: Recognizable digits.
• Epoch 50: Sharp images.

3. Comparison and Applications

Model Summary

3. Applications of GANs

• Image Generation:
  • StyleGAN (realistic faces).
  • DeepFake (videos).
• Super-Resolution:
  • SRGAN (upscale image resolution).
• Data Synthesis:
  • Generate training data (e.g., medical images).

4. Hands-on Exercises

For the Audience

1. MLP/CNN:
   • Modify the number of layers/neurons. How does it affect accuracy?
2. GAN/DCGAN:
   • Increase latent_dim (e.g., from 100 to 200). Do images improve?
   • Train with more epochs (e.g., 50). Are images sharper?
3. Challenge:
   • Implement a Conditional GAN (cGAN) to generate specific digits.

4. Resources and References

• Papers:
  • GAN (Goodfellow et al., 2014)
  • DCGAN (Radford et al., 2015)
• Tutorials:
  • PyTorch DCGAN
  • TensorFlow GAN
• Tools:
  • Marp (for slides).
  • Google Colab (to run code).

Q&A

Discussion

• Why do CNNs work better than MLPs for images?
• How would you evaluate the quality of a GAN?
• What practical applications do you see for GANs?

Thank You!

Contact

• Miguel Atencia
• Email: miguel.atencia@example.com
• GitHub: @miguelatencia