Artificial Neural Networks, Machine Learning, Deep Thinking Training Course

DAY 1 - ARTIFICIAL NEURAL NETWORKS

Introduction and ANN Structure

• Comparison between biological and artificial neurons.
• Theoretical model of an ANN.
• Activation functions employed in ANNs.
• Common categories of network architectures.

Mathematical Foundations and Learning Mechanisms

• Review of vector and matrix algebra.
• Understanding state-space concepts.
• Principles of optimization.
• Error-correction learning approaches.
• Memory-based learning methods.
• Hebbian learning principles.
• Competitive learning frameworks.

Single Layer Perceptrons

• Structure and learning dynamics of perceptrons.
• Introduction to pattern classifiers and Bayes' classifiers.
• Utilizing perceptrons as pattern classifiers.
• Analysis of perceptron convergence.
• Limitations inherent to perceptrons.

Feedforward ANNs

• Structure of Multi-layer feedforward networks.
• Overview of the Back propagation algorithm.
• Training and convergence via Back propagation.
• Functional approximation using back propagation.
• Practical and design considerations in back propagation learning.

Radial Basis Function Networks

• Pattern separability and interpolation techniques.
• Regularization Theory.
• Applying Regularization to RBF networks.
• Design and training processes for RBF networks.
• Approximation characteristics of RBF networks.

Competitive Learning and Self-organizing ANNs

• General clustering procedures.
• Learning Vector Quantization (LVQ).
• Competitive learning algorithms and architectures.
• Self-organizing feature maps.
• Key properties of feature maps.

Fuzzy Neural Networks

• Neuro-fuzzy systems.
• Background on fuzzy sets and logic.
• Design of fuzzy systems.
• Design of fuzzy ANNs.

Applications

• Discussion on various Neural Network application examples, highlighting their advantages and challenges.

DAY 2 - MACHINE LEARNING

• The PAC Learning Framework
  • Guarantees for finite hypothesis sets – consistent case
  • Guarantees for finite hypothesis sets – inconsistent case
  • Generalities
    • Deterministic vs. Stochastic scenarios
    • Bayes error noise
    • Estimation and approximation errors
    • Model selection
• Rademacher Complexity and VC-Dimension
• Bias-Variance tradeoff
• Regularisation
• Over-fitting
• Validation
• Support Vector Machines
• Kriging (Gaussian Process regression)
• PCA and Kernel PCA
• Self-Organization Maps (SOM)
• Kernel induced vector space
  • Mercer Kernels and Kernel-induced similarity metrics
• Reinforcement Learning

DAY 3 - DEEP LEARNING

This module integrates concepts covered in Day 1 and Day 2

• Logistic and Softmax Regression
• Sparse Autoencoders
• Vectorization, PCA, and Whitening
• Self-Taught Learning
• Deep Networks
• Linear Decoders
• Convolution and Pooling
• Sparse Coding
• Independent Component Analysis
• Canonical Correlation Analysis
• Demos and Applications