Course Information

Semester: 2025-2 (114-2)
Curriculum No.: GIBMS 7015 (454 M0390)
Credit: 3
Time: Fridays 9:10 - 12:10
Venue: College of Medicine RM508 (register here for NTU Eduroam access for Future Classroom Wifi)
Instructor(s): Joshua Goh
Language: English

Objectives

Learn the basic principles of how neural network models work.
Make one's own simple neural networks.
Learn how to evaluate neural network models.

Description

This course will introduce first principles of neural networks in relation to how it implements human cognition and it taught using the approach of applied practical programming of simple neural networks. Students will read three real modeling papers and apply the neural network models in these papers to create their own neural networks, in addition to regular class assignments. You will learn by doing. Four examples of network implementation will be covered:

Basic Perceptron
Hopfield Attractor Network (Hopfield, 1982)
Error Backpropagation (Multi-Layered Perceptron; Rumelhart et al., 1986)
Self Organization via Unsupervised Learning (Von Der Malsburg, 1973)

Note that the approach in this course emphasizes ground-up basic theory about neural network principles. This course will briefly consider but will not significantly engage with current artificial intelligence tools (e.g. Large Language Models) as in many contemporary Engineering applications. However, students are free to use these intelligence tools to append their understanding or in application in assignments as is creative.

Pre-Requisites

Students in Graduate Institute of Brain and Mind Sciences (others, case-by-case).
Your own computer with coding environment (below) ready to go.
Confidence in computer programming is not a requirement but is a plus.

Coding Environments

Either of the environments below are highly encouraged. The course will primarily use Python in class and some R.

Python in Jupyter Lab
R in R Studio
Matlab (see National Taiwan University campus-wide license, scroll to the bottom of the page)

Grading

Assignments
- Homework tutorial exercises.
- Three main assignments: Program the above three neural networks and apply them to real-life problems or simulations of human cognition.
Rubric
- Based on quality of assignments
  - Model success
  - Comprehensiveness and strategicness of model evaluation.
- Homeworks afford bonus credits.

Schedule

Week	Date	Topic	Resource
1	2026/02/27	Peace Memorical Day [no class]	Do spend the time to brush up on linear algebra if not confident (see Jordan, 1986), and importantly, start reading Aggarwal, 2018, Ch. 1.
2	2026/03/06	Introduction: Biology, why model, and general approach. [video]	Biological neural networks Models and simulations 2025 videos 1, 2
3	2026/03/13	Linear algebra: Vectors, matrices, and matrix operations. [video]	Jordan, 1986 Linear Algebra Colab Jupyter Notebook Blackboard HW1 [2025 video, buggy main screen not recorded - follow in Jupyter Notebook]
4	2026/03/20	Perceptrons: Nomenclature, general neural network framework, application in logic problems. [video]	Aggarwal, 2018, Ch. 1 Perceptron Colab Jupyter Notebook Blackboard HW2 [2025 video]
5	2026/03/27	Attractor networks 1: Introduction to the principles of autoencoding and memory. [video][blackboard]	Hopfield, 1982 Blackboard [2025 video]
6	2026/04/03	Childrens Day, Tomb Sweeping Day (no class)
7	2026/04/10	Attractor networks 2: Simple autoencoder architecture and learning rule to instantiate content-addressable memory, attractor properties. [blackboard][video]	Blackboard 1 Blackboard 2 Binary Autoencoder Colab Jupyter Notebook [2025 video]
8	2026/04/17	Attractor networks 3: Evaluating and describing the autoencoder model. [video]	Assignment 1 Josh's Hopfield Network Gitlab [2025 video]
9	2026/04/24	Backpropagation 1: Introduction to the principles of multi-layered perceptrons and error-based learning. [Derivation sigmoid][video]	Rumelhart et al., 1986 Blackboard [2025 video]
10	2026/05/01	Labor Day (no class)
11	2026/05/08	Backpropagation 2: Simple multi-layered perceptron to instantiate error-based learning, non-linear input-output mappings. [video][Blackboard]	Multi-Layer-Perceptron Colab Jupyter Notebook Blackboard [2025 video]
12	2026/05/15	Backpropagation 3: Evaluating and describing the multi-layered perceptron model. [video]	Assignment 2 [2025 video]
13	2026/05/22	Unsupervised learning 1: Introduction to the principles of functional self-organization and convolution in V1 orientation selectivity. [Blackboard][video]	von der Malsburg, 1973 Hubel & Wiesel, 1963 Hubel & Wiesel, 1974 Bosking et al., 1997 [2025 Blackboard] [2025 video]
14	2026/05/29	Unsupervised learning 2: Unpacking the VDM model. [video]	Self-Organizing VDM Network Colab Jupyter Notebook [2025 video]
15	2026/06/05	Unsupervised Learning 3: Evaluating the VDM model. [video]	Assignment 3 [2025 video]
16	2026/06/12	Recurrent Neural Networks, Convolutional Neural Networks, Deep Neural Networks, Exploding and Diminishing Gradients, Overfitting, Regularization, Large Language Models. [video][blackboard]	Aggarwal, 2018, Ch. 2 Aggarwal, 2018, Ch. 3 [2025 video]

Bibliography

Jordan M. I. (1986). An introduction to linear algebra in parallel distributed processing. In Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations. Ed. David E. Rumelhart & James L. McLelland, p365–422, MIT Press, Cambridge: USA.
Neural Networks and Deep Learning: A Textbook. (2018). Charu C. Aggarwal, Springer, Cham, Switzerland. Ch. 1.
Hopfield, J. J. (1982). Neural networks and physical systems with emergent collective computational abilities. Proceedings of the National Academy of Sciences USA, 79, 2554-2558.
Rumelhart, D., Hinton, G., & Williams, R. (1986). Learning internal representations by error propagation. MIT Press Cambridge, MA, USA.
Von Der Malsburg, C. (1973). Self-organization of orientation sensitive cells in the striate cortex. Kybernetik, 14, 85-100.
Goh, J. O. S. (2007). Hopfield Network. Unpublished. https://gitlab.com/joshuagoh/hopfield-network
Goh, J. O. S. (2007). Backpropagation in a non-linear layered network: learning from past mistakes. Unpublished. https://gitlab.com/joshuagoh/multi-layer-perceptron
Goh, J. O. S. (2007). Extensions of the von der Malsburg self-organized visual cortex model. Unpublished. https://gitlab.com/joshuagoh/unsupervised-learning.

Other Resources