Homepage / Notes / Computer Science / Artificial Intelligence / Machine Learning
https://www.youtube.com/watch?v=zjkBMFhNj_g
https://explainextended.com/2023/12/31/happy-new-year-15/
Process of reducing the number of bits (weights and biases) of a model/neural network. The primary goal is to compress the model in size for faster execution/computation without sacrificing too much in terms of accuracy.
Example: FP32 ("full precision"), FP16 ("half precision")
The more bits we use to represent a value, the more precise it generally is. The more bits we have available, the larger the range of values that can be represented.
https://newsletter.maartengrootendorst.com/p/a-visual-guide-to-quantization
Efficient training method by creating and updating low-rank approximations of the original weight matrices (update matrices).
http://introtodeeplearning.com/
https://atcold.github.io/pytorch-Deep-Learning/
https://www2.eecs.berkeley.edu/Courses/CS182/
https://www.deeplearningbook.org/
https://sebastianraschka.com/blog/2021/dl-course.html
https://physicsbaseddeeplearning.org/intro.html
by François Chollet
A free course designed for people with some coding experience, who want to learn how to apply deep learning and machine learning to practical problems.
https://www.youtube.com/watch?v=quh7z1q7-uc
A method of mathematical optimization. Algorithm to find the local minimum of a loss function.
A course by Andrej Karpathy on building neural networks, from scratch, in code. We start with the basics of backpropagation and build up to modern deep neural networks, like GPT. In my opinion language models are an excellent place to learn deep learning, even if your intention is to eventually go to other areas like computer vision because most of what you learn will be immediately transferable. This is why we dive into and focus on languade models. Prerequisites: solid programming (Python), intro-level math (e.g. derivative, gaussian).
https://karpathy.ai/zero-to-hero.html
The AI Developer Platform Weights & Biases helps AI developers build better models faster. Quickly track experiments, version and iterate on datasets, evaluate model performance, reproduce models, and manage your ML workflows end-to-end.
https://cornelltech.github.io/cs5785-fall-2018/index.html
by Aurélien Géron