Convolutional Neural Networks | Study

  References L. Fei-Fei, Justin Johnson (Spring 2017)CS231n: Convolutional Neural Networks for Visual Recognition. [LINK] Jefkine (5 September 2016). Backpropagation In Convolutional Neural Networks. [LINK] Convnet: Implementing Convolution Layer with Numpy [LINK] CNN의 역전파(backpropagation) [LINK]

CS231n: Convolutional Neural Networks for Visual Recognition | Course

Lecture 6 | Training Neural Networks I Sigmoid Problems of the sigmoid activation function Problem 1: Saturated neurons kill the gradients. Problem 2: Sigmoid outputs are not zero-centered. Suppose a given feed-forward neural network has hidden layers and all activation functions are sigmoid. Then, except the first layer, the other layers get only positive inputs. […]

Applying to Ph.D. Programs in Computer Science

  Author: Mor Harchol-Balter (Computer Science Department Carnegie Mellon University) Last updated: 2014 1 Introduction This document is intended for people applying to Ph.D. programs in computer science or related areas. The author is a professor of computer science at CMU, and has been involved in the Ph.D. admissions process at CMU, U.C. Berkeley, and MIT. 2 Do I […]

Neural Networks and Learning Machines. 3rd Ed. Simon O. Haykins. Pearson. 2008

Chapter 8. Principal-Components Analysis 8.1 Introduction 8.2 Principles of Self-Organization Principle 1. Self-Amplification Principle 2. Competition Principle 3. Cooperation Principle 4. Structural Information 8.3 Self-Organized Feature Analysis 8.4 Principal-Components Analysis: Perturbation Theory 8.5 Hebbian-Based maximum Eigenfilter 8.6 Hebbian-Based Principal Components Analysis 8.7 Case Study: Image Coding 8.8 Kernel Principal-Components Analysis 8.9 Basic Issues Involved in […]

Computational Neuroscience | Course | MS CogSci

  Range 8.1~8.7 9.1~9.10 10.1~10.14 10.19~10.21 Chapter 8. Principal-Components Analysis 8.1. Introduction Self-organized learning Self-organized learning is a type of unsupervised learning. locality of learning 8.2. Principles of Self-Organization Principle 1: self-amplification The following rule is based on Hebb’s postulate of learning. If two neurons of a synapse are activated simultaneously, then synaptic strength is […]

Sequence to Sequence Learning with Neural Networks | Summary

References Ilya Sutskever, Oriol Vinyals, Quoc V. Le (2014). “Sequence to Sequence Learning with Neural Networks”. NIPS 2014: 3104-3112. [PDF] Sequence-to-Sequence Models. TensorFlow [LINK] The official tutorial for sequence-to-sequence models. Seq2seq Library (contrib). Tensorflow [LINK] Translation with a Sequence to Sequence Network and Attention. PyTorch. [LINK]