mirror of
https://github.com/NotXia/unibo-ai-notes.git
synced 2025-12-15 19:12:22 +01:00
Minor changes <noupdate>
This commit is contained in:
@ -77,13 +77,14 @@ There are two types of learning:
|
||||
|
||||
The extinct association can return in the future
|
||||
(this is more evident when the context is the same as the acquisition phase).
|
||||
|
||||
\begin{figure}[H]
|
||||
\centering
|
||||
\includegraphics[width=0.95\linewidth]{./img/pavlovian_extinction.png}
|
||||
\caption{Example of acquisition, extinction, and \ac{cr} return}
|
||||
\end{figure}
|
||||
\end{remark}
|
||||
|
||||
\begin{figure}[H]
|
||||
\centering
|
||||
\includegraphics[width=0.95\linewidth]{./img/pavlovian_extinction.png}
|
||||
\caption{Example of acquisition, extinction, and \ac{cr} return}
|
||||
\end{figure}
|
||||
|
||||
\begin{description}
|
||||
\item[Generalization] \marginnote{Generalization}
|
||||
@ -469,7 +470,7 @@ There is strong evidence that the dopaminergic system is the major neural mechan
|
||||
\end{casestudy}
|
||||
|
||||
\item[Probability encoding] \marginnote{Dopamine probability encoding}
|
||||
The phasic activation of dopamine neurons varies monotonically with the reward probability
|
||||
The phasic activation of dopamine neurons varies monotonically with the reward probability.
|
||||
\begin{casestudy}
|
||||
\phantom{}
|
||||
\begin{center}
|
||||
|
||||
@ -22,6 +22,7 @@
|
||||
\begin{remark}
|
||||
Multiple competing sub-systems contribute to learning and controlling behavior in animals.
|
||||
|
||||
\indenttbox
|
||||
\begin{example}[Freud's theory of the mind structure]
|
||||
The mind is composed of three structures:
|
||||
\begin{descriptionlist}
|
||||
|
||||
@ -3,7 +3,7 @@
|
||||
|
||||
Deep learning has a large impact on neuroscience. For instance:
|
||||
\begin{casestudy}[Neural network to represent parietal neurons \cite{parietal_network}]
|
||||
A monkey is tasked to maintain fixation at a point with a stimulus visible within its receptive field. The fixation point and the stimulus are moved together so that, for the retina, the stimulus is at the same retinal coordinate.
|
||||
A monkey is tasked to maintain fixation at a point with a stimulus visible within its receptive field. The fixation point and the stimulus are moved together so that, for the retina, the stimulus is at the same retinal coordinates.
|
||||
|
||||
\begin{figure}[H]
|
||||
\centering
|
||||
@ -116,7 +116,7 @@ There are at least two factors that cause the sample inefficiency of deep reinfo
|
||||
\end{remark}
|
||||
|
||||
\item[Long-term memory] \marginnote{Long-term memory}
|
||||
Inactive past memories that have to be reactivated to use them. In humans, it comes in different forms:
|
||||
Inactive past memories that have to be reactivated in order to use them. In humans, it comes in different forms:
|
||||
\begin{descriptionlist}
|
||||
\item[Declarative/Explicit]
|
||||
Memory that can be expressed in a propositional form (i.e. by words).
|
||||
@ -138,7 +138,7 @@ There are at least two factors that cause the sample inefficiency of deep reinfo
|
||||
\end{figure}
|
||||
|
||||
\begin{remark}
|
||||
On the other spectrum, there are short-term and working memories.
|
||||
On the other spectrum, there is short-term and working memory.
|
||||
The former, once formed, is sort of read-only (e.g. repeat a sequence of numbers). The latter allows the manipulation of its content (e.g. repeat a sequence of numbers in reverse order).
|
||||
\end{remark}
|
||||
|
||||
@ -212,7 +212,7 @@ There are at least two factors that cause the sample inefficiency of deep reinfo
|
||||
|
||||
\begin{casestudy}[Replay in rats]
|
||||
The movements and the activity of a neuron (place cell) in the hippocampus of a rat are recorded.
|
||||
It has been observed that the place cell fires only at a specific spatial area (and it changes if the rat is moved to another environment).
|
||||
It has been observed that the place cell fires only at a specific spatial area (which changes if the rat is moved to another environment).
|
||||
\begin{figure}[H]
|
||||
\centering
|
||||
\includegraphics[width=0.55\linewidth]{./img/hippocampus_replay1.png}
|
||||
@ -220,14 +220,21 @@ There are at least two factors that cause the sample inefficiency of deep reinfo
|
||||
|
||||
\indenttbox
|
||||
\begin{remark}
|
||||
Grid cells are another type of cells in the medial entorhinal cortex that fire based on spatial features.
|
||||
Differently from place cells, this type of cell is active in multiple zones arranged in a regular manner (i.e. triangular or hexagonal units).
|
||||
\phantom{}\\[0.5em]
|
||||
\begin{minipage}{0.6\linewidth}
|
||||
Grid cells are another type of cells in the medial entorhinal cortex that fire based on spatial features.
|
||||
Differently from place cells, this type of cell is active in multiple zones arranged in a regular manner (i.e. triangular or hexagonal units).
|
||||
|
||||
In other words, place cells are based on landmarks and grid cells are based on self-motion.
|
||||
|
||||
\end{minipage}
|
||||
\begin{minipage}{0.3\linewidth}
|
||||
\begin{figure}[H]
|
||||
\centering
|
||||
\includegraphics[width=0.9\linewidth]{./img/grid_place_cell.png}
|
||||
\end{figure}
|
||||
\end{minipage}
|
||||
|
||||
In other words, place cells are based on landmarks and grid cells are based on self-motion.
|
||||
\begin{figure}[H]
|
||||
\centering
|
||||
\includegraphics[width=0.25\linewidth]{./img/grid_place_cell.png}
|
||||
\end{figure}
|
||||
\end{remark}
|
||||
|
||||
By placing the rat on a triangular track where some reward is available, a correlation between two place cells have been recorded.
|
||||
@ -247,7 +254,7 @@ There are at least two factors that cause the sample inefficiency of deep reinfo
|
||||
\phantom{}
|
||||
\begin{description}
|
||||
\item[Experiment 1]
|
||||
In each trial, candidates are asked to choose a slot machine to spin (bandit problem). Each slot has a different point reward that changes at each trial.
|
||||
In each trial, candidates are asked to choose a slot machine to spin (bandit problem). Each slot has a different reward in points that changes at each trial.
|
||||
\begin{figure}[H]
|
||||
\centering
|
||||
\includegraphics[width=0.55\linewidth]{./img/memory_decision1.png}
|
||||
@ -294,7 +301,7 @@ There are at least two factors that cause the sample inefficiency of deep reinfo
|
||||
Monkeys are presented with two unfamiliar objects and are asked to grab one of them.
|
||||
Under an object lays either food or nothing.
|
||||
The same procedure is repeated for six trials and the position of the two objects can be swapped.
|
||||
After the round of trials, new rounds are repeated with new objects.
|
||||
After the first round of trials, new rounds are repeated with new objects.
|
||||
|
||||
It has been observed that after some rounds, monkeys are able to learn the task in one shot.
|
||||
|
||||
|
||||
@ -360,7 +360,7 @@ Despite that, dopamine still seems to integrate predictive information from mode
|
||||
|
||||
\begin{casestudy}[Monkey saccade \cite{saccade}]
|
||||
Monkeys are required to solve a memory-guided saccade task where, after fixation,
|
||||
a light is flashed in one of the four directions indicating the saccade to be made after the fixation point went off.
|
||||
a light is flashed in one of the four directions indicating the saccade to be made after the fixation point goes off.
|
||||
\begin{figure}[H]
|
||||
\centering
|
||||
\includegraphics[width=0.4\linewidth]{./img/saccade1.png}
|
||||
@ -379,9 +379,9 @@ Despite that, dopamine still seems to integrate predictive information from mode
|
||||
|
||||
It is expected that the animal's reward prediction increases after each non-rewarded trial.
|
||||
In other words, as the reward is more likely after each non-rewarded trial, positive prediction error should decrease and
|
||||
negative prediction error should be stronger (i.e. decrease).
|
||||
negative prediction error should be stronger.
|
||||
|
||||
Results show that dopamine neurons are less active if the reward is delivered
|
||||
Results show that dopamine neurons are less active if the reward is delivered later
|
||||
and more depressed if the reward is omitted after each non-rewarded trial.
|
||||
\begin{figure}[H]
|
||||
\centering
|
||||
@ -389,7 +389,7 @@ Despite that, dopamine still seems to integrate predictive information from mode
|
||||
\end{figure}
|
||||
|
||||
The results are in contrast with an exclusive model-free view of dopamine as, if this were the case,
|
||||
learning would only involve past non-rewarded trials causing positive prediction error to decrease and negative prediction error to be weaker (i.e. increase).
|
||||
learning would only involve past non-rewarded trials causing positive prediction error to decrease and negative prediction error to be weaker.
|
||||
Therefore, dopamine might process prediction error in both model-free and model-based approaches.
|
||||
\end{casestudy}
|
||||
|
||||
|
||||
@ -79,7 +79,7 @@
|
||||
|
||||
\begin{casestudy}[Agnosia]
|
||||
Patients with agnosia have their last level of vision damaged.
|
||||
They can see (e.g. avoid obstacles) but cannot recognize object or get easily confused.
|
||||
They can see (e.g. avoid obstacles) but cannot recognize objects or get easily confused.
|
||||
\end{casestudy}
|
||||
\end{descriptionlist}
|
||||
|
||||
@ -183,6 +183,7 @@
|
||||
\begin{remark}
|
||||
Neurons might only react to a specific type of stimuli in the receptive field (e.g. color, direction, \dots).
|
||||
|
||||
\indenttbox
|
||||
\begin{casestudy}
|
||||
It has been seen that a neuron fires only if a stimulus is presented in its receptive field while moving upwards.
|
||||
\end{casestudy}
|
||||
@ -218,7 +219,7 @@
|
||||
have a visual angle of about $0.1^\circ$ while the neurons at the visual periphery reach up to $1^\circ$ of visual angle.
|
||||
|
||||
Accordingly, more cortical space is dedicated to the central part of the visual field.
|
||||
This densely packed amount of smaller receptive fields allows for the highest spatial resolution at the center of the visual field.
|
||||
This densely packed amount of smaller receptive fields allows to obtain the highest spatial resolution at the center of the visual field.
|
||||
|
||||
\begin{figure}[H]
|
||||
\centering
|
||||
@ -359,7 +360,7 @@ This is the result of the alignment of different circular receptive fields of th
|
||||
\marginnote{Complex cells}
|
||||
|
||||
Neurons with a rectangular receptive field larger than simple cells.
|
||||
They respond to linear stimuli with a specific orientation and move in a particular direction (position invariance).
|
||||
They respond to linear stimuli with a specific orientation and with a specific movement direction (position invariance).
|
||||
|
||||
\begin{remark}
|
||||
At this stage, the position of the stimulus is not relevant anymore as the ON and OFF zones of the previous cells are mixed.
|
||||
@ -369,7 +370,7 @@ They respond to linear stimuli with a specific orientation and move in a particu
|
||||
\centering
|
||||
\begin{subfigure}{0.45\linewidth}
|
||||
\centering
|
||||
\includegraphics[width=0.9\linewidth]{./img/complex_cell.png}
|
||||
\includegraphics[width=0.85\linewidth]{./img/complex_cell.png}
|
||||
\end{subfigure}
|
||||
\begin{subfigure}{0.45\linewidth}
|
||||
\centering
|
||||
|
||||
@ -711,7 +711,7 @@ Therefore, the complete workflow for image formation becomes the following:
|
||||
\end{remark}
|
||||
|
||||
\item[Initial distortion parameters guess]
|
||||
The current estimate of the homographies $\matr{H}_i$ project WRF points into ideal (undistorted) IRF coordinates $\vec{m}_\text{undist}$.
|
||||
The current estimate of the homographies $\matr{H}_i$ project WRF points into ideal (undistorted) IRF coordinates $\vec{m}_\text{undist}$ (this is an assumption).
|
||||
On the other hand, the coordinates $\vec{m}$ of the corners in the actual image are distorted.
|
||||
|
||||
The original algorithm estimates the parameters of the radial distortion function defined as:
|
||||
|
||||
Reference in New Issue
Block a user