mirror of
https://github.com/NotXia/unibo-ai-notes.git
synced 2025-12-14 18:51:52 +01:00
@ -39,7 +39,7 @@
|
||||
\begin{description}
|
||||
\item[Training]
|
||||
Given the predicted distribution $\hat{\vec{y}}^{(t)}$ and ground-truth $\vec{y}^{(t)}$ at step $t$, the loss is computed as the cross-entropy:
|
||||
\[ \mathcal{L}^{(t)}(\matr{\theta}) = - \sum_{v \in V} \vec{y}_v^{(t)} \log\left( \hat{\vec{y}}_w^{(t)} \right) \]
|
||||
\[ \mathcal{L}^{(t)}(\matr{\theta}) = - \sum_{v \in V} \vec{y}_v^{(t)} \log\left( \hat{\vec{y}}_v^{(t)} \right) \]
|
||||
|
||||
\begin{description}
|
||||
\item[Teacher forcing] \marginnote{Teacher forcing}
|
||||
@ -68,4 +68,4 @@
|
||||
\item[Greedy] Select the token with the highest probability.
|
||||
\item[Sampling] Randomly sample the token following the probabilities of the output distribution.
|
||||
\end{descriptionlist}
|
||||
\end{description}
|
||||
\end{description}
|
||||
|
||||
Reference in New Issue
Block a user