softmax 函数

https://blog.csdn.net/m0_37769093/article/details/107732606

softmax 函数如下所示：

$$y_i=\frac{\exp (x_i)}{{\sum }_{j=1}^n\exp (x_j)}$$

softmax求导如下：

$i=j$ 的情况：

$$\frac{\partial y_i}{\partial x_i}=\frac{\exp (x_i)}{{\sum }_{j=1}^n\exp (x_j)}-\frac{(\exp (x_i){)}^2}{({\sum }_{j=1}^n\exp (x_j){)}^2}$$
$$\frac{\partial y_i}{\partial x_i}=y_i-(y_i{)}^2$$

$i\ne j$ 的情况：

$$\frac{\partial y_i}{\partial x_j}=-\frac{(\exp (x_i)\times \exp (x_j))}{({\sum }_{j=1}^n\exp (x_j){)}^2}$$
$$\frac{\partial y_i}{\partial x_j}=-y_i{y}_j$$