CoCalc Logo Icon
StoreFeaturesDocsShareSupportNewsAboutPoliciesSign UpSign In
y33-j3T

Real-time collaboration for Jupyter Notebooks, Linux Terminals, LaTeX, VS Code, R IDE, and more,
all in one place. Commercial Alternative to JupyterHub.

GitHub Repository: y33-j3T/Coursera-Deep-Learning
 Path: blob/master/Sequence Models/Week 1/Building a Recurrent Neural Network - Step by Step/__pycache__/rnn_utils.cpython-36.pyc
Views: 13383
3

&CZ#�@s.ddlZdd�Zdd�Zdd�Zddd
�ZdS)�NcCs$tj|tj|��}||jdd�S)Nr)�axis)�np�exp�max�sum)�xZe_x�r�X/home/jovyan/work/Week 1/Building a Recurrent Neural Network - Step by Step/rnn_utils.py�softmaxsr
cCsddtj|�S)N�)rr)rrrr	�sigmoidsrcCs�t|�d}i}i}x�t|�D]�}tj|dt|d�j�|dt|d�<tj|dt|d�j�|dt|d�<tj|dt|d�j�|dt|d�<tj|dt|d�j�|dt|d�<qW||fS)a&
    Initializes v and s as two python dictionaries with:
                - keys: "dW1", "db1", ..., "dWL", "dbL" 
                - values: numpy arrays of zeros of the same shape as the corresponding gradients/parameters.
    
    Arguments:
    parameters -- python dictionary containing your parameters.
                    parameters["W" + str(l)] = Wl
                    parameters["b" + str(l)] = bl
    
    Returns: 
    v -- python dictionary that will contain the exponentially weighted average of the gradient.
                    v["dW" + str(l)] = ...
                    v["db" + str(l)] = ...
    s -- python dictionary that will contain the exponentially weighted average of the squared gradient.
                    s["dW" + str(l)] = ...
                    s["db" + str(l)] = ...

    ��Wr�dW�b�db)�len�ranger�zeros�str�shape)�
parameters�L�v�s�lrrr	�initialize_adams,,,0r�{�G�z�?��������?�+�����?�:�0�yE>c	
	Cs�t|�d}	i}
i}�x�t|	�D�]�}||dt|d�d||dt|d�|dt|d�<||dt|d�d||dt|d�|dt|d�<|dt|d�d|||
dt|d�<|dt|d�d|||
dt|d�<||dt|d�d||dt|d�d|dt|d�<||dt|d�d||dt|d�d|dt|d�<|dt|d�d|||dt|d�<|dt|d�d|||dt|d�<|dt|d�||
dt|d�tj|dt|d�|�|dt|d�<|dt|d�||
dt|d�tj|dt|d�|�|dt|d�<q W|||fS)a;
    Update parameters using Adam
    
    Arguments:
    parameters -- python dictionary containing your parameters:
                    parameters['W' + str(l)] = Wl
                    parameters['b' + str(l)] = bl
    grads -- python dictionary containing your gradients for each parameters:
                    grads['dW' + str(l)] = dWl
                    grads['db' + str(l)] = dbl
    v -- Adam variable, moving average of the first gradient, python dictionary
    s -- Adam variable, moving average of the squared gradient, python dictionary
    learning_rate -- the learning rate, scalar.
    beta1 -- Exponential decay hyperparameter for the first moment estimates 
    beta2 -- Exponential decay hyperparameter for the second moment estimates 
    epsilon -- hyperparameter preventing division by zero in Adam updates

    Returns:
    parameters -- python dictionary containing your updated parameters 
    v -- Adam variable, moving average of the first gradient, python dictionary
    s -- Adam variable, moving average of the squared gradient, python dictionary
    r
rrrrr)rrrr�sqrt)
rZgradsrr�tZ
learning_rateZbeta1Zbeta2�epsilonrZv_correctedZs_correctedrrrr	�update_parameters_with_adam1sDD00HH00Z^r$)rrrr )�numpyrr
rrr$rrrr	�<module>s
%