CoCalc Logo Icon
StoreFeaturesDocsShareSupportNewsAboutPoliciesSign UpSign In
y33-j3T

Real-time collaboration for Jupyter Notebooks, Linux Terminals, LaTeX, VS Code, R IDE, and more,
all in one place. Commercial Alternative to JupyterHub.

GitHub Repository: y33-j3T/Coursera-Deep-Learning
 Path: blob/master/Sequence Models/Week 1/Building a Recurrent Neural Network - Step by Step/__pycache__/utils.cpython-36.pyc
Views: 13381
3

�)Z^�@sfddlZdd�Zdd�Zdd�Zdd	�Zd
d�Zdd�Zd
d�Zdd�Zdd�Z	ddd�Z
dd�ZdS)�NcCs$tj|tj|��}||jdd�S)Nr)�axis)�np�exp�max�sum)�x�e_x�r	�T/home/jovyan/work/Week 1/Building a Recurrent Neural Network - Step by Step/utils.py�softmaxsrcCs|d|dS)Ng+�����?g����MbP?r	)�lossZcur_lossr	r	r
�smoothsr
cs*dj�fdd�|D��}td|f�dS)N�c3s|]}�|VqdS)Nr	)�.0�ix)�
ix_to_charr	r
�	<genexpr>szprint_sample.<locals>.<genexpr>z----
 %s 
----)�join�print)Z	sample_ixr�txtr	)rr
�print_sample
srcCstjd|�|S)Ng�?)r�log)�
vocab_sizeZ
seq_lengthr	r	r
�get_initial_losssrcCs$tj|tj|��}||jdd�S)Nr)r)rrrr)rrr	r	r
rsc	Csrtjjd�tjj||�d}tjj||�d}tjj||�d}tj|df�}tj|df�}|||||d�}|S)ak
    Initialize parameters with small random values
    
    Returns:
    parameters -- python dictionary containing:
                        Wax -- Weight matrix multiplying the input, numpy array of shape (n_a, n_x)
                        Waa -- Weight matrix multiplying the hidden state, numpy array of shape (n_a, n_a)
                        Wya -- Weight matrix relating the hidden-state to the output, numpy array of shape (n_y, n_a)
                        b --  Bias, numpy array of shape (n_a, 1)
                        by -- Bias relating the hidden-state to the output, numpy array of shape (n_y, 1)
    �g{�G�z�?)�Wax�Waa�Wya�b�by)r�random�seed�randn�zeros)	Zn_aZn_xZn_yrrrrr�
parametersr	r	r
�initialize_parameterssr%c
Csj|d|d|d|d|df\}}}}}tjtj||�tj||�|�}ttj||�|�}	||	fS)Nrrrrr)r�tanh�dotr)
r$�a_prevrrrrrrZa_nextZp_tr	r	r
�rnn_step_forward,s,"r)cCs�|dtj||j�7<|d|7<tj|dj|�|d}d|||}|d|7<|dtj||j�7<|dtj||j�7<tj|d	j|�|d<|S)
N�dWya�dbyr�da_nextr�db�dWax�dWaar)rr'�T)�dy�	gradientsr$r�ar(�daZdarawr	r	r
�rnn_step_backward4sr5cCs�|d||d7<|d||d7<|d||d7<|d||d7<|d	||d
7<|S)Nrr.rr/rr*rr-rr+r	)r$r2Zlrr	r	r
�update_parameters@sr6�GcCs�iii}}}tj|�|d<d}xxtt|��D]h}	tj|df�||	<d||	||	<t|||	d||	�\||	<||	<|tj||	||	df�8}q0W|||f}
||
fS)Nrr�����)r�copy�range�lenr#r)r)�X�Y�a0r$rrr3�y_hatr�t�cacher	r	r
�rnn_forwardIs$"
rBc
Cs�i}|\}}}|d|d|d|d|df\}}	}
}}tj|	�tj|�tj|
�|d<|d<|d<tj|�tj|�|d	<|d
<tj|d�|d<x\ttt|���D]H}
tj||
�}|||
d
8<t|||||
||
||
d
�}q�W||fS)Nrrrrrr.r/r*r-r+rr,r)r�
zeros_like�reversedr:r;r9r5)r<r=r$rAr2r?r3rrrrrrr@r1r	r	r
�rnn_backwardcs
,.&rE)r7)�numpyrrr
rrr%r)r5r6rBrEr	r	r	r
�<module>s