CoCalc Logo Icon
StoreFeaturesDocsShareSupportNewsAboutPoliciesSign UpSign In
y33-j3T

Real-time collaboration for Jupyter Notebooks, Linux Terminals, LaTeX, VS Code, R IDE, and more,
all in one place. Commercial Alternative to JupyterHub.

GitHub Repository: y33-j3T/Coursera-Deep-Learning
 Path: blob/master/Sequence Models/Week 1/Dinosaur Island -- Character-level language model/__pycache__/utils.cpython-36.pyc
Views: 13420
3

0�oZ1�@sfddlZdd�Zdd�Zdd�Zdd	�Zd
d�Zdd�Zd
d�Zdd�Zdd�Z	ddd�Z
dd�ZdS)�NcCs$tj|tj|��}||jdd�S)Nr)�axis)�np�exp�max�sum)�x�e_x�r	�S/home/jovyan/work/Week 1/Dinosaur Island -- Character-level language model/utils.py�softmaxsrcCs|d|dS)Ng+�����?g����MbP?r	)�lossZcur_lossr	r	r
�smoothsr
csFdj�fdd�|D��}|dj�|dd�}td|fdd�dS)N�c3s|]}�|VqdS)Nr	)�.0�ix)�
ix_to_charr	r
�	<genexpr>szprint_sample.<locals>.<genexpr>r�z%s)�end)�join�upper�print)Z	sample_ixr�txtr	)rr
�print_sample
srcCstjd|�|S)Ng�?)r�log)�
vocab_sizeZ
seq_lengthr	r	r
�get_initial_losssrcCs$tj|tj|��}||jdd�S)Nr)r)rrrr)rrr	r	r
rsc	Csrtjjd�tjj||�d}tjj||�d}tjj||�d}tj|df�}tj|df�}|||||d�}|S)ak
    Initialize parameters with small random values
    
    Returns:
    parameters -- python dictionary containing:
                        Wax -- Weight matrix multiplying the input, numpy array of shape (n_a, n_x)
                        Waa -- Weight matrix multiplying the hidden state, numpy array of shape (n_a, n_a)
                        Wya -- Weight matrix relating the hidden-state to the output, numpy array of shape (n_y, n_a)
                        b --  Bias, numpy array of shape (n_a, 1)
                        by -- Bias relating the hidden-state to the output, numpy array of shape (n_y, 1)
    rg{�G�z�?)�Wax�Waa�Wya�b�by)r�random�seed�randn�zeros)	Zn_aZn_xZn_yrrrr r!�
parametersr	r	r
�initialize_parameterssr'c
Csj|d|d|d|d|df\}}}}}tjtj||�tj||�|�}ttj||�|�}	||	fS)Nrrrr!r )r�tanh�dotr)
r&�a_prevrrrrr!r Za_nextZp_tr	r	r
�rnn_step_forward-s,"r+cCs�|dtj||j�7<|d|7<tj|dj|�|d}d|||}|d|7<|dtj||j�7<|dtj||j�7<tj|d	j|�|d<|S)
N�dWya�dbyr�da_nextr�db�dWax�dWaar)rr)�T)�dy�	gradientsr&r�ar*�daZdarawr	r	r
�rnn_step_backward5sr7cCs�|d||d7<|d||d7<|d||d7<|d||d7<|d	||d
7<|S)Nrr0rr1rr,r r/r!r-r	)r&r4Zlrr	r	r
�update_parametersAsr8�cCs�iii}}}tj|�|d<d}x�tt|��D]t}	tj|df�||	<||	dkrbd||	||	<t|||	d||	�\||	<||	<|tj||	||	df�8}q0W|||f}
||
fS)Nrr�����)r�copy�range�lenr%r+r)�X�Y�a0r&rrr5�y_hatr�t�cacher	r	r
�rnn_forwardJs$"
rDc
Cs�i}|\}}}|d|d|d|d|df\}}	}
}}tj|	�tj|�tj|
�|d<|d<|d<tj|�tj|�|d	<|d
<tj|d�|d<x\ttt|���D]H}
tj||
�}|||
d
8<t|||||
||
||
d
�}q�W||fS)Nrrrr!r r0r1r,r/r-rr.r)r�
zeros_like�reversedr<r=r;r7)r>r?r&rCr4rAr5rrrrr!r rBr3r	r	r
�rnn_backwardfs
,.&rG)r9)�numpyrrr
rrr'r+r7r8rDrGr	r	r	r
�<module>s