Path: blob/master/Sequence Models/Week 1/Building a Recurrent Neural Network - Step by Step/__pycache__/utils.cpython-36.pyc
24841 views
3
�)Z^ � @ sf d dl Zdd� Zdd� Zdd� Zdd � Zd
d� Zd d � Zd
d� Zdd� Zdd� Z ddd�Z
dd� Z dS )� Nc C s$ t j| t j| � �}||jdd� S )Nr )�axis)�np�exp�max�sum)�x�e_x� r �T/home/jovyan/work/Week 1/Building a Recurrent Neural Network - Step by Step/utils.py�softmax s r c C s | d |d S )Ng+�����?g����MbP?r )�lossZcur_lossr r r
�smooth s r
c s* dj � fdd�| D ��}td|f � d S )N� c 3 s | ]}� | V qd S )Nr )�.0�ix)�
ix_to_charr r
� <genexpr> s z print_sample.<locals>.<genexpr>z----
%s
----)�join�print)Z sample_ixr �txtr )r r
� print_sample
s r c C s t jd| � | S )Ng �?)r �log)�
vocab_sizeZ
seq_lengthr r r
�get_initial_loss s r c C s$ t j| t j| � �}||jdd� S )Nr )r )r r r r )r r r r r
r s c C sr t jjd� t jj| |�d }t jj| | �d }t jj|| �d }t j| df�}t j|df�}|||||d�}|S )ak
Initialize parameters with small random values
Returns:
parameters -- python dictionary containing:
Wax -- Weight matrix multiplying the input, numpy array of shape (n_a, n_x)
Waa -- Weight matrix multiplying the hidden state, numpy array of shape (n_a, n_a)
Wya -- Weight matrix relating the hidden-state to the output, numpy array of shape (n_y, n_a)
b -- Bias, numpy array of shape (n_a, 1)
by -- Bias relating the hidden-state to the output, numpy array of shape (n_y, 1)
� g{�G�z�?)�Wax�Waa�Wya�b�by)r �random�seed�randn�zeros) Zn_aZn_xZn_yr r r r r �
parametersr r r
�initialize_parameters s r% c
C sj | d | d | d | d | d f\}}}}}t jt j||�t j||� | �}tt j||�| �} || fS )Nr r r r r )r �tanh�dotr )
r$ �a_prevr r r r r r Za_nextZp_tr r r
�rnn_step_forward, s ,"r) c C s� |d t j| |j�7 < |d | 7 < t j|d j| �|d }d|| | }|d |7 < |d t j||j�7 < |d t j||j�7 < t j|d j|�|d< |S )
N�dWya�dbyr �da_nextr �db�dWax�dWaar )r r' �T)�dy� gradientsr$ r �ar( �daZdarawr r r
�rnn_step_backward4 s r5 c C s� | d | |d 7 < | d | |d 7 < | d | |d 7 < | d | |d 7 < | d | |d
7 < | S ) Nr r. r r/ r r* r r- r r+ r )r$ r2 Zlrr r r
�update_parameters@ s r6 �G c C s� i i i }}}t j|�|d< d}xxtt| ��D ]h} t j|df�|| < d|| | | < t||| d || �\|| < || < |t j|| || df �8 }q0W |||f}
||
fS )Nr r �����)r �copy�range�lenr# r) r ) �X�Y�a0r$ r r r3 �y_hatr �t�cacher r r
� rnn_forwardI s $"
rB c
C s� i }|\}}}|d |d |d |d |d f\}} }
} } t j| �t j|�t j|
� |d< |d< |d< t j| �t j| � |d < |d
< t j|d �|d < x\ttt| ���D ]H}
t j||
�}|||
d
8 < t|||||
||
||
d
�}q�W ||fS )Nr r r r r r. r/ r* r- r+ r r, r )r �
zeros_like�reversedr: r; r9 r5 )r< r= r$ rA r2 r? r3 r r r r r r r@ r1 r r r
� rnn_backwardc s
,. &rE )r7 ) �numpyr r r
r r r% r) r5 r6 rB rE r r r r
�<module> s