Real-time collaboration for Jupyter Notebooks, Linux Terminals, LaTeX, VS Code, R IDE, and more,
all in one place. Commercial Alternative to JupyterHub.
Real-time collaboration for Jupyter Notebooks, Linux Terminals, LaTeX, VS Code, R IDE, and more,
all in one place. Commercial Alternative to JupyterHub.
Path: blob/master/Convolutional Neural Networks/dummy/__pycache__/input_data.cpython-35.pyc
Views: 13377
�߱Yh � @ s d Z d d l m Z d d l m Z d d l m Z d d l Z d d l Z d d l Z d d l Z d d l
m Z d d l
m Z d d l Z
d Z d d
� Z d d � Z d
d � Z d d d � Z d d d � Z Gd d � d e � Z d d e
j d d � Z d S)z1Functions for downloading and reading MNIST data.� )�absolute_import)�division)�print_functionN)�urllib)�xrangez!http://yann.lecun.com/exdb/mnist/c C s� t j j | � s t j | � t j j | | � } t j j | � s� t j j t | | � \ } } t j | � } t
d | | j d � | S)z@Download the data from Yann's website, unless it's already here.zSuccessfully downloadedzbytes.) �os�path�exists�mkdir�joinr �request� urlretrieve�
SOURCE_URL�stat�print�st_size)�filename�work_directory�filepath�_Zstatinfo� r �/output/input_data.py�maybe_download s
r c C s; t j t j � j d � } t j | j d � d | �d S)N�>� �dtyper )�numpyr �uint32� newbyteorder�
frombuffer�read)�
bytestream�dtr r r �_read32% s r# c C s� t d | � t j | � �� } t | � } | d k rM t d | | f � � t | � } t | � } t | � } | j | | | � } t j | d t j �} | j | | | d � } | SWd QRXd S)zDExtract the images into a 4D uint8 numpy array [index, y, x, depth].�
Extractingi z/Invalid magic number %d in MNIST image file: %sr � N)
r �gzip�openr# �
ValueErrorr r r �uint8�reshape)r r! �magic�
num_images�rows�cols�buf�datar r r �extract_images( s
r1 �
c C sP | j d } t j | � | } t j | | f � } d | j | | j � <| S)z5Convert class labels from scalars to one-hot vectors.r r% )�shaper �arange�zeros�flat�ravel)� labels_dense� num_classes�
num_labels� index_offset�labels_one_hotr r r �dense_to_one_hot8 s
r= Fc C s� t d | � t j | � �| } t | � } | d k rM t d | | f � � t | � } | j | � } t j | d t j �} | r� t | � S| SWd QRXd S)z7Extract the labels into a 1D uint8 numpy array [index].r$ i z/Invalid magic number %d in MNIST label file: %sr N)
r r& r' r# r( r r r r) r= )r �one_hotr! r+ � num_itemsr/ �labelsr r r �extract_labels? s
rA c @ s e Z d Z d d e j d d � Z e d d � � Z e d d � � Z e d d � � Z e d
d � � Z
d d d
� Z d S)�DataSetFc C s: t j | � j } | t j t j f k r: t d | � � | rU d | _ | | _ n� | j d | j d k s� t d | j | j f � � | j d | _ | j d d k s� t � | j
| j d | j d | j d � } | t j k r| j t j � } t j
| d � } | | _ | | _ d | _ d | _ d
S) z�Construct a DataSet.
one_hot arg is used only if fake_data is true. `dtype` can be either
`uint8` to leave the input as `[0, 255]`, or `float32` to rescale into
`[0, 1]`.
z1Invalid image dtype %r, expected uint8 or float32i' r z!images.shape: %s labels.shape: %s� r% � g �?g �o@Ngp?)�tf�as_dtype�
base_dtyper) �float32� TypeError�
_num_examplesr> r3 �AssertionErrorr* �astyper �multiply�_images�_labels�_epochs_completed�_index_in_epoch)�self�imagesr@ � fake_datar> r r r r �__init__O s*
zDataSet.__init__c C s | j S)N)rN )rR r r r rS o s zDataSet.imagesc C s | j S)N)rO )rR r r r r@ r s zDataSet.labelsc C s | j S)N)rJ )rR r r r � num_examplesu s zDataSet.num_examplesc C s | j S)N)rP )rR r r r �epochs_completedx s zDataSet.epochs_completedc sA | ru d g d � | j r3 d g d g d � n d � � f d d � t | � D� � f d d � t | � D� f S| j } | j | 7_ | j | j k r| j d 7_ t j | j � } t j j | � | j | | _ | j
| | _
d } | | _ | | j k st � | j } | j | | � | j
| | � f S)z9Return the next `batch_size` examples from this data set.r% i r � c s g | ] } � � q Sr r )�.0r )�
fake_imager r �
<listcomp>� s z&DataSet.next_batch.<locals>.<listcomp>c s g | ] } � � q Sr r )rY r )�
fake_labelr r r[ � s ) r> r rQ rJ rP r r4 �random�shufflerN rO rK )rR �
batch_sizerT �start�perm�endr )rZ r\ r �
next_batch{ s(
zDataSet.next_batchN) �__name__�
__module__� __qualname__rE rH rU �propertyrS r@ rV rW rc r r r r rB N s rB c s� Gd d � d t � } | � } | rb � � f d d � } | � | _ | � | _ | � | _ | Sd } d } d } d }
d } t | | � } t | � }
t | | � } t | d
� �} t | | � } t | � } t |
| � } t | d
� �} |
d | � } | d | � } |
| d � }
| | d � } t |
| d � �| _ t | | d � �| _ t | | d � �| _ | S) Nc @ s e Z d Z d S)z read_data_sets.<locals>.DataSetsN)rd re rf r r r r �DataSets� s rh c s t g g d d d � d � �S)NrT Tr> r )rB r )r r> r r �fake� s z read_data_sets.<locals>.fakeztrain-images-idx3-ubyte.gzztrain-labels-idx1-ubyte.gzzt10k-images-idx3-ubyte.gzzt10k-labels-idx1-ubyte.gzi� r> r )�object�train�
validation�testr r1 rA rB )� train_dirrT r> r rh Z data_setsri � TRAIN_IMAGES� TRAIN_LABELS� TEST_IMAGES� TEST_LABELSZVALIDATION_SIZE�
local_file� train_images� train_labels� test_images� test_labels�validation_images�validation_labelsr )r r> r �read_data_sets� s< rz )�__doc__�
__future__r r r r& r �tensorflow.python.platform�
tensorflowr � six.movesr r rE r r r# r1 r= rA rj rB rH rz r r r r �<module> s$
G