dataset.cpython-35.pyc



��Y�%�@sZdd
lZ
dd
lZddlmZ
d
dd�
ZGdd�d�Zdd�Zd
S)	�N)
�cachecCs6|
d
krtj
|�
d}
tj|
dt�

|S)a�

    Generate the One-Hot encoded class-labels from an array of integers.
    For example, if class_number=2 and num_classes=4 then
    the one-hot encoded label is the float array: [0. 0. 1. 0.]
    :param class_numbers:
        Array of integers with class-numbers.
        Assume the integers are from zero to num_classes-1 inclusive.
    :param num_classes:
        Number of classes. If None then use max(class_numbers)+1.
    :return:
        2-dim array of shape: [len(class_numbers), num_classes]
    N�
�dtype)�np�max�eye�float)�
class_numbers�num_classes�r�/output/dataset.py�one_hot_encodeds
r
c@sReZ
dZd
dd�
Zdd�Zddd�
Zd	d
�Zdd�Zd
S)�DataSetz.jpgc	Csm
tj
j|
�
}
|
|_td
d�|D�
�
|_g|_g|_g|_g|_	g|_
d|_x�tj|
�
D]�}tj
j
|
|�}tj
j|�
r}|jj|�

|j|�
}|jj|�

|j}|g
t|�
}|j	j|�

|jtj
j
|d��
}|jj|�

|g
t|�
}|j
j|�

|jd7_q}WdS)a�
        Create a data-set consisting of the filenames in the given directory
        and sub-dirs that match the given filename-extensions.
        For example, the knifey-spoony data-set (see knifey.py) has the
        following dir-structure:
        knifey-spoony/forky/
        knifey-spoony/knifey/
        knifey-spoony/spoony/
        knifey-spoony/forky/test/
        knifey-spoony/knifey/test/
        knifey-spoony/spoony/test/
        This means there are 3 classes called: forky, knifey, and spoony.
        If we set in_dir = "knifey-spoony/" and create a new DataSet-object
        then it will scan through these directories and create a training-set
        and test-set for each of these classes.
        The training-set will contain a list of all the *.jpg filenames
        in the following directories:
        knifey-spoony/forky/
        knifey-spoony/knifey/
        knifey-spoony/spoony/
        The test-set will contain a list of all the *.jpg filenames
        in the following directories:
        knifey-spoony/forky/test/
        knifey-spoony/knifey/test/
        knifey-spoony/spoony/test/
        See the TensorFlow Tutorial #09 for a usage example.
        :param in_dir:
            Root-dir for the files in the data-set.
            This would be 'knifey-spoony/' in the example above.
        :param exts:
            String or tuple of strings with valid filename-extensions.
            Not case-sensitive.
        :return:
            Object instance.
        c
ss|]}
|
j�V
qdS)
N)
�lower)�.0�extrrr�	<genexpr>bsz#DataSet.__init__.<locals>.<genexpr>r
�testrN)�os�path�abspath�in_dir�tuple�exts�class_names�	filenames�filenames_testr	�class_numbers_testr
�listdir�join�isdir�append�_get_filenames�extend�len)	�selfrr�nameZcurrent_dirrZclass_numberr	rrrr�__init__6s.&								zDataSet.__init__cCs[g}tj
j|
�
rWx<tj|
�
D]+}|j�j|j�
r(|j|�

q(W|S)
a

        Create and return a list of filenames with matching extensions in the given directory.
        :param dir:
            Directory to scan for files. Sub-dirs are not scanned.
        :return:
            List of filenames. Only filenames. Does not include the directory.
        )rr�existsrr�endswithrr!)r%�dirr�filenamerrrr"�s


zDataSet._get_filenamesFccs�|
r!|j}|j
}d
}n|j}|j}d}xGt||�D]6\}}tjj|j|j	|||�}|V
qIWdS)z�
        Get the full paths for the files in the data-set.
        :param test:
            Boolean. Return the paths for the test-set (True) or training-set (False).
        :return:
            Iterator with strings for the path-names.
        ztest/�N)
rrrr	�ziprrrrr)r%rrr	Ztest_dirr+�clsrrrr�	get_paths�s		
			
	%zDataSet.get_pathsc

Cs:t|j
��
tj|j�
td
|jd|j�fS)z�
        Return the list of paths for the files in the training-set,
        and the list of class-numbers as integers,
        and the class-numbers as one-hot encoded arrays.
        r	r
)�listr/r�asarrayr	r
r
)
r%rrr�get_training_set�s


zDataSet.get_training_setc

Cs@t|j
d
d�
�
tj|j�
td|jd|j�fS)z�
        Return the list of paths for the files in the test-set,
        and the list of class-numbers as integers,
        and the class-numbers as one-hot encoded arrays.
        rTr	r
)r0r/rr1rr
r
)
r%rrr�get_test_set�s


zDataSet.get_test_setN)�__name__�
__module__�__qualname__r'r"r/r2r3rrrrr5s

lrcCs-td
|
�

t
d|dtd|
�}|S)a�
    Wrapper-function for creating a DataSet-object, which will be
    loaded from a cache-file if it already exists, otherwise a new
    object will be created and saved to the cache-file.
    This is useful if you need to ensure the ordering of the
    filenames is consistent every time you load the data-set,
    for example if you use the DataSet-object in combination
    with Transfer Values saved to another cache-file, see e.g.
    Tutorial #09 for an example of this.
    :param cache_path:
        File-path for the cache-file.
    :param in_dir:
        Root-dir for the files in the data-set.
        This is an argument for the DataSet-init function.
    :return:
        The DataSet-object.
    z$Creating dataset from the files in: �
cache_path�fnr)�printrr)r7r�datasetrrr�load_cached�s
r;)�numpyrrrr
rr;rrrr�<module>s


�