Incremental GSC parsing (exact copy of the tutorials shared as a separate project) / tqdm / _tqdm.pyc
1576 views
9�W�� � @ s d Z d d l m Z d d l m Z d d l m Z m Z m Z m Z m Z m
Z
m Z d d l Z d d l
m
Z
d d d
d d d
d g i Z d d d d d g Z Gd d � d e � Z Gd d � d e � Z Gd d � d e � Z Gd d � d e � Z d d � Z d S)z�
Customisable progressbar decorator for iterators.
Includes a default (x)range iterator printing to stderr.
Usage:
>>> from tqdm import trange[, tqdm]
>>> for i in trange(10): #same as: for i in tqdm(xrange(10))
... ...
� )�absolute_import)�division� )�_supports_unicode�_environ_cols_wrapper�_range�_unich�
_term_move_up�_unicode�WeakSetN)�timez github.com/ZnoamraphZobiwanusZkmikeZhadimZ casperdclZlrq3000�tqdm�trange�
TqdmTypeError� TqdmKeyError�TqdmDeprecationWarningc @ s e Z d Z d S)r N)�__name__�
__module__� __qualname__� r r �)lib/python3.5/site-packages/tqdm/_tqdm.pyr s c @ s e Z d Z d S)r N)r r r r r r r r s c s% e Z d Z d � f d d � Z � S)r Nc sM | d k r- | d t | � j � d � n t t | � j | | | � d S)Nz
TqdmDeprecationWarning: �
)�str�rstrip�superr �__init__)�self�msg�fp_write�a�k)� __class__r r r % s !z TqdmDeprecationWarning.__init__)r r r r r r )r! r r # s c @ s e Z d Z d Z e d d d � � Z e d d � � Z e d d � � Z e d d d
d d
d d d d
� � Z d d � Z e
d d d � � Z e
d d � � Z e
e
j d d d � � Z e
d d � � Z d d d d e
j d d d d d d
d d
d
d d d d d
d d � Z d d! � Z d" d# � Z d$ d% � Z d& d' � Z d( d) � Z d* d+ � Z d, d- � Z d. d/ � Z d0 d1 � Z d2 d3 � Z d4 d5 � Z d6 d7 � Z d8 d9 � Z d: d; d< � Z d= d> � Z! d? d@ � Z" d dA dB � Z# dC dD � Z$ d
dE dF � Z% dG dH � Z& d S)Ir
z�
Decorate an iterable object, returning an iterator which acts exactly
like the original iterable, but prints a dynamically updating
progressbar every time a value is requested.
� c C s� x� d d d d d d d d g D]� } t | � d k r� t | � d
k r� t | � d k rp d j | � | | Sd
j | � | | Sd j | � | | S| d } q Wd j | � | S)a~
Formats a number (greater than unity) with SI Order of Magnitude
prefixes.
Parameters
----------
num : float
Number ( >= 1) to format.
suffix : str, optional
Post-postfix [default: ''].
Returns
-------
out : str
Number with Order of Magnitude SI unit postfix.
r" �K�M�G�T�P�E�Zg�����?�@g������X@g=
ףp�#@z{0:1.2f}z{0:2.1f}z{0:3.0f}g @�@z {0:3.1f}Y)�abs�format)Znum�suffix�unitr r r �
format_sizeof2 s %ztqdm.format_sizeofc C s] t t | � d � \ } } t | d � \ } } | rI d j | | | � Sd j | | � Sd S)z�
Formats a number of seconds as a clock time, [H:]MM:SS
Parameters
----------
t : int
Number of seconds.
Returns
-------
out : str
[H:]MM:SS
�< z{0:d}:{1:02d}:{2:02d}z{0:02d}:{1:02d}N)�divmod�intr+ )�tZmins�s�h�mr r r �format_intervalN s
ztqdm.format_intervalc sU | � t � d d d � � � � � f d d � � d g � � � f d d � } | S) z�
Manage the printing and in-place updating of a line of characters.
Note that if the string is longer than a line, then in-place
updating may not work (it will print a new line at each refresh).
�flushc S s d S)Nr r r r r �<lambda>k s z%tqdm.status_printer.<locals>.<lambda>c s � j t | � � � � d S)N)�writer
)r3 )�fp�fp_flushr r r m s z%tqdm.status_printer.<locals>.fp_writer c sA t | � } � d | d t � d | d � � | � d <d S)N�
� r )�len�max)r3 Zlen_s)r �last_lenr r � print_statuss s 'z)tqdm.status_printer.<locals>.print_status)�getattr)�filerA r )r: r; r r@ r �status_printerc s ztqdm.status_printerNF�itc
C s
| r | | k r d } t j }
|
| � } | d k rI | rI | | } | re | d k re d | n d } t j }
| r� | r� |
| r� | n | � q� d j | r� | n | � n d | r� d n | d | r� | n d } | r
|
| � } | r|
| � n d } n t | � } t | � } | r�| | } | d } | rV|
| | | � n d } | rh| n d d j | � } d
j | | | | | � } | d k r�| d d$ � | d d � S| r�d | d
| d | d | d | d | d k r�| n | d | d | r*| r|
| � q-d j | � n d | d d | d | d | d | d | d | rb| n d i } d | k r�| j d � \ } } | j | � } | j | � } n
| j | � S| r�t d | t | � t | � � n d } | r;t t | | d � d � \ } } d | } | r2t
d | � n d } nO t t | | d � d � \ } } t d! � | } | r�t d" | � n d } | | k r�| | d t | | d d � } n | d t | | d � } | | | S| r�| n d d# j | | | | � Sd S)%a�
Return a string-based progress bar given some parameters
Parameters
----------
n : int
Number of finished iterations.
total : int
The expected total number of iterations. If meaningless (), only
basic progress statistics are displayed (no ETA).
elapsed : float
Number of seconds passed since start.
ncols : int, optional
The width of the entire output message. If specified,
dynamically resizes the progress meter to stay within this bound
[default: None]. The fallback meter width is 10 for the progress
bar + no limit for the iterations counter and statistics. If 0,
will not print any meter (only stats).
prefix : str, optional
Prefix message (included in total width) [default: ''].
ascii : bool, optional
If not set, use unicode (smooth blocks) to fill the meter
[default: False]. The fallback is to use ASCII characters
(1-9 #).
unit : str, optional
The iteration unit [default: 'it'].
unit_scale : bool, optional
If set, the number of iterations will printed with an
appropriate SI metric prefix (K = 10^3, M = 10^6, etc.)
[default: False].
rate : float, optional
Manual override for iteration rate.
If [default: None], uses n/elapsed.
bar_format : str, optional
Specify a custom bar string formatting. May impact performance.
[default: '{l_bar}{bar}{r_bar}'], where l_bar is
'{desc}{percentage:3.0f}%|' and r_bar is
'| {n_fmt}/{total_fmt} [{elapsed_str}<{remaining_str}, {rate_fmt}]'
Possible vars: bar, n, n_fmt, total, total_fmt, percentage,
rate, rate_fmt, elapsed, remaining, l_bar, r_bar, desc.
Returns
-------
out : Formatted meter and stats, ready to display.
Nr z{0:5.2f}�?r3 �/�d r" z
{0:3.0f}%|z| {0}/{1} [{2}<{3}, {4}]r �n�n_fmt�total� total_fmt�
percentage�rateZ
rate_noinvZrate_noinv_fmtz/s�rate_fmt�elapsedZ remaining�l_bar�r_bar�descz{bar}�
�#�0 r= � i�% i�% z{0}{1} [{2}, {3}]�����) r
r6 r. r+ r �splitr? r> r0 r1 �chrr ) rI rK rP �ncols�prefix�asciir- �
unit_scalerN �
bar_formatr6 Z elapsed_strZinv_rater. rO rJ rL ZfracrM Z
remaining_strrQ rR Zbar_argsZ
l_bar_userZ
r_bar_userZN_BARSZ
bar_lengthZfrac_bar_lengthZbarZfrac_barZfull_barr r r � format_metery s� 3
" i
2
)
# ztqdm.format_meterc O s> t j | � } d | j k r* t � | _ | j j | � | S)N�
_instances)�object�__new__�__dict__r ra �add)�cls�args�kwargs�instancer r r rc s
z tqdm.__new__c sm y( t � f d d � | j D� � d SWn> t k
rh } z d t | � k rS d S� WYd d } ~ Xn Xd S)z Skips specified instance c 3 s$ | ] } | � k r | j Vq d S)N)�pos)�.0�inst)ri r r � <genexpr>, s z%tqdm._get_free_pos.<locals>.<genexpr>r zarg is an empty sequencer N)r? ra �
ValueErrorr )rf ri �er )ri r �
_get_free_pos( s ztqdm._get_free_posc C sb yI | j j | � x2 | j D]' } | j | j k r | j d 8_ q WWn t k
r] Yn Xd S)zs
Remove from list and reposition other bars
so that newer bars won't overlap previous bars
r N)ra �removerj �KeyError)rf ri rl r r r �_decr_instances3 s
ztqdm._decr_instancesr c C s� | } g } xb t | d g � D]N } | j | k sV t d d � | | j f D� � r | j � | j | � q W| j | � | j | � x | D] } | j � q� Wd S)zF
Print a message via tqdm (without overlap with bars)
ra c s s' | ] } | t j t j f k Vq d S)N)�sys�stdout�stderr)rk �fr r r rm N s z tqdm.write.<locals>.<genexpr>N)rB r: �all�clear�appendr9 �refresh)rf r3 rC �endr: Z inst_clearedrl r r r r9 A s
z
tqdm.writec
sI d d l m } d d l m � d d l m } d d l m } d d l m } d d l m } d d l m
} � j d d
� g � d � � � � � f d d
� } | � � _ | � | _ | d � � _
| d � | _
| � | _ | � | _ | d � | _ | � | _ | � | _ | � | _ | d � | _ | d � | _ d
S)a&
Registers the given `tqdm` class with
pandas.core.
( frame.DataFrame
| series.Series
| groupby.DataFrameGroupBy
| groupby.SeriesGroupBy
).progress_apply
A new instance will be create every time `progress_apply` is called,
and each instance will automatically close() upon completion.
Parameters
----------
targs, tkwargs : arguments for the tqdm instance
Examples
--------
>>> import pandas as pd
>>> import numpy as np
>>> from tqdm import tqdm, tqdm_gui
>>>
>>> df = pd.DataFrame(np.random.randint(0, 100, (100000, 6)))
>>> tqdm.pandas(ncols=50) # can use tqdm_gui, optional kwargs, etc
>>> # Now you can use `progress_apply` instead of `apply`
>>> df.groupby(0).progress_apply(lambda x: x**2)
References
----------
https://stackoverflow.com/questions/18603270/
progress-indicator-during-pandas-operations-python
r )� DataFrame)�Series)�DataFrameGroupBy)�
SeriesGroupBy)�GroupBy)� PanelGroupBy)�Panel� deprecated_tNZapplyc s% � � � � � � f d d � } | S)Nc s� t | d d � } | d k rO t | � � r9 t | � n | j t | � } n
| d 7} � d d k r� � d � d � d <n � � d | � � � � � f d d � } t | � � | | | � } � j � | S)aZ
Parameters
----------
df : (DataFrame|Series)[GroupBy]
Data (may be grouped).
func : function
To be applied on the (grouped) data.
*args, *kwargs : optional
Transmitted to `df.apply()`.
ZngroupsNr r rK c s � j � � | | � S)N)�update)rg rh )�funcr2 r r �wrapper� s
zDtqdm.pandas.<locals>.inner_generator.<locals>.inner.<locals>.wrapper)rB �
isinstancer> �size�close)Zdfr� rg rh rK r� �result)r~ r� � df_function�targs�tclass�tkwargs)r� r2 r �inner� s
z3tqdm.pandas.<locals>.inner_generator.<locals>.innerr )r� r� )r~ r� r� r� r� )r� r �inner_generator� s !&z$tqdm.pandas.<locals>.inner_generator�mapZapplymapZ aggregateZ transform)Zpandas.core.framer} Zpandas.core.seriesr~ Zpandas.core.groupbyr r� r� r� �pandasr� �popZprogress_applyZ progress_mapZprogress_applymapZprogress_aggregateZprogress_transform)
r� r� r� r} r r� r� r� r� r� r )r~ r� r� r� r� r r� Z s* "!+ z tqdm.pandasTg�������?g $@g333333�?r c K s� | r> | | _ | | _ | j | � | _ | j j | � d S| r� d | _ | j | � | _ | j j | � d | k r� t d d t | d t j j
� �n t d t | � � � | d k r� | d k r� y t
| � } Wn t t f k
r� d } Yn X| d k r"| t j t j f k s(| rU| rFt � } | | � } n t � | � } | d k rpd } d } n d } | d k r�d } | d k r�d } |
d k r�t | � }
| r�|
r�t | � } | d k r�d } | | _ | r�| d
n d | _ | | _ | | _ | | _ | | _ | | _ | | _ | | _ | | _ |
| _ | | _ | | _ |
| _ | | _ | | _! | | _" d | _# t$ | _% | | _& | | _' | | _( | d k r�| j | � n | | _ | so| j) | j � | _* | j r | j+ | j � | j* | j, | j( | d | r6| | � n | | j |
| |
d | �
� | j ro| j+ | j � | j% � | _- | _. d S) a�
Parameters
----------
iterable : iterable, optional
Iterable to decorate with a progressbar.
Leave blank to manually manage the updates.
desc : str, optional
Prefix for the progressbar.
total : int, optional
The number of expected iterations. If unspecified,
len(iterable) is used if possible. As a last resort, only basic
progress statistics are displayed (no ETA, no progressbar).
If `gui` is True and this parameter needs subsequent updating,
specify an initial arbitrary large positive integer,
e.g. int(9e9).
leave : bool, optional
If [default: True], keeps all traces of the progressbar
upon termination of iteration.
file : `io.TextIOWrapper` or `io.StringIO`, optional
Specifies where to output the progress messages
[default: sys.stderr]. Uses `file.write(str)` and `file.flush()`
methods.
ncols : int, optional
The width of the entire output message. If specified,
dynamically resizes the progressbar to stay within this bound.
If unspecified, attempts to use environment width. The
fallback is a meter width of 10 and no limit for the counter and
statistics. If 0, will not print any meter (only stats).
mininterval : float, optional
Minimum progress update interval, in seconds [default: 0.1].
maxinterval : float, optional
Maximum progress update interval, in seconds [default: 10.0].
miniters : int, optional
Minimum progress update interval, in iterations.
If specified, will set `mininterval` to 0.
ascii : bool, optional
If unspecified or False, use unicode (smooth blocks) to fill
the meter. The fallback is to use ASCII characters `1-9 #`.
disable : bool, optional
Whether to disable the entire progressbar wrapper
[default: False].
unit : str, optional
String that will be used to define the unit of each iteration
[default: it].
unit_scale : bool, optional
If set, the number of iterations will be reduced/scaled
automatically and a metric prefix following the
International System of Units standard will be added
(kilo, mega, etc.) [default: False].
dynamic_ncols : bool, optional
If set, constantly alters `ncols` to the environment (allowing
for window resizes) [default: False].
smoothing : float, optional
Exponential moving average smoothing factor for speed estimates
(ignored in GUI mode). Ranges from 0 (average speed) to 1
(current/instantaneous speed) [default: 0.3].
bar_format : str, optional
Specify a custom bar string formatting. May impact performance.
If unspecified, will use '{l_bar}{bar}{r_bar}', where l_bar is
'{desc}{percentage:3.0f}%|' and r_bar is
'| {n_fmt}/{total_fmt} [{elapsed_str}<{remaining_str}, {rate_fmt}]'
Possible vars: bar, n, n_fmt, total, total_fmt, percentage,
rate, rate_fmt, elapsed, remaining, l_bar, r_bar, desc.
initial : int, optional
The initial counter value. Useful when restarting a progress
bar [default: 0].
position : int, optional
Specify the line offset to print this bar (starting from 0)
Automatic if unspecified.
Useful to manage multiple bars at once (eg, from threads).
gui : bool, optional
WARNING: internal parameter - do not use.
Use tqdm_gui(...) instead. If set, will attempt to use
matplotlib animations for a graphical output [default: False].
Returns
-------
out : decorated iterator.
NT�nestedzO`nested` is deprecated and automated. Use position instead for manual control.
r r9 zUnknown argument(s): r Fz: r" )/�iterable�disablerp rj ra rq r rB rt rv r9 r r r> � TypeError�AttributeErrorru r r r
rS rK �leaver: r[ � mininterval� maxinterval�miniters�dynamic_minitersr] r- r^ �gui�
dynamic_ncols� smoothing�avg_timer �_timer_ � last_print_nrI rD �sp�movetor` �start_t� last_print_t)r r� rS rK r� rC r[ r� r� r� r] r� r- r^ r� r� r_ �initialZpositionr� rh r� r r r r � s� U 0 $
$ z
tqdm.__init__c C sE | j d k r> t | j d � r1 | j j d qD t | j � S| j S)N�shaper )r� �hasattrr� r> rK )r r r r �__len__x s >z tqdm.__len__c C s | S)Nr )r r r r � __enter__} s ztqdm.__enter__c G s | j � d S)NF)r� )r �excr r r �__exit__� s
z
tqdm.__exit__c C s | j � d S)N)r� )r r r r �__del__� s z tqdm.__del__c C s` | j | j | j t � | j | j | j | j | j | j | j
rS d | j
n d | j �
S)Nr ) r` rI rK r r� r[ rS r] r- r^ r� r_ )r r r r �__repr__� s
z
tqdm.__repr__c C s | j | j k S)N)rj )r �otherr r r �__lt__� s z tqdm.__lt__c C s | | k p | | k S)Nr )r r� r r r �__le__� s z tqdm.__le__c C s | j | j k S)N)rj )r r� r r r �__eq__� s z tqdm.__eq__c C s | | k S)Nr )r r� r r r �__ne__� s z tqdm.__ne__c C s | | k S)Nr )r r� r r r �__gt__� s z tqdm.__gt__c C s | | k S)Nr )r r� r r r �__ge__� s z tqdm.__ge__c C s
t | � S)N)�id)r r r r �__hash__� s z
tqdm.__hash__c c s� | j } | j r+ x�| D] } | Vq Wn�| j } | j } | j } | j } | j } | j } | j } | j }
| j
} | j } | j }
| j
} | j } | j } | j } | j } | j } | j } y
| j } Wn9 t k
rt d d t | j d t j j � �� Yn Xx�| D]�} | V| d 7} | |
| k r | � | } | | k r | � } | |
} | | } | r�| r�| d k r�| | n | | | d | | } | j r�| j | j � | | | | j | | r�| | j � n | | j |
| | | rd | n d | �
� | j r=| j | j � | r�| rf| | k rf| | | } nC | r�| r�| | | | d | | } n | | d | | } | | _
| _ }
| | _ } q W|
| _ | | _
| j � d S)z8 Backward-compatibility to use: for x in tqdm(iterable) z<Please use `tqdm_gui(...)` instead of `tqdm(..., gui=True)`
r r9 r N)!r� r� r[ r� r� r� r� r- r^ r] r� r� r� rI r� r� r� r_ r� r` r� r� r rB r: rt rv r9 rj r� rK rS r� )r r� �objr[ r� r� r� r� r- r^ r] r� r� r� rI r� r� r� r_ r� r` r� �delta_t�cur_t�delta_itrP r r r �__iter__� s|
)
! z
tqdm.__iter__r c C s� | j r
d S| d k r. t d j | � � � | j | 7_ | j | j | j k r�| j � | j } | | j k r�| j � } | j | j } | | j } | j
r� | r� | j d k r� | | n | j
| | d | j
| j | _ t | d � s%t
d d t | j d t j j � �� | j r>| j | j � | j | j | j | j | | j rt| j | j � n | j | j | j | j | j | j r�d | j n d | j �
� | j r�| j | j � | j rn| j r
| | j k r
| j | j | | _ na | j rL| rL| j
| | j | d | j
| j | _ n" | j
| d | j
| j | _ | j | _ | | _ d S) at
Manually update the progress bar, useful for streams
such as reading files.
E.g.:
>>> t = tqdm(total=filesize) # Initialise
>>> for current_buffer in stream:
... ...
... t.update(len(current_buffer))
>>> t.close()
The last line is highly recommended, but possibly not necessary if
`t.update()` will be called in such a way that `filesize` will be
exactly reached and printed.
Parameters
----------
n : int
Increment to add to the internal counter of iterations
[default: 1].
Nr zn ({0}) cannot be negativer r� z<Please use `tqdm_gui(...)` instead of `tqdm(..., gui=True)`
r r9 )!r� rn r+ rI r� r� r� r� r� r� r� r� r� r rB r: rt rv r9 rj r� r� r` rK r� r[ rS r] r- r^ r_ r� r� )r rI r� r� r� rP r r r r� sN
$
z tqdm.updatec s� � j r
d Sd � _ � j } � j � � t � d � s? d S� f d d � } y | d � Wn> t k
r� } z d t | � k r� d S� WYd d } ~ Xn X| r� � j | � � j ri� j � j k rE� j
� } � j � j � j � j
| � j � j r� j � j � n � j � j � j � j � j d � j �
� | r\� j | � q�| d � n. � j d � | r�� j | � n
| d � d S)
zE
Cleanup and (if leave=False) close the progressbar.
NTr� c s � j j t | � � d S)N)r: r9 r
)r3 )r r r r h s z tqdm.close.<locals>.fp_writer" �closedr r< )r� rj rs r� rn r r� r� r� rI r� r� r` rK r� r� r: r[ rS r] r- r^ r_ )r rj r ro r� r )r r r� U s@
z
tqdm.closec C s/ | j � } | j | | j 7_ | | _ d S)z:
Restart tqdm timer from last print time.
N)r� r� r� )r r� r r r �unpause� s z tqdm.unpausec C s | r | d n d | _ d S)z=
Set/modify description of the progress bar.
z: r" N)rS )r rS r r r �set_description� s ztqdm.set_descriptionc C s* | j j t d | t � | � � d S)Nr )r: r9 r
r )r rI r r r r� � s z tqdm.movetoc C sw | s | j | j � | j j d � | j j d | j rD | j n d � | j j d � | ss | j | j � d S)z+
Clear current bar display
r< r= rT N)r� rj r: r9 r[ )r �nomover r r ry � s &z
tqdm.clearc C sK | j | j � | j d d � | j j | j � � | j | j � d S)z7
Force refresh the display of this bar
r� TN)r� rj ry r: r9 r� )r r r r r{ � s z tqdm.refresh)'r r r �__doc__� staticmethodr. r6 rD r` rc � classmethodrp rs rt ru r9 r� rv r r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� ry r{ r r r r r
, sT �
g � `R 5
c O s t t | � | � S)zh
A shortcut for tqdm(xrange(*args), **kwargs).
On Python3+ range is used instead of xrange.
)r
r )rg rh r r r r � s )r� Z
__future__r r Z_utilsr r r r r r
r rt r �
__author__�__all__r� r rr r � Exceptionr rb r
r r r r r �<module> s" 4 � � � �