Edit File: poolmanager.cpython-36.pyc
3 nf�\YB � @ s d dl mZ d dlZd dlZd dlZddlmZ ddlmZm Z ddlm Z ddl m Z m Z mZ ddlmZ dd lmZ dd lmZ dd lmZ dd lmZ d ddgZeje�Z d6Z d7Z ej d/e �Z d0d1� Z!ej"e!e �ej"e!e �d2�Z#ee d2�Z$G d3d � d e�Z%G d4d� de%�Z&d5d� Z'dS )8� )�absolute_importN� )�RecentlyUsedContainer)�HTTPConnectionPool�HTTPSConnectionPool)�port_by_scheme)�LocationValueError� MaxRetryError�ProxySchemeUnknown)�six)�urljoin)�RequestMethods)� parse_url)�Retry� PoolManager� ProxyManager�proxy_from_url�key_file� cert_file� cert_reqs�ca_certs� ssl_version� ca_cert_dir� ssl_context� key_scheme�key_host�key_port� key_timeout� key_retries� key_strict� key_block�key_source_address� key_key_file� key_cert_file� key_cert_reqs� key_ca_certs�key_ssl_version�key_ca_cert_dir�key_ssl_context� key_maxsize� key_headers� key__proxy�key__proxy_headers�key_socket_options�key__socks_options�key_assert_hostname�key_assert_fingerprint�key_server_hostname�PoolKeyc C s� |j � }|d j� |d< |d j� |d< x4d D ],}||kr.|| dk r.t|| j� �||< q.W |jd�}|dk r|t|�|d< x&t|j� �D ]}|j|�|d| < q�W x | j D ]}||kr�d||< q�W | f |�S ) a� Create a pool key out of a request context dictionary. According to RFC 3986, both the scheme and host are case-insensitive. Therefore, this function normalizes both before constructing the pool key for an HTTPS request. If you wish to change this behaviour, provide alternate callables to ``key_fn_by_scheme``. :param key_class: The class to use when constructing the key. This should be a namedtuple with the ``scheme`` and ``host`` keys at a minimum. :type key_class: namedtuple :param request_context: A dictionary-like object that contain the context for a request. :type request_context: dict :return: A namedtuple that can be used as a connection pool key. :rtype: PoolKey �scheme�host�headers�_proxy_headers�_socks_optionsNZsocket_optionsZkey_)r5 r6 r7 ) �copy�lower� frozenset�items�get�tuple�list�keys�pop�_fields)Z key_class�request_context�context�keyZ socket_optsZfield� rE �!/usr/lib/python3.6/poolmanager.py�_default_key_normalizer; s rG )�http�httpsc @ sx e Zd ZdZdZd dd�Zdd� Zdd � Zd d d �Zd d � Z d dd�Z dd� Z d dd�Z d dd�Z dd� Zd!dd�ZdS )"r a$ Allows for arbitrary requests while transparently keeping track of necessary connection pools for you. :param num_pools: Number of connection pools to cache before discarding the least recently used pool. :param headers: Headers to include with all requests, unless other headers are given explicitly. :param \**connection_pool_kw: Additional parameters are used to create fresh :class:`urllib3.connectionpool.ConnectionPool` instances. Example:: >>> manager = PoolManager(num_pools=2) >>> r = manager.request('GET', 'http://google.com/') >>> r = manager.request('GET', 'http://google.com/mail') >>> r = manager.request('GET', 'http://yahoo.com/') >>> len(manager.pools) 2 N� c K s8 t j| |� || _t|dd� d�| _t| _tj� | _d S )Nc S s | j � S )N)�close)�prE rE rF �<lambda>� s z&PoolManager.__init__.<locals>.<lambda>)Z dispose_func)r �__init__�connection_pool_kwr �pools�pool_classes_by_scheme�key_fn_by_schemer8 )�self� num_poolsr5 rO rE rE rF rN � s zPoolManager.__init__c C s | S )NrE )rS rE rE rF � __enter__� s zPoolManager.__enter__c C s | j � dS )NF)�clear)rS �exc_typeZexc_valZexc_tbrE rE rF �__exit__� s zPoolManager.__exit__c C sf | j | }|dkr | jj� }xdD ]}|j|d� q"W |dkrXxtD ]}|j|d� qDW |||f|�S )a� Create a new :class:`ConnectionPool` based on host, port, scheme, and any additional pool keyword arguments. If ``request_context`` is provided, it is provided as keyword arguments to the pool class used. This method is used to actually create the connection pools handed out by :meth:`connection_from_url` and companion methods. It is intended to be overridden for customization. Nr3 r4 �portrH )r3 r4 rY )rQ rO r8 r@ � SSL_KEYWORDS)rS r3 r4 rY rB Zpool_clsrD �kwrE rE rF � _new_pool� s zPoolManager._new_poolc C s | j j� dS )z� Empty our store of pools and direct them all to close. This will not affect in-flight connections, but they will not be re-used after completion. N)rP rV )rS rE rE rF rV � s zPoolManager.clearrH c C sT |s t d��| j|�}|p d|d< |s:tj|d j� d�}||d< ||d< | j|�S )a� Get a :class:`ConnectionPool` based on the host, port, and scheme. If ``port`` isn't given, it will be derived from the ``scheme`` using ``urllib3.connectionpool.port_by_scheme``. If ``pool_kwargs`` is provided, it is merged with the instance's ``connection_pool_kw`` variable and used to create the new connection pool, if one is needed. zNo host specified.rH r3 �P rY r4 )r �_merge_pool_kwargsr r< r9 �connection_from_context)rS r4 rY r3 � pool_kwargsrB rE rE rF �connection_from_host� s z PoolManager.connection_from_hostc C s, |d j � }| j| }||�}| j||d�S )z� Get a :class:`ConnectionPool` based on the request context. ``request_context`` must at least contain the ``scheme`` key and its value must be a key in ``key_fn_by_scheme`` instance variable. r3 )rB )r9 rR �connection_from_pool_key)rS rB r3 Zpool_key_constructor�pool_keyrE rE rF r_ � s z#PoolManager.connection_from_contextc C s` | j j�N | j j|�}|r |S |d }|d }|d }| j||||d�}|| j |< W dQ R X |S )z� Get a :class:`ConnectionPool` based on the provided pool key. ``pool_key`` should be a namedtuple that only contains immutable objects. At a minimum it must have the ``scheme``, ``host``, and ``port`` fields. r3 r4 rY )rB N)rP �lockr< r\ )rS rc rB Zpoolr3 r4 rY rE rE rF rb � s z$PoolManager.connection_from_pool_keyc C s t |�}| j|j|j|j|d�S )a� Similar to :func:`urllib3.connectionpool.connection_from_url`. If ``pool_kwargs`` is not provided and a new pool needs to be constructed, ``self.connection_pool_kw`` is used to initialize the :class:`urllib3.connectionpool.ConnectionPool`. If ``pool_kwargs`` is provided, it is used instead. Note that if a new pool does not need to be created for the request, the provided ``pool_kwargs`` are not used. )rY r3 r` )r ra r4 rY r3 )rS �urlr` �urE rE rF �connection_from_url s z PoolManager.connection_from_urlc C sZ | j j� }|rVxF|j� D ]:\}}|dkrJy ||= W qR tk rF Y qRX q|||< qW |S )a Merge a dictionary of override values for self.connection_pool_kw. This does not modify self.connection_pool_kw and returns a new dict. Any keys in the override dictionary with a value of ``None`` are removed from the merged dictionary. N)rO r8 r; �KeyError)rS �overrideZbase_pool_kwargsrD �valuerE rE rF r^ s z PoolManager._merge_pool_kwargsTc K s� t |�}| j|j|j|jd�}d|d< d|d< d|krD| jj� |d< | jdk rj|jdkrj|j||f|�}n|j||j f|�}|o�|j � }|s�|S t ||�}|j dkr�d }|j d �} t| t�s�tj| |d �} | jo�|j|� �r ttj|d ��} x*| D ]"} | j� | jkr�|d j| d� q�W y| j||||d �} W n tk �rR | j�rN� |S X | |d < ||d< tj d ||� | j||f|�S )a] Same as :meth:`urllib3.connectionpool.HTTPConnectionPool.urlopen` with custom cross-host redirect logic and only sends the request-uri portion of the ``url``. The given ``url`` parameter must be absolute, such that an appropriate :class:`urllib3.connectionpool.ConnectionPool` can be chosen for it. )rY r3 FZassert_same_host�redirectr5 NrH i/ ZGET�retries)rk )�responseZ_poolzRedirecting %s -> %s) r ra r4 rY r3 r5 r8 �proxy�urlopenZ request_uriZget_redirect_locationr Zstatusr< � isinstancer Zfrom_intZremove_headers_on_redirectZ is_same_hostr> r Ziterkeysr9 r@ Z incrementr Zraise_on_redirect�log�info) rS �methodre rk r[ rf Zconnrm Zredirect_locationrl r5 �headerrE rE rF ro / sD zPoolManager.urlopen)rJ N)N)NrH N)N)N)T)�__name__� __module__� __qualname__�__doc__rn rN rU rX r\ rV ra r_ rb rg r^ ro rE rE rE rF r { s c sH e Zd ZdZd� fdd� Zd� fdd� Zdd d �Zd� fd d � Z� ZS )r ax Behaves just like :class:`PoolManager`, but sends all requests through the defined proxy, using the CONNECT method for HTTPS URLs. :param proxy_url: The URL of the proxy to be used. :param proxy_headers: A dictionary containing headers that will be sent to the proxy. In case of HTTP they are being sent with each request, while in the HTTPS/CONNECT case they are sent only once. Could be used for proxy authentication. Example: >>> proxy = urllib3.ProxyManager('http://localhost:3128/') >>> r1 = proxy.request('GET', 'http://google.com/') >>> r2 = proxy.request('GET', 'http://httpbin.org/') >>> len(proxy.pools) 1 >>> r3 = proxy.request('GET', 'https://httpbin.org/') >>> r4 = proxy.request('GET', 'https://twitter.com/') >>> len(proxy.pools) 3 rJ Nc s� t |t�r d|j|j|jf }t|�}|jsFtj|jd�}|j|d�}|jdkrZt |j��|| _ |pfi | _ | j |d< | j |d< t t | �j||f|� d S ) Nz %s://%s:%ir] )rY rH rI �_proxyr6 )rH rI )rp r r3 r4 rY r r r< �_replacer rn � proxy_headers�superr rN )rS � proxy_urlrT r5 r{ rO rn rY )� __class__rE rF rN � s zProxyManager.__init__rH c sD |dkr t t| �j||||d�S t t| �j| jj| jj| jj|d�S )NrI )r` )r| r ra rn r4 rY r3 )rS r4 rY r3 r` )r~ rE rF ra � s z!ProxyManager.connection_from_hostc C s0 ddi}t |�j}|r ||d< |r,|j|� |S )z� Sets headers needed by proxies: specifically, the Accept and Host headers. Only sets headers not provided by the user. ZAcceptz*/*ZHost)r �netloc�update)rS re r5 Zheaders_r rE rE rF �_set_proxy_headers� s z ProxyManager._set_proxy_headersTc sN t |�}|jdkr0|jd| j�}| j||�|d< tt| �j||fd|i|��S )z@Same as HTTP(S)ConnectionPool.urlopen, ``url`` must be absolute.rH r5 rk )r r3 r< r5 r� r| r ro )rS rs re rk r[ rf r5 )r~ rE rF ro � s zProxyManager.urlopen)rJ NN)NrH N)N)T) ru rv rw rx rN ra r� ro � __classcell__rE rE )r~ rF r m s c K s t f d| i|��S )Nr} )r )re r[ rE rE rF r � s )r r r r r r r )r r r r r r r r! r" r# r$ r% r& r' r( r) r* r+ r, r- r. r/ r0 r1 )(Z __future__r � collections� functoolsZlogging� _collectionsr Zconnectionpoolr r r � exceptionsr r r Zpackagesr Z packages.six.moves.urllib.parser Zrequestr Zutil.urlr Z util.retryr �__all__Z getLoggerru rq rZ Z _key_fields� namedtupler2 rG �partialrR rQ r r r rE rE rE rF �<module> sd 6 sW