File: //usr/local/lib/python3.7/site-packages/urllib3/__pycache__/poolmanager.cpython-37.pyc
B
���gfY � @ s� d dl mZ d dlZd dlZd dlZd dlZd dlmZ d dlm Z ddl
m Z m Z ddl
mZ ddlmZ dd lmZmZmZ dd
lmZmZmZmZ dd lmZ dd l m Z dd
l m Z ddl m!Z! ddl"m#Z# ddl$m%Z%m&Z& ej'r�d dl(Z(d dl)m*Z* dddgZ+e�,e-�Z.dZ/dZ0e�1d�Z2G dd� dej3�Z4dddd �d d �Z5e�6e5e4�e�6e5e4�d �Z7eed �Z8G d d� de�Z9G d!d� de9�Z:d"d#dd$�d%d�Z;dS )&� )� annotationsN)�
TracebackType)�urljoin� )�HTTPHeaderDict�RecentlyUsedContainer)�RequestMethods)� ProxyConfig)�HTTPConnectionPool�HTTPSConnectionPool�port_by_scheme)�LocationValueError�
MaxRetryError�ProxySchemeUnknown�URLSchemeUnknown)�BaseHTTPResponse)�_TYPE_SOCKET_OPTIONS)� connection_requires_http_tunnel)�Retry)�Timeout)�Url� parse_url)�Literal� PoolManager� ProxyManager�proxy_from_url) �key_file� cert_file� cert_reqs�ca_certs� ssl_versionZssl_minimum_versionZssl_maximum_version� ca_cert_dir� ssl_context� key_password�server_hostnamei @ �_SelfTc @ s� e Zd ZU dZded< ded< ded< ded< d ed
< d ed < d
ed< ded< ded< ded< ded< ded< ded< ded< ded< ded< ded < ded < d ed < d ed!< d ed"< d#ed$< d%ed&< d ed'< d(ed)< ded*< ded+< ded,< d-S ).�PoolKeyz�
All known keyword arguments that could be provided to the pool manager, its
pools, or the underlying connections.
All custom key schemes should include the fields in this key at a minimum.
�str�
key_scheme�key_hostz
int | None�key_portz Timeout | float | int | None� key_timeoutzRetry | bool | int | None� key_retriesz bool | None� key_blockztuple[str, int] | None�key_source_addressz
str | None� key_key_file�key_key_password�
key_cert_file�
key_cert_reqs� key_ca_certszint | str | None�key_ssl_versionzssl.TLSVersion | NoneZkey_ssl_minimum_versionZkey_ssl_maximum_version�key_ca_cert_dirzssl.SSLContext | None�key_ssl_context� key_maxsizez!frozenset[tuple[str, str]] | None� key_headersz
Url | None�
key__proxy�key__proxy_headerszProxyConfig | None�key__proxy_configz_TYPE_SOCKET_OPTIONS | None�key_socket_options�key__socks_optionszbool | str | None�key_assert_hostname�key_assert_fingerprint�key_server_hostname�
key_blocksizeN)�__name__�
__module__� __qualname__�__doc__�__annotations__� rG rG �=/usr/local/lib/python3.7/site-packages/urllib3/poolmanager.pyr& 9 s:
r& z
type[PoolKey]zdict[str, typing.Any])� key_class�request_context�returnc C s� |� � }|d �� |d< |d �� |d< x4dD ],}||kr.|| dk r.t|| �� �||< q.W |�d�}|dk r|t|�|d< x&t|�� �D ]}|�|�|d| < q�W x | j D ]}||kr�d||< q�W |�d�dkr�t
|d< | f |�S )a�
Create a pool key out of a request context dictionary.
According to RFC 3986, both the scheme and host are case-insensitive.
Therefore, this function normalizes both before constructing the pool
key for an HTTPS request. If you wish to change this behaviour, provide
alternate callables to ``key_fn_by_scheme``.
:param key_class:
The class to use when constructing the key. This should be a namedtuple
with the ``scheme`` and ``host`` keys at a minimum.
:type key_class: namedtuple
:param request_context:
A dictionary-like object that contain the context for a request.
:type request_context: dict
:return: A namedtuple that can be used as a connection pool key.
:rtype: PoolKey
�scheme�host)�headers�_proxy_headers�_socks_optionsN�socket_options�key_rA ) �copy�lower� frozenset�items�get�tuple�list�keys�pop�_fields�_DEFAULT_BLOCKSIZE)rI rJ �context�key� socket_opts�fieldrG rG rH �_default_key_normalizer_ s"
rb )�http�httpsc s e Zd ZU dZdZded< dZded< dAdd d
d d �� fd
d�
Zddd�dd�Zddddd�dd�Z dBddddd d �d d �Z
d d �d!d"�Z dCd$d%d$dd d&�d'd(�Z d)d d*�d+d,�Z
d-d)d d.�d/d0�ZdDddd d1�d2d3�Zdd)d4�d5d6�Zd7d8d9�d:d;�ZdEddd8d
d=d>�d?d@�Z� ZS )Fr aV
Allows for arbitrary requests while transparently keeping track of
necessary connection pools for you.
:param num_pools:
Number of connection pools to cache before discarding the least
recently used pool.
:param headers:
Headers to include with all requests, unless other headers are given
explicitly.
:param \**connection_pool_kw:
Additional parameters are used to create fresh
:class:`urllib3.connectionpool.ConnectionPool` instances.
Example:
.. code-block:: python
import urllib3
http = urllib3.PoolManager(num_pools=2)
resp1 = http.request("GET", "https://google.com/")
resp2 = http.request("GET", "https://google.com/mail")
resp3 = http.request("GET", "https://yahoo.com/")
print(len(http.pools))
# 2
Nz
Url | None�proxyzProxyConfig | None� proxy_config�
�intz typing.Mapping[str, str] | Nonez
typing.Any�None)� num_poolsrN �connection_pool_kwrK c s4 t � �|� || _| t|�| _t| _t�� | _d S )N)�super�__init__rk r �pools�pool_classes_by_scheme�key_fn_by_schemerS )�selfrj rN rk )� __class__rG rH rm � s
zPoolManager.__init__r% )rq rK c C s | S )NrG )rq rG rG rH � __enter__� s zPoolManager.__enter__ztype[BaseException] | NonezBaseException | NonezTracebackType | NonezLiteral[False])�exc_type�exc_val�exc_tbrK c C s | � � dS )NF)�clear)rq rt ru rv rG rG rH �__exit__� s zPoolManager.__exit__r' z dict[str, typing.Any] | Noner
)rL rM �portrJ rK c C s| | j | }|dkr | j�� }|�d�dkr2t|d< xdD ]}|�|d� q8W |dkrnxtD ]}|�|d� qZW |||f|�S )a�
Create a new :class:`urllib3.connectionpool.ConnectionPool` based on host, port, scheme, and
any additional pool keyword arguments.
If ``request_context`` is provided, it is provided as keyword arguments
to the pool class used. This method is used to actually create the
connection pools handed out by :meth:`connection_from_url` and
companion methods. It is intended to be overridden for customization.
N� blocksize)rL rM ry rc )ro rk rS rW r] r[ � SSL_KEYWORDS)rq rL rM ry rJ �pool_clsr_ �kwrG rG rH � _new_pool� s
zPoolManager._new_pool)rK c C s | j �� dS )z�
Empty our store of pools and direct them all to close.
This will not affect in-flight connections, but they will not be
re-used after completion.
N)rn rw )rq rG rG rH rw s zPoolManager.clearrc z
str | Nonez
int | None)rM ry rL � pool_kwargsrK c C sT |s t d��| �|�}|p d|d< |s:t�|d �� d�}||d< ||d< | �|�S )a�
Get a :class:`urllib3.connectionpool.ConnectionPool` based on the host, port, and scheme.
If ``port`` isn't given, it will be derived from the ``scheme`` using
``urllib3.connectionpool.port_by_scheme``. If ``pool_kwargs`` is
provided, it is merged with the instance's ``connection_pool_kw``
variable and used to create the new connection pool, if one is
needed.
zNo host specified.rc rL �P ry rM )r
�_merge_pool_kwargsr rW rT �connection_from_context)rq rM ry rL r rJ rG rG rH �connection_from_host s
z PoolManager.connection_from_hostzdict[str, typing.Any])rJ rK c C sX d|kr t �dt� |�d� |d �� }| j�|�}|sBt|��||�}| j||d�S )z�
Get a :class:`urllib3.connectionpool.ConnectionPool` based on the request context.
``request_context`` must at least contain the ``scheme`` key and its
value must be a key in ``key_fn_by_scheme`` instance variable.
�strictzdThe 'strict' parameter is no longer needed on Python 3+. This will raise an error in urllib3 v2.1.0.rL )rJ ) �warnings�warn�DeprecationWarningr[ rT rp rW r �connection_from_pool_key)rq rJ rL �pool_key_constructor�pool_keyrG rG rH r� 1 s
z#PoolManager.connection_from_contextr& )r� rJ rK c C s` | j j�N | j �|�}|r |S |d }|d }|d }| j||||d�}|| j |< W dQ R X |S )a
Get a :class:`urllib3.connectionpool.ConnectionPool` based on the provided pool key.
``pool_key`` should be a namedtuple that only contains immutable
objects. At a minimum it must have the ``scheme``, ``host``, and
``port`` fields.
rL rM ry )rJ N)rn �lockrW r~ )rq r� rJ �poolrL rM ry rG rG rH r� J s
z$PoolManager.connection_from_pool_key)�urlr rK c C s t |�}| j|j|j|j|d�S )a�
Similar to :func:`urllib3.connectionpool.connection_from_url`.
If ``pool_kwargs`` is not provided and a new pool needs to be
constructed, ``self.connection_pool_kw`` is used to initialize
the :class:`urllib3.connectionpool.ConnectionPool`. If ``pool_kwargs``
is provided, it is used instead. Note that if a new pool does not
need to be created for the request, the provided ``pool_kwargs`` are
not used.
)ry rL r )r r� rM ry rL )rq r� r �urG rG rH �connection_from_urld s
z PoolManager.connection_from_url)�overriderK c C sZ | j �� }|rVxF|�� D ]:\}}|dkrJy
||= W qR tk
rF Y qRX q|||< qW |S )a
Merge a dictionary of override values for self.connection_pool_kw.
This does not modify self.connection_pool_kw and returns a new dict.
Any keys in the override dictionary with a value of ``None`` are
removed from the merged dictionary.
N)rk rS rV �KeyError)rq r� �base_pool_kwargsr_ �valuerG rG rH r� v s
z PoolManager._merge_pool_kwargsr �bool)�
parsed_urlrK c C s" | j dkrdS t| j | j|j� S )z�
Indicates if the proxy requires the complete destination URL in the
request. Normally this is only needed when not using an HTTP CONNECT
tunnel.
NF)re r rf rL )rq r� rG rG rH �!_proxy_requires_url_absolute_form� s
z-PoolManager._proxy_requires_url_absolute_formTr )�methodr� �redirectr} rK c K s� t |�}|jdkr"tjdtdd� | j|j|j|jd�}d|d< d|d< d |krZ| j|d < | � |�rv|j
||f|�}n|j
||j f|�}|o�|� � }|s�|S t
||�}|jd
kr�d }d|d < t|d ��� |d < |�d
�} t| t�s�tj| |d�} | j�rJ|�|��sJ|d �� }
x.|d D ]"} | �� | jk�r |
�| d� �q W |
|d < y| j||||d�} W n( tk
�r� | j �r�|� � � |S X | |d
< ||d< t � d||� |� � | j
||f|�S )aN
Same as :meth:`urllib3.HTTPConnectionPool.urlopen`
with custom cross-host redirect logic and only sends the request-uri
portion of the ``url``.
The given ``url`` parameter must be absolute, such that an appropriate
:class:`urllib3.connectionpool.ConnectionPool` can be chosen for it.
Na URLs without a scheme (ie 'https://') are deprecated and will raise an error in a future version of urllib3. To avoid this DeprecationWarning ensure all URLs start with 'https://' or 'http://'. Read more in this issue: https://github.com/urllib3/urllib3/issues/2920� )�category�
stacklevel)ry rL F�assert_same_hostr� rN i/ �GET�body�retries)r� )�response�_poolzRedirecting %s -> %s) r rL r� r� r� r� rM ry rN r� �urlopen� request_uri�get_redirect_locationr �statusr Z_prepare_for_method_changerW �
isinstancer �from_int�remove_headers_on_redirect� is_same_hostrS rT r[ � incrementr �raise_on_redirect�
drain_conn�log�info) rq r� r� r� r} r� �connr� �redirect_locationr� � new_headers�headerrG rG rH r� � sX
zPoolManager.urlopen)rg N)N)Nrc N)N)T)rB rC rD rE re rF rf rm rs rx r~ rw r� r� r� r� r� r� r� �
__classcell__rG rG )rr rH r � s$
! c s� e Zd ZdZd!dddddd d
d d d
d�
� fdd�
Zd"d dd ddd�� fdd�
Zd#dddd�dd�Zd$ddd d d d �� fd d �
Z� ZS )%r a>
Behaves just like :class:`PoolManager`, but sends all requests through
the defined proxy, using the CONNECT method for HTTPS URLs.
:param proxy_url:
The URL of the proxy to be used.
:param proxy_headers:
A dictionary containing headers that will be sent to the proxy. In case
of HTTP they are being sent with each request, while in the
HTTPS/CONNECT case they are sent only once. Could be used for proxy
authentication.
:param proxy_ssl_context:
The proxy SSL context is used to establish the TLS connection to the
proxy when using HTTPS proxies.
:param use_forwarding_for_https:
(Defaults to False) If set to True will forward requests to the HTTPS
proxy to be made on behalf of the client instead of creating a TLS
tunnel via the CONNECT method. **Enabling this flag means that request
and response headers and content will be visible from the HTTPS proxy**
whereas tunneling keeps request and response headers and content
private. IP address, target hostname, SNI, and port are always visible
to an HTTPS proxy even when this flag is disabled.
:param proxy_assert_hostname:
The hostname of the certificate to verify against.
:param proxy_assert_fingerprint:
The fingerprint of the certificate to verify against.
Example:
.. code-block:: python
import urllib3
proxy = urllib3.ProxyManager("https://localhost:3128/")
resp1 = proxy.request("GET", "https://google.com/")
resp2 = proxy.request("GET", "https://httpbin.org/")
print(len(proxy.pools))
# 1
resp3 = proxy.request("GET", "https://httpbin.org/")
resp4 = proxy.request("GET", "https://twitter.com/")
print(len(proxy.pools))
# 3
rg NFr' rh z typing.Mapping[str, str] | Nonezssl.SSLContext | Noner� zNone | str | Literal[False]z
str | Nonez
typing.Anyri )
� proxy_urlrj rN �
proxy_headers�proxy_ssl_context�use_forwarding_for_https�proxy_assert_hostname�proxy_assert_fingerprintrk rK c
s� t |t�r&|j� d|j� d|j� �}
n|}
t|
�} | jdkrFt| j��| jsft�| jd�} | j | d�} | | _
|pri | _ || _ t
||||�| _| j
| d< | j | d< | j| d< t� j||f| � d S ) Nz://�:)rc rd r� )ry �_proxyrO �
_proxy_config)r� r
rL rM ry r r r rW �_replacere r� r� r rf rl rm )
rq r� rj rN r� r� r� r� r� rk Z
str_proxy_urlre ry )rr rG rH rm # s*
zProxyManager.__init__rc z
int | Nonez dict[str, typing.Any] | Noner
)rM ry rL r rK c s< |dkr t � j||||d�S t � j| jj| jj| jj|d�S )Nrd )r )rl r� re rM ry rL )rq rM ry rL r )rr rG rH r� L s
z!ProxyManager.connection_from_hostztyping.Mapping[str, str])r� rN rK c C s0 ddi}t |�j}|r ||d< |r,|�|� |S )z�
Sets headers needed by proxies: specifically, the Accept and Host
headers. Only sets headers not provided by the user.
�Acceptz*/*�Host)r �netloc�update)rq r� rN �headers_r� rG rG rH �_set_proxy_headers\ s
z ProxyManager._set_proxy_headersTr )r� r� r� r} rK c sR t |�}t| j| j|j�s8|�d| j�}| �||�|d< t� j ||fd|i|��S )z@Same as HTTP(S)ConnectionPool.urlopen, ``url`` must be absolute.rN r� )
r r re rf rL rW rN r� rl r� )rq r� r� r� r} r� rN )rr rG rH r� m s
zProxyManager.urlopen)rg NNNFNN)Nrc N)N)T) rB rC rD rE rm r� r� r� r� rG rG )rr rH r � s 5 &# r' z
typing.Any)r� r} rK c K s t f d| i|��S )Nr� )r )r� r} rG rG rH r | s )<�
__future__r � functools�logging�typingr� �typesr � urllib.parser � _collectionsr r Z_request_methodsr �
connectionr �connectionpoolr
r r �
exceptionsr
r r r r� r Zutil.connectionr Z
util.proxyr Z
util.retryr Z util.timeoutr Zutil.urlr r �
TYPE_CHECKING�ssl�typing_extensionsr �__all__� getLoggerrB r� r{ r] �TypeVarr% �
NamedTupler& rb �partialrp ro r r r rG rG rG rH �<module> sH
&<
L