HEX

File: //lib/python3.6/site-packages/ndg/httpsclient/__pycache__/utils.cpython-36.pyc
3

�<�YS=�@s�dZdZdZdZdZdZdZddlZdd	lm	Z	ddl
Z
ddlZejdd
kr�ddl
jZddljZddlmZddlmZdd
lmZddlmZddlmZddlmZddl m!Z"ddl#j$Z%nlddl&Zddl'Zddl(mZddl(mZdd
l(mZddl(mZddl(mZddl(mZddl(m!Z"ddl)Z%ddl*m+Z+ddl,m-Z-ddl.m/Z/ej0e1�Z2Gdd�de�Z3Gdd�de4�Z5d*dd�Z6d+dd�Z7d,dd�Z8d-dd �Z9d.d!d"�Z:d#d$�Z;Gd%d&�d&e<�Z=d'd(�Z>e1d)k�r�ej?�e>�dS)/z`Utilities using NDG HTTPS Client, including a main module that can be used to
fetch from a URL.
z
R B Wilkinsonz09/12/11z2(C) 2011 Science and Technology Facilities Councilz-BSD - see LICENSE file in top-level directoryzPhilip.Kershaw@stfc.ac.ukz$Id$�N)�OptionParser�)�Request)�HTTPHandler)�HTTPCookieProcessor)�HTTPBasicAuthHandler)�HTTPPasswordMgrWithDefaultRealm)�ProxyHandler)�	HTTPError)�build_opener)�HTTPSContextHandler)�ssl_context_utilc@seZdZdZdd�ZeZdS)�AccumulatingHTTPCookieProcessorztCookie processor that adds new cookies (instead of replacing the existing
    ones as HTTPCookieProcessor does)
    cCsxd}t|j�|ji|j|j�}|jj|�|j|�}|rt|j|�rh|j|�}dj	||g�}|j
||�n|j
||�|S)z�Processes cookies for a HTTP request.
        @param request: request to process
        @type request: urllib2.Request
        @return: request
        @rtype: urllib2.Request
        ZCookiez; )�Request_�get_full_url�dataZorigin_req_hostZunverifiable�	cookiejarZadd_cookie_headerZ
get_headerZ
has_header�joinZadd_unredirected_header)�self�requestZCOOKIE_HEADER_NAMEZtmp_requestZnew_cookiesZold_cookiesZmerged_cookies�r�/usr/lib/python3.6/utils.py�http_request3s


z,AccumulatingHTTPCookieProcessor.http_requestN)�__name__�
__module__�__qualname__�__doc__rZ
https_requestrrrrr/src@seZdZdZdS)�
URLFetchErrorzError fetching content from URLN)rrrrrrrrrQsrcCsDt||||d�\}}}|r8|tjkr8|j�}|j�|St|��dS)z�Returns data retrieved from a URL.
    @param url: URL to attempt to open
    @type url: basestring
    @param config: SSL context configuration
    @type config: Configuration
    @return data retrieved from URL or None
    )r�handlersN)�open_url�http_client_�OK�read�closer)�url�configrr�return_code�return_message�response�return_datarrr�fetch_from_urlUsr*c
Cs\t||||d�\}}}|tjkrL|j�}|j�t|d�}	|	j|�|	j�|||tjkfS)a�Writes data retrieved from a URL to a file.
    @param url: URL to attempt to open
    @type url: basestring
    @param config: SSL context configuration
    @type config: Configuration
    @param output_file: output file
    @type output_file: basestring
    @return: tuple (
        returned HTTP status code or 0 if an error occurred
        returned message
        boolean indicating whether access was successful)
    )rr�w)rr r!r"r#�open�write)
r$r%�output_filerrr&r'r(r)Zoutfilerrr�fetch_from_url_to_filefs



r/cCs4t||||d�\}}}|r(|tjkr(|St|��dS)a�Returns data retrieved from a URL.
    @param url: URL to attempt to open
    @type url: basestring
    @param config: SSL context configuration
    @type config: Configuration
    @param data: HTTP POST data
    @type data: str
    @param handlers: list of custom urllib2 handlers to add to the request
    @type handlers: iterable
    @return: data retrieved from URL or None
    @rtype: file derived type
    )rrN)rr r!r)r$r%rrr&r'r(rrr�fetch_stream_from_urls

r0cCs*|jr
dnd}|jr|j}ntj�}t|�}|s4g}|j|�|jrjt|d�}t|j|d�}|j	||g�|j
r�tt��}	|	j
d||j
d|j
dd�|j|	�t||j�s�|jti��tjd�n&|jr�|jt|j��tjd|j�t|d|ji�}
|j}|dk�ri}t|||�}d}
d	}d}yN|
j|�}|j}|j}
tjtj��rvx$t|�D]\}}tjd
||��qZWWn�tk
�r�}z6|j}
d|j}tjtj��r�tjd|j|j�WYdd}~XnTtk
�r}z6d|j�}tjtj��rddl }tj|j!��WYdd}~XnX|
||fS)
a�Attempts to open a connection to a specified URL.
    @param url: URL to attempt to open
    @param config: SSL context configuration
    @type config: Configuration
    @param data: HTTP POST data
    @type data: str
    @param handlers: list of custom urllib2 handlers to add to the request
    @type handlers: iterable
    @return: tuple (
        returned HTTP status code or 0 if an error occurred
        returned message or error description
        response object)
    �r)�
debuglevelN)ZrealmZuri�userZpasswdzNot using proxyzConfiguring proxies: %s�ssl_context�z	%s  :  %sz	Error: %sz%s %s)"�debug�cookie�
cookiejar_Z	CookieJarr�append�HTTPHandler_rr4�extend�http_basicauth�HTTPBasicAuthHandler_� HTTPPasswordMgrWithDefaultRealm_Zadd_password�_should_use_proxy�no_proxy�
ProxyHandler_�log�proxiesr�headersrr,�msg�codeZisEnabledFor�logging�DEBUG�	enumerate�
HTTPError_�	Exception�__str__�	traceback�
format_exc)r$r%rrr2ZcjZcookie_handlerZhttp_handlerZ
https_handlerZauth_handler�openerrDrr&r'r(�indexr7�excrMrrrr�sf






" rcCsZ|dkrtjjdd�}n|}tjt|��}x*dd�|jd�D�D]}|j|kr@dSq@WdS)	z�Determines whether a proxy should be used to open a connection to the 
    specified URL, based on the value of the no_proxy environment variable.
    @param url: URL
    @type url: basestring or urllib2.Request
    Nr@r5cSsg|]}|j��qSr)�strip)�.0�hrrr�
<listcomp>�sz%_should_use_proxy.<locals>.<listcomp>�,FT)�os�environ�get�	urlparse_�urlparse�_url_as_string�splitZhostname)r$r@Zno_proxy_effectiveZurlObjZnprrrr?�s
r?cCs4t|t�r|j�St|t�r |Stdttf��dS)z�Returns the URL string from a URL value that is either a string or
    urllib2.Request..
    @param url: URL
    @type url: basestring or urllib2.Request
    @return: URL string
    @rtype: basestring
    zExpected type %r or %rN)�
isinstancerr�str�	TypeError)r$rrrr\s

r\c@seZdZdZddd�ZdS)�
ConfigurationzConnection configuration.
    FNcCs.||_||_||_||_||_||_||_dS)a�
        @param ssl_context: SSL context to use with this configuration
        @type ssl_context: OpenSSL.SSL.Context
        @param debug: if True, output debugging information
        @type debug: bool
        @param proxies: proxies to use for 
        @type proxies: dict with basestring keys and values
        @param no_proxy: hosts for which a proxy should not be used
        @type no_proxy: basestring
        @param cookie: cookies to set for request
        @type cookie: cookielib.CookieJar (python 3 - http.cookiejar)
        @param http_basicauth: http authentication, or None
        @type http_basicauth: tuple of (username,password)
        @param headers: http headers
        @type headers: dict
        N)r4r6rCr@r7r<rD)rr4r6rCr@r7r<rDrrr�__init__szConfiguration.__init__)FNNNNN)rrrrrbrrrrrasracCs�tdd�}|jddddtjjd�dd	�|jd
dddd
dd	�|jddddd
dd	�|jddddddd�|jddddd
dd	�|jdd d!dd
d"d	�|jd#d$dd%dd&d�|jd'd(d)d*d
d+d	�|jd,d-d.d/d0d1�|j�\}}t|�d2kr�|jd3�|d4}|j�rt	j
�jt	j�|j
�r8tjj|j
��r8|j
}nd
}|j�r\tjj|j��r\|j}nd
}|j�r�tjj|j��r�|j}nd
}|j}|j�r�tjj|j��r�t|j�}|j�}	|j�nd
}	|j�r�|jjd5d2�}
nd
}
i}|j�rx0|jD]&}|jd5d2�\}
}|j�||
j�<�q�Wtj||d
|||�}t||j|
|d6�}|j�rtt|||j|	�d
d7�\}}t ||��nt!||�}	t"|	�d
S)8zHUtility to fetch data using HTTP or HTTPS GET from a specified URL.
    z%prog [options] url)Zusagez-cz
--certificate�	cert_file�FILEz~/credentials.pemz4Certificate file - defaults to $HOME/credentials.pem)�dest�metavar�default�helpz-kz
--private-key�key_fileNz3Private key file - defaults to the certificate filez-tz--ca-certificate-dir�ca_dir�PATHz%Trusted CA certificate file directoryz-dz--debug�
store_truer6FzPrint debug information.)�actionrergrhz-pz--post-data-file�	data_filezPOST data filez-fz--fetchr.zOutput filez-nz--no-verify-peer�no_verify_peerz&Skip verification of peer certificate.z-az--basicauth�	basicauthzUSER:PASSWDzHTTP authentication credentialsz--headerr9rDz
HEADER: VALUEzAdd HTTP header to request)rmrerfrhr1zIncorrect number of argumentsr�:)r<rDr)#rZ
add_optionrW�path�
expanduser�
parse_args�len�errorr6rG�	getLoggerZsetLevelrHri�existsrcrjrornr,r"r#rpr]rD�lstriprRr
Zmake_ssl_contextrar.r/�
SystemExitr*�print)�parserZoptions�argsr$rircrjZverify_peerrnrr<rDrT�key�valr4r%r&r'rrr�main4s�











r��__main__)NN)NN)NN)NN)N)@r�
__author__Z__date__Z
__copyright__Z__license__Z__contact__Z__revision__rGZoptparserrW�sys�version_infoZhttp.cookiejarrr8Zhttp.clientZclientr Zurllib.requestrrrr:rZHTTPCookieProcessor_rr=rr>r	rAZurllib.errorr
rJZurllib.parse�parserZZ	cookielibZhttplibZurllib2r[Z$ndg.httpsclient.urllib2_build_openerrZndg.httpsclient.httpsrZndg.httpsclientr
rwrrBrrKrr*r/r0rr?r\�objectrar�ZbasicConfigrrrr�<module>s`


"



_
e