3

mk�h�p�@s�dd
lZdd
l
Z
dd
lZddlmZ
ddlmZ
ddlmZm	Z	
ydd
l
Z
dZWnek
rh


dZYnXydd
l
Z
dZWnek
r�


dZYnXGdd�d�Zd
S)	�N)
�getpass)
�_sos)�
is_executable�TIMEOUT_DEFAULTTFc@s�
eZ
dZd
ZdZdZdZdZdZdZ	dZ
dZdZdZ
dZdZdZdZdZdZdZdZdZdZdZdZdZdZdZddd	gddddd
d	d
dddddddd�ZdSdd
�
Zdd�Zdd�Z e!dd��
Z"dd�Z#dd�Z$dd�Z%dd�Z&dd�Z'dd�Z(d d!�Z)d"d#�Z*d$d%�Z+d&d'�Z,d(d)�Z-dTd*d+�
Z.d,d-�Z/d.d/�Z0d0d1�Z1d2d3�Z2d4d5�Z3d6d7�Z4d8d9�Z5d:d;�Z6d<d=�Z7d>d?�Z8d@dA�Z9dUdBdC�
Z:dDdE�Z;dVdGdH�
Z<dIdJ�Z=dWdKdL�
Z>dMdN�Z?dXdOdP�
Z@dYdQdR�
ZAdS)Z�UploadTargetz�
    This class is designed to upload files to a distribution
    defined location. These files can be either sos reports,
    sos collections, or other kind of files like: vmcores,
    application cores, logs, etc.

    z�
            Upload a file (can be an sos report, a must-gather, or others) to
             a distribution defined remote location
            zGeneric UploadZgenericN�
/zhttps://s3.amazonaws.com�F�auto)�upload_file�case_id�low_priorityZprofiles�
upload_url�upload_directory�upload_user�upload_pass�
upload_method�upload_no_ssl_verify�upload_protocol�upload_s3_endpoint�upload_s3_region�upload_s3_bucket�upload_s3_access_key�upload_s3_secret_key�upload_s3_object_prefix�
upload_targetcCs"tj
d
�
|_|
|_||_||_dS)NZsos_ui)�loggingZ	getLogger�ui_log�parser�cmdline�args)�selfrrr�r!�/usr/lib/python3.6/__init__.py�__init__as


zUploadTarget.__init__c


Csd
S)aS
This should be overridden by upload targets

        This is called by sos upload on each target type that exists, and
        is meant to return True when the upload target matches a criteria
        that indicates that is the local upload target that should be used.

        Only the first upload target to determine a match is selectedFr!)
r r!r!r"�check_distributionhszUploadTarget.check_distributionc


Cs|jS)
N)
�upload_target_id)
r r!r!r"�
get_target_idrs
zUploadTarget.get_target_idc


Cs|jr|jS|j
j�S)
z-Returns the upload target's name as a string.)�upload_target_name�__name__�lower)
�clsr!r!r"�nameus

zUploadTarget.namec

Cs*|jd
|jd|jd
j
|jd
jd�S)N�cmdlineopts�policy)r,r-rr)�hook_commonsrr)
r r!r!r"�get_commons|s



zUploadTarget.get_commonscCs
|
|_d
S)zMSet common host data for the Upload targets
            to reference
        N)
�commons)r r0r!r!r"�set_commons�szUploadTarget.set_commonscCs�|
|_|j
�|_|jd
}|jd}|jr2|j�
|j|_|j|_|j|_|j|_	d|_
|j|_|j|_|j
|_
|j|_|j|_|j|_|jr�|jr�|j�r�|jdkr�|j�
|j�
n*|jdkr�|j�
|j�
|j�
|j�
|jjd�

dS)Nr,r-r�s3)r.r/r0rZ_configure_low_priorityr
rrr�upload_password�upload_archive_namerrrrrrZbatch�quiet�get_upload_urlr�prompt_for_upload_user�prompt_for_upload_password�prompt_for_upload_s3_bucket�prompt_for_upload_s3_endpoint�prompt_for_upload_s3_access_key�prompt_for_upload_s3_secret_keyr�info)r r.Zcmdline_optsr-r!r!r"�pre_work�s:



























zUploadTarget.pre_workc
Cs4|j�s0d
|j
��d|j��d�}
tt|
�
�
|_dS)zuShould be overridden by targets to determine if an access key needs
        to be provided for upload or not
        z0Please provide the upload access key for bucket z via endpoint z: N)�get_upload_s3_access_key�get_upload_s3_bucket�get_upload_s3_endpoint�input�
_r)r �msgr!r!r"r;�sz,UploadTarget.prompt_for_upload_s3_access_keyc
Cs0|j�s,d
|j
��d|j��d�}
t|
�
|_dS)ztShould be overridden by targets to determine if a secret key needs
        to be provided for upload or not
        z0Please provide the upload secret key for bucket z via endpoint z: N)�get_upload_s3_secret_keyr@rArr)r rDr!r!r"r<�sz,UploadTarget.prompt_for_upload_s3_secret_keyc
CsH|jsB|j
r*|j
jd
�
r*|j
dd�|_nttd�
�
}
|
jd�
|_|jS)zpShould be overridden by targets to determine if a bucket needs to
        be provided for upload or not
        zs3://�Nz"Please provide the upload bucket: r)rr
�
startswithrBrC�strip)r �
user_inputr!r!r"r9�s



z(UploadTarget.prompt_for_upload_s3_bucketc
Cs>|j}
|j
s8d
|j��d|
�d�}tt|�
�
}|p4|
|_
|j
S)zsShould be overridden by targets to determine if an endpoint needs
        to be provided for upload or not
        z.Please provide the upload endpoint for bucket z (default: z): )�_upload_s3_endpointrr@rBrC)r Zdefault_endpointrDrIr!r!r"r:�s



z*UploadTarget.prompt_for_upload_s3_endpointc
Cs*|j�s&d
|j
��d�}
tt|
�
�
|_dS)zcShould be overridden by targets to determine if a user needs to
        be provided or not
        zPlease provide upload user for z: N)�get_upload_userr6rBrCr)r rDr!r!r"r7�s

z#UploadTarget.prompt_for_upload_userc
Cs6|j�r2|j
�|jkr2d
|j
��d�}
t|
�
|_dS)zrShould be overridden by targets to determine if a password needs to
        be provided for upload or not
        z'Please provide the upload password for z: N)�get_upload_passwordrK�_upload_userrr3)r rDr!r!r"r8�s

z'UploadTarget.prompt_for_upload_passwordcCsL|
|_|j
s|j�|_
|j
s$td
�
�
|j�}|jjtd|j����
�

|�S)a	
        Entry point for sos attempts to upload the generated archive to a
        target or user specified location.

        Currently there is support for HTTPS, SFTP, and FTP. HTTPS uploads are
        preferred for target-defined defaults.

        Targets that need to override uploading methods should override the
        respective upload_https(), upload_sftp(), and/or upload_ftp() methods
        and should NOT override this method.

        :param archive: The archive filepath to use for upload
        :type archive: ``str``

        In order to enable this for a target, that target needs to implement
        the following:

        Required Class Attrs

        :_upload_url:     The default location to use. Note these MUST include
                          protocol header
        :_upload_user:    Default username, if any else None
        :_upload_password: Default password, if any else None

        The following Class Attrs may optionally be overidden by the Target

        :_upload_directory:     Default FTP server directory, if any


        The following methods may be overridden by ``Target`` as needed

        `prompt_for_upload_user()`
            Determines if sos should prompt for a username or not.

        `get_upload_user()`
            Determines if the default or a different username should be used

        `get_upload_https_auth()`
            Format authentication data for HTTPS uploads

        `get_upload_url_string()`
            Print a more human-friendly string than vendor URLs
        zBNo upload destination provided by upload target or by --upload-urlzAttempting upload to )	r4r
r6�	Exception�_determine_upload_typerr=rC�get_upload_url_string)r �archiveZupload_funcr!r!r"�upload_archive�s,






zUploadTarget.upload_archivec
Csv|j|j
|j|jd
�}
|jdj|
kr6|
|jdjSd|jkrHtd�
�
|jjd�
\}}||
krntd|���
�
|
|S)a
Based on the url provided, determine what type of upload to attempt.

        Note that this requires users to provide a FQDN address, such as
        https://myvendor.com/api or ftp://myvendor.com instead of
        myvendor.com/api or myvendor.com
        )Zftp�sftpZhttpsr2r,z://z#Must provide protocol in upload URLz&Unsupported or unrecognized protocol: )	�
upload_ftp�upload_sftp�upload_https�	upload_s3r0rr
rN�split)r ZprotsZprotrCr!r!r"rO3
s











z#UploadTarget._determine_upload_typecCs&|
s|j�}
|s|j
�}tjj|
|�S)
ah
Formats the user/password credentials using basic auth

        :param user: The username for upload
        :type user: ``str``

        :param password: Password for `user` to use for upload
        :type password: ``str``

        :returns: The user/password auth suitable for use in requests calls
        :rtype: ``requests.auth.HTTPBasicAuth()``
        )rKrL�requests�authZ
HTTPBasicAuth)r �user�passwordr!r!r"�get_upload_https_authI
s



z"UploadTarget.get_upload_https_authc

Cstj
d
d�p|jp|jS)z�Helper function to determine if we should use the target default
        upload access key or one provided by the user

        :returns: The access_key to use for upload
        :rtype: ``str``
        ZSOSUPLOADS3ACCESSKEYN)�os�getenvr�_upload_s3_access_key)
r r!r!r"r?\
s

z%UploadTarget.get_upload_s3_access_keyc


Cs|js|j
�
|jS)
z�Helper function to determine if we should use the target default
        upload endpoint or one provided by the user

        :returns: The S3 Endpoint to use for upload
        :rtype: ``str``
        )rr:)
r r!r!r"rAg
s

z#UploadTarget.get_upload_s3_endpointc


Cs|jp
|j
S)
z�Helper function to determine if we should use the target default
        upload region or one provided by the user

        :returns: The S3 region to use for upload
        :rtype: ``str``
        )r�_upload_s3_region)
r r!r!r"�get_upload_s3_regionr
sz!UploadTarget.get_upload_s3_regionc
Csb|jrH|jj
d
�
rH|jdd�jdd�}
|
d|_t|
�
dkrH|
d|_|jsV|j�
|jp`|jS)z�Helper function to determine if we should use the target default
        upload bucket or one provided by the user

        :returns: The S3 bucket to use for upload
        :rtype: ``str``
        zs3://rFNr�
r
)r
rGrXr�lenrr9�_upload_s3_bucket)r Zbucket_and_prefixr!r!r"r@{
s








z!UploadTarget.get_upload_s3_bucketc


Cs|jp
|j
S)
z�Helper function to determine if we should use the target default
        upload object prefix or one provided by the user

        :returns: The S3 object prefix to use for upload
        :rtype: ``str``
        )r�_upload_s3_object_prefix)
r r!r!r"�get_upload_s3_object_prefix�
sz(UploadTarget.get_upload_s3_object_prefixc

Cstj
d
d�p|jp|jS)z�Helper function to determine if we should use the target default
        upload secret key or one provided by the user

        :returns: The S3 secret key to use for upload
        :rtype: ``str``
        ZSOSUPLOADS3SECRETKEYN)r^r_r�_upload_s3_secret_key)
r r!r!r"rE�
s

z%UploadTarget.get_upload_s3_secret_keyc
CsH|jr<|j
r<|jr<|jr<|j�}
|j�}d
|
�d|��|_|jpF|jS)z�Helper function to determine if we should use the target default
        upload url or one provided by the user

        :returns: The URL to use for upload
        :rtype: ``str``
        zs3://r)r
rrrr@rg�_upload_url)r �bucket�prefixr!r!r"r6�
s





zUploadTarget.get_upload_urlcCsd
}tj
|d|
�}|S)Nz([^:]+://[^:]+:)([^@]+)(@.+)z\1********\3)�re�sub)r �url�patternZobfuscated_urlr!r!r"�_get_obfuscated_upload_url�
s


z'UploadTarget._get_obfuscated_upload_urlc

Cs|j|j
��
S)
z�Used by upload targets to potentially change the string used to
        report upload location from the URL to a more human-friendly string
        )rpr6)
r r!r!r"rP�
sz"UploadTarget.get_upload_url_stringc

Cstj
d
d�p|jp|jS)z�Helper function to determine if we should use the target default
        upload user or one provided by the user

        :returns: The username to use for upload
        :rtype: ``str``
        Z
SOSUPLOADUSERN)r^r_rrM)
r r!r!r"rK�
s

zUploadTarget.get_upload_userc

Cstj
d
d�p|jp|jS)a_
Helper function to determine if we should use the target default
        upload password or one provided by the user

        A user provided password, either via option or the 'SOSUPLOADPASSWORD'
        environment variable will have precendent over any target value

        :returns: The password to use for upload
        :rtype: ``str``
        ZSOSUPLOADPASSWORDN)r^r_r3�_upload_password)
r r!r!r"rL�
s


z UploadTarget.get_upload_passwordcCs:td
�
st
d�
�
yddl}Wn,tk
rH
}
zt
d�
|�WYdd}~XnXd}|
sZ|j�}
|sf|j�}|j�jdd�}d	|
�d
|��}|j|dd�}d
dd|j	|j
g}	|j|	dd�}
|
dkr�d}n�|
dk�
r|j|�

d
d|j	|j
g}|j|dd�dk}|�
sr|j
�
t
d|j����
�
nX|
dk�
r:t
d|j��d��
�
n8|
dk�
rXt
d|j����
�
n|
dk�
rrt
d|j���
�
|�
s�|j
�
t
d|j����
�
d|j�d |j���}|j|�

d!|j	|j
d"g}
|j|
d#d�}|dk�
r�|jd$�

dS|dk�
r�t
d%�
�
|dk�rt
d&|j���
�
|dk�r&t
d'�
�
t
d(|j���
�
dS))a�
Attempts to upload the archive to an SFTP location.

        Due to the lack of well maintained, secure, and generally widespread
        python libraries for SFTP, sos will shell-out to the system's local ssh
        installation in order to handle these uploads.

        Do not override this method with one that uses python-paramiko, as the
        upstream sos team will reject any PR that includes that dependency.
        rSzSFTP is not locally supportedr
NzFSFTP upload requires python3-pexpect, which is not currently installedFzsftp://rz sftp -oStrictHostKeyChecking=no �
@zutf-8)
�encodingzsftp>z	password:zConnection refused�)
�timeoutTrczPermission denied�
z#Incorrect username or password for �zConnection refused by z. Incorrect port?�z!Timeout hit trying to connect to �z,Unexpected error trying to connect to sftp: zUnable to connect via SFTP to zput �
 z100%zNo such file or directory�ZbyezTimeout expired while uploadingzUnknown error during upload: z&Unable to write archive to destinationz!Unexpected response from server: )rrN�pexpect�ImportErrorrKrLr6�replaceZspawnZTIMEOUTZEOF�expectZsendline�closerPZbeforer4�_get_sftp_upload_name)r r[r\r|�errZsftp_connectedZsftp_urlZsftp_cmd�retZsftp_expects�idxZpass_expectsZput_cmdZput_expectsZput_successr!r!r"rU�
sv


















































zUploadTarget.upload_sftpc
Cs*|jj
d
�
d}
|jr&tjj|j|
�}
|
S)a$
If a specific file name pattern is required by the SFTP server,
        override this method in the relevant Upload Target. Otherwise the
        archive's name on disk will be used

        :returns:       Filename as it will exist on the SFTP server
        :rtype:         ``str``
        rrc���)r4rXrr^�path�join)r Zfnamer!r!r"r�7s


z"UploadTarget._get_sftp_upload_nameTcCstj
|j�|
|j�|td
�S)z�If upload_https() needs to use requests.put(), use this method.

        Targets should override this method instead of the base upload_https()

        :param archive:     The open archive file object
        )�datarZ�verifyru)rY�putr6r]r)r rQr�r!r!r"�_upload_https_putDs

zUploadTarget._upload_https_putc


CsiS)
zJDefine any needed headers to be passed with the POST request here
        r!)
r r!r!r"�_get_upload_headersOsz UploadTarget._get_upload_headerscCs:d
|
jj
d�
d|
|j�fi
}tj|j�||j�|td�S)z�If upload_https() needs to use requests.post(), use this method.

        Targets should override this method instead of the base upload_https()

        :param archive:     The open archive file object
        �filerrc)�filesrZr�rur�)r+rXr�rYZpostr6r]r)r rQr�r�r!r!r"�_upload_https_postTs



zUploadTarget._upload_https_postc
Cs�tst
d
�
�
t|jd���}
|jdjdkr2|j}n|jdj}|jdjdk}|dkrd|j|
|�}n|j	|
|�}|j
dkr�|j
d	kr�t
d
�
�
t
d|j
�d|j���
�
d
SQRXdS)z�Attempts to upload the archive to an HTTPS location.

        :returns: ``True`` if upload is successful
        :rtype: ``bool``

        :raises: ``Exception`` if upload was unsuccessful
        z7Unable to upload due to missing python requests library�rbr,r	Fr�����i�
z/Authentication failed: invalid user credentialszPOST request returned z: TN)r�r�)�REQUESTS_LOADEDrN�openr4r0r�_upload_methodrr�r�Zstatus_code�reason)r Zarc�methodr��
rr!r!r"rVcs 











zUploadTarget.upload_httpscCs�
d
dl}d
dl
}|
s|j�}
|
dkr,td�
�
|
jdd�}
|sD|j�}|sP|j�}|s`|jp^|j}y.|j	|
||dd�}|s�td�
�
|j
|�

W�
n|jk
r�
}
ztd	|
���
|�WYdd}~Xn�|jk
r�
}
ztd
|
���
|�WYdd}~Xn�|j
k
�
r�
}
z|t|�
j�d
}	|	dk�
r4td|�d
��
|�|	dk�
rPtd|�d
��
|�|	dk�
rjtd|���
|�tdt|�
���
|�WYdd}~XnXt|jd��$}
|jd|jjd�
d��|
�
WdQRX|j�
dS)aTAttempts to upload the archive to either the target defined or user
        provided FTP location.

        :param url: The URL to upload to
        :type url: ``str``

        :param directory: The directory on the FTP server to write to
        :type directory: ``str`` or ``None``

        :param user: The user to authenticate with
        :type user: ``str``

        :param password: The password to use for `user`
        :type password: ``str``

        :returns: ``True`` if upload is successful
        :rtype: ``bool``

        :raises: ``Exception`` if upload in unsuccessful
        r
NzPno FTP server specified by upload target, use --upload-url to specify a locationzftp://rrt)
ruz3connection failed, did you set a user and password?z timeout hit while connecting to zunable to connect to Z503zcould not login as '�
'Z530zinvalid password for user 'Z550z"could not set upload directory to z#error trying to establish session: r�zSTOR rrcTr�)�ftplib�socketr6rNr~rKrLr�_upload_directoryZFTP�cwdruZgaierrorZ
error_perm�strrXr�r4Z
storbinary�quit)r rnZ	directoryr[r\r�r�Zsessionr��errnoZ_arcfiler!r!r"rT�sL











 

 















$
zUploadTarget.upload_ftpc
Cs
tst
d
�
�
|
s|j�}
|s$|j�}|s6|j�jd�
}|s�|j�}|dkr`|jd�
r`|dd�}|dkr�|jd�
r�|r�|�d�nd}|s�|j	�}|s�|j
�}tjd|
|||d�}y(||j
jd�
d
}|j|j
||�
dSt
k
�
r
}	
zt
d	t|	�
���
|	�WYdd}	~	XnXdS)a�Attempts to upload the archive to an S3 bucket.

        :param endpoint: The S3 endpoint to upload to
        :type endpoint: str

        :param region: The S3 region to upload to
        :type region: str

        :param bucket: The name of the S3 bucket to upload to
        :type bucket: str

        :param prefix: The prefix for the S3 object/key
        :type prefix: str

        :param access_key: The access key for the S3 bucket
        :type access_key: str

        :param secret_key: The secret key for the S3 bucket
        :type secret_key: str

        :returns: True if upload is successful
        :rtype: bool

        :raises: Exception if upload is unsuccessful
        z4Unable to upload due to missing python boto3 libraryrrrcNr2)Zendpoint_urlZregion_nameZaws_access_key_idZaws_secret_access_keyTzFailed to upload to S3: r�)�BOTO3_LOADEDrNrArbr@rHrgrG�endswithr?rE�boto3Zclientr4rXr
r�)
r ZendpointZregionrjrkZ
access_keyZ
secret_keyZ	s3_client�key�
er!r!r"rW�s:




















zUploadTarget.upload_s3)NNN)NN)NN)
T)
T)NNNN)NNNNNN)Br(�
__module__�__qualname__�__doc__Zdescr'r%Z_upload_filerir�rMrqr�rJrer`rhrarfr
rr3rrrrrrrZarg_defaultsr#r$r&�classmethodr+r/r1r>r;r<r9r:r7r8rRrOr]r?rArbr@rgrEr6rprPrKrLrUr�r�r�r�rVrTrWr!r!r!r"r s�









































	*

8
		
c


F
r)r^rlrrZsosrrCZ
sos.utilitiesrrrYr�r}r�r�rr!r!r!r"�<module>s