Edit File: mysql_governor_lib.cpython-37.pyc
B � e6� � @ s� d dl mZ d dl mZ d dl mZ d dlZd dlm mZ d dl Z d dl Z d dl m Z d dl mZ d dlmZmZmZ d dlmZ d d lmZmZ G d d � d �ZG d d � d e�ZG dd� de�ZG dd� de�ZG dd� d�Z dS )� )�absolute_import)�division)�print_functionN)� LooseVersion)� ExpatError)�Tuple�Optional�Dict)�FormattedException)� run_command�ExternalProgramFailedc @ s e Zd ZdZdZdZdS )�GovernorStatusZenabled�errorZdisabledN)�__name__� __module__� __qualname__�ENABLED�ERROR�DISABLED� r r ��/builddir/build/BUILDROOT/alt-python27-cllib-3.3.5-1.el8.cloudlinux.x86_64/opt/alt/python37/lib/python3.7/site-packages/clcommon/lib/mysql_governor_lib.pyr s r c @ s e Zd ZdS )�MySQLGovExceptionN)r r r r r r r r s r c s e Zd ZdZ� fdd�Z� ZS )�MySQLGovernorDisabledz< Exception raised when dbgovernor daemon is offline c s t t| ��dddd�d�� d S )NzR%(util)s is disabled in the system. Please, run "%(command)s" to start the servicezMySQL governorzservice db_governor restart)�util�command)�message�context)�superr �__init__)�self)� __class__r r r ( s z MySQLGovernorDisabled.__init__)r r r �__doc__r � __classcell__r r )r r r $ s r c s e Zd ZdZ� fdd�Z� ZS )�MySQLGovernorAbsentz: Exception raised when dbgovernor isn't installed c s t t| ��dddid�� d S )Nz %(util)s not present in systemr ZGovernor)r r )r r# r )r )r r r r 8 s z MySQLGovernorAbsent.__init__)r r r r! r r" r r )r r r# 4 s r# c @ s� e Zd ZdZdZdZdZdZdZdZ dZ d Z d d � Z d d � Z dd� ZdWdd�Zdd� Zdd� Zdd� Zedd� �Zdd � Zd d � Zd d � Zed!d"� �Zed#d$� �Zd%d&� Zd'd(� ZdXe e e e f d)�d*d+�Z d,d-� Z d.d/� Z!d0d1� Z"d2d3� Z#dYd5d6�Z$e%e& e'd7�d8d9�Z(e)e)e)e e e f d:�d;d<�Z*dZe%e& e%e+ d7�d=d>�Z,e&e)d?�d@dA�Z-e&e%e e%e dB�dCdD�Z.e&e%e e%e dB�dEdF�Z/d[e&e%e e%e dB�dGdH�Z0dIdJ� Z1dKdL� Z2dMdN� Z3ee&e&dO�dPdQ��Z4d\dRdS�Z5e e'e'dT�dUdV�Z6d4S )]� MySQLGovernorz MysqlGovernor library )g��(\��?gR� ��Q�?g�z�G��?)gffffff�?gףp= ��?g �?z/usr/sbin/dbctlz7/usr/share/lve/dbgovernor/governor_package_limitting.pyz/usr/sbin/db_governorz!/etc/container/mysql-governor.xmlz"/var/run/mysql-governor-config.xmlZDEFAULTc C s: d | _ d | _d| _tj�| j�| _| �� | _ | � � | _ d S )NF) �_governor_ignored_users�_governor_limits�_is_governor_error�os�path�isfile� _UTILITY_PATH�_is_governor_present�_detect_governor_mode�_governor_mode�get_governor_version�_governor_version)r r r r r S s zMySQLGovernor.__init__c C s | j S )N)r. )r r r r �get_governor_modee s z MySQLGovernor.get_governor_modec C sF | j r>y | �� }|�d�d �d�S ttfk r: d S X nd S d S )NZlver Zuse)r, �_get_xml_config�getElementsByTagName� getAttributer � IndexError)r � governor_cfgr r r r- h s z#MySQLGovernor._detect_governor_modeFc C sh t | jgt|� dd�\}}}d|kr@d| _d| _d| _t� �n |r^|sL|r^tdd|id���|||fS )a Run dbctl utility with given arguments and handle common errors: - governor is down: MySQLGovernorDisabled :param check_exit_code: whether we should raise exception when dbctl returs code != 0 T)�return_full_outputzcan't connect to socketNzdbctl error: %(output)s�output)r r )r r+ �listr' r& r% r r )r �args�check_exit_code�ret_code�std_out�std_errr r r �_run_dbctl_with_argsr s z"MySQLGovernor._run_dbctl_with_argsc C s | � � S )N)�_detect_governor_status)r r r r �get_governor_status� s z!MySQLGovernor.get_governor_statusc C s� | � � rzy | �� W nZ tk r< } z tj|fS d }~X Y n> tk rn } ztjtdt|�i�fS d }~X Y n X tjd fS tjd fS )Nr ) �is_governor_present� _load_infor r r r �strr r )r �er r r r@ � s z%MySQLGovernor._detect_governor_statusc C s� | � � }y$t|d��}t�|�� �S Q R X W n� ttfk rp } z"d| _td|t |�d�d���W d d }~X Y n@ t k r� } z"d| _td|t |�d�d���W d d }~X Y nX d S )N�rTzNAn error occured while loading governor config from %(path)s. Error: %(error)s)r) r )r r zNAn error occured while parsing governor config from %(path)s. Error: %(error)s) �_get_config_path�open�xmlZ parseString�read�IOError�OSErrorr' r rD r )r Z config_path�frE r r r r2 � s $z MySQLGovernor._get_xml_configc C s� t � }y| �d�d }W n& tk r> tdddid���Y nX |�d�}x8|D ]0}|�d�}|dkrP|�d �rP|�|�d �� qPW |S ) z� Loads information about igrored users :type governor_cfg: xml.Document :return: list of ignore users Zgovernorr zFMalformed mysql-governor config. Unable to find element '%(element)s'.Zelement)r r �user�mode�ignore�name)r9 r3 r5 r r4 �append)r6 Zignore_users_listZgov_dataZusers_data_listZ user_dataZ user_moder r r �_load_ignore_users_from_xml� s z)MySQLGovernor._load_ignore_users_from_xmlc C sh | � d�r|d }d}n|}|dkr(d}d}|| �t|d �| j� }|| �t|d �| j� }||fS )z� Calculate db R/W IO limits based on governor version :param io: requested limits in MB/s :return: string values suitable to pass to "dbctl set ..." z1.2-17i z%sb,%sb,%sb,%sb� � z %s,%s,%s,%s)�_is_governor_newer_then� _percentage�int� IO_PERCENTS)r �ioZ io_limitsZ limits_tmplrJ �writer r r �_calc_rw_io_limits� s z MySQLGovernor._calc_rw_io_limitsc C sV | � � s d S y*tj| jdgdd�}|�� �d�d }|S tjtfk rP d S X d S )Nz --versionT)�text� rU )rB � subprocessZ check_output�_GOVERNOR_BINARY_PATH�strip�splitZCalledProcessErrorrL )r �res�versionr r r r/ � s z"MySQLGovernor.get_governor_versionc C s | j }|sdS t|�t|�k S )NF)r0 r )r rd Zcurrentr r r rV � s z%MySQLGovernor._is_governor_newer_thenc C s dd� | � � D �S )a� Convert data line from dbctl to list :param line: Data line could be like: "default 400/380/350/300 953/791/724/562 953/791/724/562" or: "default 400/380/350/300 1000/830/760/590 1000/830/760/590" depending on --kb/mb/bb option passed to dbctl :return: list: ['default', '400/380/350/300', '953/791/724/562', '953/791/724/562'] c S s g | ] }|r|�qS r r )�.0�partr r r � <listcomp>� s z-MySQLGovernor._parse_line.<locals>.<listcomp>)rb )�liner r r � _parse_line� s zMySQLGovernor._parse_linec C s. | g}x |D ]}|� t| | �� q W t|�S )zX Calculate full list of governor limits by one value and percents koeff )rR rX �tuple)�valueZpercentsrc �kr r r rW s zMySQLGovernor._percentagec C s | j S )z= Get governor presence flag :return: )r, )r r r r rB s z!MySQLGovernor.is_governor_presentc C s | � � || jkrdS dS )z� Get MySQL governor status for supplied user :param username: Username for get status :return: Governor status: "watched"/"ignored" or None if error ZignoredZwatched)rC r% )r �usernamer r r � get_governor_status_by_username s z-MySQLGovernor.get_governor_status_by_username)�with_package_mark�returnc C sR | � � || jkr&| j| \}}}}n| jd \}}}}|rF||f}n||f}|S )a Get MySQL governor limits for supplied user :param username: Username for read limits :param with_package_mark: False - without package limits difference mark (for compatibility with non-package governor, used in cloudlinux-top/cloudlinux-statistics), True - with package limits difference mark :return: Tuple (CPU limit, IO limit). Examples: (150, 3072) - with_package_mark == False ('*150', '3072') - with_package_mark == True ('*150', '*4096') - with_package_mark == True * - user has individual limits, differ from package limit MySQLGovException will be thrown if governor not present or error �default)rC r& )r rm ro �user_cpu_limit�user_io_limit_kb�user_cpu_limit_marked�user_io_limit_markedZlimits_for_returnr r r �get_limits_by_user s z MySQLGovernor.get_limits_by_userc C sP | � � |rdnd}| �||g�\}}}|s0|rLd||p:|d�d�}t|��dS )z� Set MySQLGovernor status for single user :param: `str` username: Username for set status :param: `bool` status: True for "monitor", False for "ignore" :return: `bool`: operation status result ZmonitorrP z.Set governor status error(%(ret)s): %(output)s)�retr8 )r r r )rC r? r )r rm �status� status_cmdrw r= r> � exc_messager r r � set_governor_status_for_user; s z*MySQLGovernor.set_governor_status_for_userc C s\ | � � |rdnd}|dkr d}| �||g�\}}}|s<|rXd||pF|d�d�}t|��dS ) z� Set user restricted with dbctl utility :param: `str` username: Username for set restricted status :param: `bool` status: True for "restricted", False for "unrestricted" :return: `bool`: operation status result ZrestrictZ unrestrict)�root�adminrq z,Set user restrict error(%(ret)s): %(output)s)rw r8 )r r r )rC r? r )r rm rx ry rw r= r> rz r r r � set_restricted_status_for_userK s z,MySQLGovernor.set_restricted_status_for_userc C sB | � � | �dg�\}}}|s"|r>d||p,|d�d�}t|��dS )zi Set user restricted with dbctl utility :return: `bool`: operation status result zunrestrict-allz:Set all users unrestrict status error(%(ret)s): %(output)s)rw r8 )r r r )rC r? r )r rw r= r> rz r r r �%set_unrestricted_status_for_all_users] s z3MySQLGovernor.set_unrestricted_status_for_all_usersc C s& | � � |dkrd}|| jkr"dS dS )z� Get MySQL governor status for supplied user :param username: Username for get status :return: Governor restricted status: "restricted"/"unrestricted" )r| r} rq Z restrictedZ unrestricted)rC �_governor_restricted_users)r rm r r r � get_restrict_status_by_usernamej s z-MySQLGovernor.get_restrict_status_by_usernameNc C s. |dkr|dkrdS | � � d|g}dd� }|dk rlt|t�rRd| �|| j� }n t|d���|�d| � |dk r�t|t�r�| �|�\}}n t|d ���|�d | � |�d | � y| �|�\}} } W n. t k r� } ztt | ���W dd} ~ X Y nX | �s|�r$d || �p| d �d�} t| ��d| _ dS )a� Set MySQLGovernor limits for user :param: username `str`: username for set limits :param: `int`|`list` cpu: governor cpu limit. when it param int - calculate by percentage other params :param: `int`|`list` io: io value means that read and write limits similar :param: `int`|`list` read: read limit :param: `int`|`list` write: write limit :return: 0 Nr �setc S s dd| id�S )NzInvalid %(param)s parameter�param)r r r )r� r r r �<lambda>� s z3MySQLGovernor.set_limits_for_user.<locals>.<lambda>z %s,%s,%s,%s�cpuz--cpu=%srZ z --read=%sz --write=%sz:Set all users unrestrict status error(%(ret)s): %(output)s)rw r8 )r r ) rC � isinstancerX rW � CPU_PERCENTSr rR r\ r? r rD r& ) r rm r� rZ �cmd�get_exc_messagerJ r[ rw r= r> rE rz r r r �set_limits_for_userx s4 z!MySQLGovernor.set_limits_for_user)� package_namerp c C s� |r$| j dd|�� �d�� �dg}n| j dddg}t|dd�\}}}|dksP|rjtd d �|�|d �d ���yt�|�}W n@ tjfk r� } z td | j t |�d�d ���W dd}~X Y nX |S )z� Retrieve MySQL Governor package limits :param package_name: Package name. If None, get all packages name :return: Dict with limits. Example: {'pack1': {'cpu': 100, 'io': 900}, 'pack2': {'cpu': 100, 'io': 900}} �getz --package=zunicode-escapez --format=kbz--allT)r7 r z''%(cmd)s' failed, stderr is: %(stderr)sr^ )r� �stderr)r r z,%(util)s output invalid, error is: %(error)s)r r N) �_PACKAGE_UTILITY_PATH�encode�decoder r �join�json�loadsZJSONDecodeErrorrD )r r� �cmd_listr< r= r> �package_data_from_utilrE r r r �$_get_package_raw_limits_from_utility� s" (z2MySQLGovernor._get_package_raw_limits_from_utility)�cpu_limits_list�read_limits_list�write_limits_listrp c C s( t |d �| �t|d �t|d ��fS )a Calculate package limits from raw governor limits :param cpu_limits_list: CPU limits list :param read_limits_list: Read limits list :param write_limits_list: Write limits list :return: Tuple: (cpu_limit, io_limit) r )rX �_get_user_io_limitrD )r r� r� r� r r r � _calc_package_limits_from_raw� s z+MySQLGovernor._calc_package_limits_from_rawc C sp y| � |�}W n tk r$ � Y nX i }x@|�� D ]4\}}| �|d |d |d �\}}||d�||< q4W |S )z� Retrieve MySQL Governor package limits :param package_name: Package name. If None, get all packages name :return: Dict with limits. Example: {'pack1': {'cpu': 100, 'io': 900}, 'pack2': {'cpu': 100, 'io': 900}} r� rJ r[ )r� rZ )r� r �itemsr� )r r� r� Z packages_dataZ pack_nameZ pack_limits� cpu_limit�io_limitr r r �get_package_limits� s z MySQLGovernor.get_package_limits)rm � limit_namesc C s` d� |�}| jdd|� �d|� �g}t|dd�\}}}|dksB|r\tdd � |�|d �d ���d S ) z� Reset users limits to default :param username: User name ro reset limits :param limit_names: Limit names list to reset �,Zreset_individualz--user=z --limits=T)r7 r z*'%(cmd)s' is failed, stderr is: %(stderr)sr^ )r� r� )r r N)r� r� r r )r rm r� Z limits_stringr� r< �_r> r r r � reset_user_limits_to_defaults� s z+MySQLGovernor.reset_user_limits_to_defaults)r� r� r� c C s� ddl m}m} d|� �}y ||�}W n ttfk r> dS X t|�dkrPdS g }d} �x*|D �] } | �d�r�|dk r�| �� �d�} t| �dkr�q`|dkr�| j } nt |�} | d �� } | | kr�|� d | � d �� d } n|� | � d �� q`| �d ��rx|dk �rx|dk�r| j }nt |�}| �� �d�} t| �dk�r8q`| d �� }||k�rf|� d|� d �� d } n|� | � d �� q`|� | � q`W | �r�|||d� dS )z� Set MySQL Governor to cPanel package file. If limits not changed, package file will not be written :param package_name: Package name :param cpu_limit: MySQL CPU limit to set :param io_limit: MySQL IO limit to set r )�get_file_lines�write_file_linesz/var/cpanel/packages/NFZ lve_mysql_cpu�=rU rT zlve_mysql_cpu=� TZ lve_mysql_ioz lve_mysql_io=�w) Zclcommon.utilsr� r� rL rK �len� startswithra rb � _S_DEFAULTrD rR )r r� r� r� r� r� Z package_pathZcpanel_package_linesZlines_to_writeZis_change_maderh �partsZ s_cpu_limitZs_old_cpu_limitZ s_io_limitrk r r r �&_set_governor_limits_to_cpanel_package sL z4MySQLGovernor._set_governor_limits_to_cpanel_packagec C s, ddl m} |� dkrdS | �|||� dS )a& Apply all MySQL Governor packages limits to cpanel package file. In not cPanel, do nothing :param package_name: Package name to update :param cpu_limit: MySQL CPU limit to set, None - not change :param io_limit: MySQL IO limit to set, None - not change r )� getCPNameZcPanelN)Zclcommon.cpapir� r� )r r� r� r� r� r r r � _apply_package_limits_to_cpanelB s z-MySQLGovernor._apply_package_limits_to_cpanelc C s" |dks|dkr4|dkr4t d|� d|� d|� d���| jdd|g}dd � }|dk r�t|t�rpd | �|| j� }n t |d ���|�d | � |dk r�t|t�r�| �|�\}}n t |d ���|�d| � |�d| � t|dd�\} } } | dks�| �rt dd� |�| | d�d���| � |||� dS )z� Set limits for Governor package :param package_name: Package name for set :param cpu_limit: MySQL CPU limit to set :param io_limit: MySQL CPU limit to set Nz�MySQLGovernor.set_package_limits arguments error: Package name and at least one limit should be provided. Current arguments: package name: z; cpu limit is z; IO limit is �;r� z --packagec S s dd| id�S )NzInvalid %(param)s parameterr� )r r r )r� r r r r� ` s z2MySQLGovernor.set_package_limits.<locals>.<lambda>z %s,%s,%s,%sr� z--cpu=%sr� z --read=%sz --write=%sT)r7 r zI'%(command)s' is failed, stdout is: '%(stdout)s', stderr is: '%(stderr)s'r^ )r �stdoutr� )r r ) r r� r� rX rW r� rR r\ r r� r� ) r r� r� r� r� r� r� rJ r[ r< r= r> r r r �set_package_limitsN s* z MySQLGovernor.set_package_limitsc C s t j�| j�r| jS | jS )zN Get config path for mysql-governor; :rtype: str|None )r( r) r* �_CONTAINER_PATH_V2�_CONTAINER_PATH)r r r r rG y s z MySQLGovernor._get_config_pathc C s@ y| � � }| �|�| _W n" tk r: d| _d| _� Y nX dS )z*Load ignore users list from container fileN)r2 rS r% r r& )r Z governor_xmlr r r �_read_ignore_users� s z MySQLGovernor._read_ignore_usersc C s� | j dk r| jdk rdS | js*d| _t� �dd| jid�}| �� | �� \}}| jdgdd�\}}}dd � |� � � d �d d� D �| _ |� d �}| � |||�| _d | jkr�d| _d| _d| _ d dd| j d�d�}t |��dS )zL Loads users info from MySQL governor :return: None NTz %(utility)s output is invalidZutility)r r zlist-restricted)r; c S s g | ]}|� � d �qS )r )rb )re �lr r r rg � s z,MySQLGovernor._load_info.<locals>.<listcomp>r� rT rq z'There is no %(what)s found in %(where)szdefault settingsz %s output)Zwhat�where)r% r& r, r' r# r+ r� �_run_dbctl_listr? ra rb r� �_parse_dbctl_data_linesr )r �utility_exc_message�is_kb_limits_ok� gov_data_strr� Zgov_restricted_strZgov_data_linesrz r r r rC � s( " zMySQLGovernor._load_info)� read_limit� write_limitc C sf yt | �t |� }W nL tk r` | | kr8dkrBn nd}n|dkrTt | �}nt |�}Y nX |S )z� Calculates the io limit. Handles the situation when user's write or read io limit is less than 1mb/s (PTCLLIB-85). :type write_limit: str :type read_limit: str :rtype: int z<1rT )rX � ValueError)r� r� � user_io_limitr r r r� � s z MySQLGovernor._get_user_io_limitc C sJ | j ddgdd�\}}}d}|s(d|krB| j dgdd�\}}}d}||fS )aW Executes dbctl list-marked --kb or dbctl list-marked :param _is_incorrect_syntax: True is emulate dbctl error. Only for testing! :return: Cortege (is_kb_limits_ok, stdout_str), where is_kb_limits_ok == True, if dbctl returned limits in KB, else - False stdout_str - dbctl stdout string z list-markedz--kbT)r; zIncorrect syntaxr9 F)r? )r Z_is_incorrect_syntaxr< r� r� r� r r r r� � s z MySQLGovernor._run_dbctl_list)r� r� rp c C st t � }�xf|D �]\}|�� }|rd|kr*qt�|�}t|�dksPt|d �dkrjd| _d| _d| _t|��|d � d�}|d � d�}|d � d�} t|�dks�t|�dks�t| �dkr�d| _d| _d| _t|��|d } t |d �} | � |d | d �} |�r| n| d } |d }|d d k�r2d |d � �}n|d }|d d k�rTd | � �}nt | �}| | ||f|| < qW |S )a� Converts data lines from dbctl stdout to dictionary :param data_lines_list: List of lines from dbctl stdout :param is_kb_limits_ok: Is limits already in KB/s :param utility_exc_message: Message dict for exception :return: Tuple(dict, dict) dbctl data dictionary. Example: {'default': (400, 1953124, '400', '1953124'), 'cltest1': (350, 2025138, '*350', '2025138') } zcpu(%)� � rU TNrT �/� r i �+�*) �dictra r$ ri r� r' r& r% r rb rX r� rD )r Zdata_lines_listr� r� Zgovernor_limitsrh Zuser_limits_listr� r� r� Z user_namerr r� rs Zmarksrt ru r r r r� � s@ $ z%MySQLGovernor._parse_dbctl_data_lines)F)F)NN)N)NN)F)7r r r r! rY r� r+ r� r` r� r� r� r r1 r- r? rA r@ r2 � staticmethodrS r\ r/ rV ri rW rB rn �boolr rX rv r{ r~ r r� r� r rD r� r� r9 r� r r� r� r� r� r� rG r� rC r� r� r� r r r r r$ ? s\ F 8 * ' r$ ) Z __future__r r r r( Zxml.dom.minidomZdomZminidomrI r_ r� Zdistutils.versionr Zxml.parsers.expatr �typingr r r Zclcommon.clexceptionr Zclcommon.utils_cmdr r r r r r# r$ r r r r �<module> s