masakari-monitors-9.0.0/0000775000175000017500000000000013656750123015212 5ustar zuulzuul00000000000000masakari-monitors-9.0.0/doc/0000775000175000017500000000000013656750123015757 5ustar zuulzuul00000000000000masakari-monitors-9.0.0/doc/requirements.txt0000664000175000017500000000050713656750014021244 0ustar zuulzuul00000000000000# The order of packages is significant, because pip processes them in the order # of appearance. Changing the order has an impact on the overall integration # process, which may cause wedges in the gate later. sphinx!=1.6.6,!=1.6.7,>=1.6.2 # BSD openstackdocstheme>=1.24.0 # Apache-2.0 # releasenotes reno>=2.5.0 # Apache-2.0 masakari-monitors-9.0.0/doc/source/0000775000175000017500000000000013656750123017257 5ustar zuulzuul00000000000000masakari-monitors-9.0.0/doc/source/usage.rst0000664000175000017500000000013613656750014021114 0ustar zuulzuul00000000000000======== Usage ======== To use masakari-monitors in a project:: import masakarimonitors masakari-monitors-9.0.0/doc/source/conf.py0000775000175000017500000000472713656750014020572 0ustar zuulzuul00000000000000# -*- coding: utf-8 -*- # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or # implied. # See the License for the specific language governing permissions and # limitations under the License. import os import sys sys.path.insert(0, os.path.abspath('../..')) # -- General configuration ---------------------------------------------------- # Add any Sphinx extension module names here, as strings. They can be # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom ones. extensions = [ 'sphinx.ext.autodoc', 'openstackdocstheme', ] # autodoc generation is a bit aggressive and a nuisance when doing heavy # text edit cycles. # execute "export SPHINX_DEBUG=1" in your terminal to disable # The suffix of source filenames. source_suffix = '.rst' # The master toctree document. master_doc = 'index' # General information about the project. project = u'masakari-monitors' copyright = u'2016, OpenStack Foundation' # If true, '()' will be appended to :func: etc. cross-reference text. add_function_parentheses = True # If true, the current module name will be prepended to all description # unit titles (such as .. function::). add_module_names = True # The name of the Pygments (syntax highlighting) style to use. pygments_style = 'sphinx' # -- Options for HTML output -------------------------------------------------- # The theme to use for HTML and HTML Help pages. Major themes that come with # Sphinx are currently 'default' and 'sphinxdoc'. # html_theme_path = ["."] # html_theme = '_theme' # html_static_path = ['static'] html_theme = 'openstackdocs' repository_name ='openstack/masakari-monitors' # Output file base name for HTML help builder. htmlhelp_basename = '%sdoc' % project # Grouping the document tree into LaTeX files. List of tuples # (source start file, target name, title, author, documentclass # [howto/manual]). latex_documents = [ ('index', '%s.tex' % project, u'%s Documentation' % project, u'OpenStack Foundation', 'manual'), ] # Example configuration for intersphinx: refer to the Python standard library. #intersphinx_mapping = {'http://docs.python.org/': None} masakari-monitors-9.0.0/doc/source/readme.rst0000664000175000017500000000003613656750014021244 0ustar zuulzuul00000000000000.. include:: ../../README.rst masakari-monitors-9.0.0/doc/source/contributing.rst0000664000175000017500000000011313656750014022512 0ustar zuulzuul00000000000000============ Contributing ============ .. include:: ../../CONTRIBUTING.rst masakari-monitors-9.0.0/doc/source/installation.rst0000664000175000017500000000033413656750014022511 0ustar zuulzuul00000000000000============ Installation ============ At the command line:: $ pip install masakari-monitors Or, if you have virtualenvwrapper installed:: $ mkvirtualenv masakari-monitors $ pip install masakari-monitors masakari-monitors-9.0.0/doc/source/index.rst0000664000175000017500000000100513656750014021113 0ustar zuulzuul00000000000000.. masakari-monitors documentation master file, created by sphinx-quickstart on Tue Jul 9 22:26:36 2013. You can adapt this file completely to your liking, but it should at least contain the root `toctree` directive. Welcome to masakari-monitors's documentation! ======================================================== Contents: .. toctree:: :maxdepth: 2 readme installation usage contributing Indices and tables ================== * :ref:`genindex` * :ref:`modindex` * :ref:`search` masakari-monitors-9.0.0/requirements.txt0000664000175000017500000000127713656750014020504 0ustar zuulzuul00000000000000# The order of packages is significant, because pip processes them in the order # of appearance. Changing the order has an impact on the overall integration # process, which may cause wedges in the gate later. automaton>=1.9.0 # Apache-2.0 libvirt-python!=4.1.0,>=3.5.0 # LGPLv2+ openstacksdk>=0.13.0 # Apache-2.0 oslo.concurrency>=3.26.0 # Apache-2.0 oslo.config>=5.2.0 # Apache-2.0 lxml!=3.7.0,>=3.4.1 # BSD oslo.cache>=1.26.0 # Apache-2.0 oslo.i18n>=3.15.3 # Apache-2.0 oslo.log>=3.36.0 # Apache-2.0 oslo.middleware>=3.31.0 # Apache-2.0 oslo.privsep>=1.23.0 # Apache-2.0 oslo.service!=1.28.1,>=1.24.0 # Apache-2.0 oslo.utils>=3.33.0 # Apache-2.0 pbr!=2.1.0,>=2.0.0 # Apache-2.0 six>=1.10.0 # MIT masakari-monitors-9.0.0/masakarimonitors/0000775000175000017500000000000013656750123020575 5ustar zuulzuul00000000000000masakari-monitors-9.0.0/masakarimonitors/utils.py0000664000175000017500000001106313656750014022307 0ustar zuulzuul00000000000000# Copyright(c) 2016 Nippon Telegraph and Telephone Corporation # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. """Utilities and helper functions.""" import contextlib import inspect import pyclbr import shutil import sys import tempfile from oslo_concurrency import lockutils from oslo_concurrency import processutils from oslo_log import log as logging from oslo_utils import importutils import six import masakarimonitors.conf from masakarimonitors.i18n import _ from masakarimonitors import privsep CONF = masakarimonitors.conf.CONF LOG = logging.getLogger(__name__) def monkey_patch(): """monkey_patch function. If the CONF.monkey_patch set as True, this function patches a decorator for all functions in specified modules. You can set decorators for each modules using CONF.monkey_patch_modules. The format is "Module path:Decorator function". name - name of the function function - object of the function """ # If CONF.monkey_patch is not True, this function do nothing. if not CONF.monkey_patch: return if six.PY2: is_method = inspect.ismethod else: def is_method(obj): # Unbound methods became regular functions on Python 3 return inspect.ismethod(obj) or inspect.isfunction(obj) # Get list of modules and decorators for module_and_decorator in CONF.monkey_patch_modules: md_value = module_and_decorator.split(':') if len(md_value) != 2: msg = _("'monkey_patch_modules' config option is not configured " "correctly") raise Exception(msg) module, decorator_name = md_value # import decorator function decorator = importutils.import_class(decorator_name) __import__(module) # Retrieve module information using pyclbr module_data = pyclbr.readmodule_ex(module) for key, value in module_data.items(): # set the decorator for the class methods if isinstance(value, pyclbr.Class): clz = importutils.import_class("%s.%s" % (module, key)) for method, func in inspect.getmembers(clz, is_method): setattr(clz, method, decorator("%s.%s.%s" % (module, key, method), func)) # set the decorator for the function if isinstance(value, pyclbr.Function): func = importutils.import_class("%s.%s" % (module, key)) setattr(sys.modules[module], key, decorator("%s.%s" % (module, key), func)) @contextlib.contextmanager def tempdir(**kwargs): argdict = kwargs.copy() if 'dir' not in argdict: argdict['dir'] = CONF.tempdir tmpdir = tempfile.mkdtemp(**argdict) try: yield tmpdir finally: try: shutil.rmtree(tmpdir) except OSError as e: LOG.error('Could not remove tmpdir: %s', e) @privsep.monitors_priv.entrypoint def privsep_execute(*cmd, **kwargs): return processutils.execute(*cmd, **kwargs) def execute(*cmd, **kwargs): """Convenience wrapper around oslo's execute() method.""" if 'run_as_root' in kwargs and kwargs.get('run_as_root'): return privsep_execute(*cmd, **kwargs) else: return processutils.execute(*cmd, **kwargs) def synchronized(name, semaphores=None, blocking=False): def wrap(f): @six.wraps(f) def inner(*args, **kwargs): lock_str = 'masakarimonitors-%s' % name int_lock = lockutils.internal_lock(lock_str, semaphores=semaphores) msg = "Lock blocking: %s on resource %s " % (lock_str, f.__name__) """Acquiring lock: %(lock_str)s on resource """ if not int_lock.acquire(blocking=blocking): raise Exception(msg) try: return f(*args, **kwargs) finally: """Releasing lock: %(lock_str)s on resource """ int_lock.release() return inner return wrap masakari-monitors-9.0.0/masakarimonitors/introspectiveinstancemonitor/0000775000175000017500000000000013656750123026630 5ustar zuulzuul00000000000000masakari-monitors-9.0.0/masakarimonitors/introspectiveinstancemonitor/__init__.py0000664000175000017500000000000013656750014030726 0ustar zuulzuul00000000000000masakari-monitors-9.0.0/masakarimonitors/introspectiveinstancemonitor/cache.py0000664000175000017500000001176413656750014030255 0ustar zuulzuul00000000000000# Copyright (c) 2018 WindRiver Systems # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or # implied. # See the License for the specific language governing permissions and # limitations under the License. """ The code related to integration between oslo.cache module and masakarimonitors. Use 'oslo_cache.dict' i.e. dogpile.cache backend that uses dictionary for storage Dogpile consists of two subsystems, one building on top of the other. dogpile provides the concept of a dogpile lock, a control structure which allows a single thread of execution to be selected as the creator of some resource, while allowing other threads of execution to refer to the previous version of this resource as the creation proceeds; if there is no previous version, then those threads block until the object is available. """ from oslo_cache import core from oslo_config import cfg from masakarimonitors.i18n import _ WEEK = 604800 def register_cache_configurations(conf): """Register all configurations required for oslo.cache. The procedure registers all configurations required for oslo.cache. It should be called before configuring of cache region :param conf: instance of configuration :returns: updated configuration """ # register global configurations for caching core.configure(conf) # register specific configurations constraint_cache_group = cfg.OptGroup('constraint_validation_cache') constraint_cache_opts = [ cfg.IntOpt('expiration_time', default=WEEK, help=_( 'TTL, in seconds, for any cached item in the ' 'dogpile.cache region used for caching of validation ' 'constraints.')), cfg.BoolOpt("caching", default=True, help=_( 'Toggle to enable/disable caching when Orchestration ' 'Engine validates property constraints of stack.' 'During property validation with constraints ' 'Orchestration Engine caches requests to other ' 'OpenStack services. Please note that the global ' 'toggle for oslo.cache(enabled=True in [cache] group) ' 'must be enabled to use this feature.')) ] conf.register_group(constraint_cache_group) conf.register_opts(constraint_cache_opts, group=constraint_cache_group) extension_cache_group = cfg.OptGroup('service_extension_cache') extension_cache_opts = [ cfg.IntOpt('expiration_time', default=WEEK, help=_( 'TTL, in seconds, for any cached item in the ' 'dogpile.cache region used for caching of service ' 'extensions.')), cfg.BoolOpt('caching', default=True, help=_( 'Toggle to enable/disable caching when Orchestration ' 'Engine retrieves extensions from other OpenStack ' 'services. Please note that the global toggle for ' 'oslo.cache(enabled=True in [cache] group) must be ' 'enabled to use this feature.')) ] conf.register_group(extension_cache_group) conf.register_opts(extension_cache_opts, group=extension_cache_group) find_cache_group = cfg.OptGroup('resource_finder_cache') find_cache_opts = [ cfg.IntOpt('expiration_time', default=WEEK, help=_( 'TTL, in seconds, for any cached item in the ' 'dogpile.cache region used for caching of OpenStack ' 'service finder functions.')), cfg.BoolOpt('caching', default=True, help=_( 'Toggle to enable/disable caching when Orchestration ' 'Engine looks for other OpenStack service resources ' 'using name or id. Please note that the global ' 'toggle for oslo.cache(enabled=True in [cache] group) ' 'must be enabled to use this feature.')) ] conf.register_group(find_cache_group) conf.register_opts(find_cache_opts, group=find_cache_group) return conf # variable that stores an initialized cache region _REGION = None def get_cache_region(): global _REGION if not _REGION: _REGION = core.create_region() _REGION.configure('oslo_cache.dict', arguments={'expiration_time': WEEK}) core.configure_cache_region( conf=register_cache_configurations(cfg.CONF), region=_REGION) return _REGION masakari-monitors-9.0.0/masakarimonitors/introspectiveinstancemonitor/instance.py0000664000175000017500000001404413656750014031010 0ustar zuulzuul00000000000000# Copyright (c) 2018 WindRiver Systems # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. import eventlet import libvirt import socket import sys import threading import time from oslo_config import cfg from oslo_log import log as oslo_logging from oslo_utils import excutils from oslo_utils import timeutils from masakarimonitors.introspectiveinstancemonitor import qemu_utils from masakarimonitors.introspectiveinstancemonitor import scheduler from masakarimonitors import manager from masakarimonitors.objects import event_constants as ec LOG = oslo_logging.getLogger(__name__) CONF = cfg.CONF class IntrospectiveInstanceMonitorManager(manager.Manager): def __init__(self, *args, **kwargs): self.init_tgm() super(IntrospectiveInstanceMonitorManager, self).__init__( service_name="introspectiveinstancemonitor", *args, **kwargs) # This keeps track of what thread is running the event loop, # (if it is run in a background thread) self.event_loop_thread = None def _reset_journal(self, eventID, eventType, detail, uuID): """To reset the monitoring to discovery stage :param event_id: Event ID :param event_type: Event type :param detail: Event code :param domain_uuid: Uuid """ noticeType = ec.EventConstants.TYPE_VM hostname = socket.gethostname() currentTime = timeutils.utcnow() def _reset_handler(event_id, event_type, detail, domain_uuid, msg): """Reset monitoring To reset monitoring to discovery stage for the following event: libvirt.VIR_DOMAIN_EVENT_STARTED """ if (event_id == libvirt.VIR_DOMAIN_EVENT_ID_LIFECYCLE) and \ (event_type == libvirt.VIR_DOMAIN_EVENT_STARTED): LOG.debug(msg) qemu_utils.resetJournal(domain_uuid) # All Event Output if debug mode is on. msg = "libvirt Event Received.type = %s \ hostname = %s uuid = %s time = %s eventID = %d eventType = %d \ detail = %d" % ( noticeType, hostname, uuID, currentTime, eventID, eventType, detail) LOG.debug("%s", msg) try: thread = threading.Thread( _reset_handler(eventID, eventType, detail, uuID, msg) ) thread.start() except KeyError as e: LOG.error("virEventFilter KeyError: {0}".format(e)) except IndexError as e: LOG.error("virEventFilter IndexError: {0}".format(e)) except TypeError as e: LOG.error("virEventFilter TypeError: {0}".format(e)) except Exception: with excutils.save_and_reraise_exception(): LOG.error("Unexpected error: %s" % sys.exc_info()[0]) def init_tgm(self): """Manages the masakari-introspectiveinstancemonitor.""" self.TG = scheduler.ThreadGroupManager() def _vir_event_loop_native_run(self): # Directly run the event loop in the current thread while True: libvirt.virEventRunDefaultImpl() def _vir_event_loop_native_start(self): libvirt.virEventRegisterDefaultImpl() self.event_loop_thread = threading.Thread( target=self._vir_event_loop_native_run, name="lib_virt_eventLoop") self.event_loop_thread.setDaemon(True) self.event_loop_thread.start() def _my_domain_event_callback(self, conn, dom, event, detail, opaque): self._reset_journal(libvirt.VIR_DOMAIN_EVENT_ID_LIFECYCLE, event, detail, dom.UUIDString()) def _my_domain_event_reboot_callback(self, conn, dom, opaque): self._reset_journal(libvirt.VIR_DOMAIN_EVENT_ID_REBOOT, -1, -1, dom.UUIDString()) def _err_handler(self, ctxt, err): LOG.warning("Error from libvirt : %s", err[2]) def _virt_event(self, uri): # Run a background thread with the event loop self._vir_event_loop_native_start() event_callback_handlers = { libvirt.VIR_DOMAIN_EVENT_ID_LIFECYCLE: self._my_domain_event_callback, libvirt.VIR_DOMAIN_EVENT_ID_REBOOT: self._my_domain_event_reboot_callback } # Connect to libvirt - If be disconnected, reprocess. self.running = True while self.running: vc = libvirt.openReadOnly(uri) # Event callback settings callback_ids = [] for event, callback in event_callback_handlers.items(): cid = vc.domainEventRegisterAny(None, event, callback, None) callback_ids.append(cid) # Connection monitoring. vc.setKeepAlive(5, 3) while vc.isAlive() == 1 and self.running: eventlet.greenthread.sleep(1) # If connection between libvirtd was lost, # clear callback connection. LOG.warning("Libvirt Connection Closed Unexpectedly.") for cid in callback_ids: try: vc.domainEventDeregisterAny(cid) except Exception: pass vc.close() del vc time.sleep(3) def stop(self): self.running = False def main(self): """Main method. Set the URI, error handler, and executes event loop processing. """ uri = CONF.libvirt.connection_uri LOG.debug("Using uri:" + uri) # set error handler & do event loop libvirt.registerErrorHandler(self._err_handler, '_virt_event') self._virt_event(uri) masakari-monitors-9.0.0/masakarimonitors/introspectiveinstancemonitor/qemu_utils.py0000775000175000017500000003324713656750014031404 0ustar zuulzuul00000000000000# Copyright (c) 2018 WindRiver Systems # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. # Introspective instance monitoring depends on the qemu guest agent to # monitoring what inside of a VM. # # A few items to get in the way of the design: # # - After VM is active, it needs time to start qemu-guest-agent. # Before error/failure is reported, we need a discovery phase to wait # until VM is guest_pingable. # # - Debouncing is needed to enforce that masakari_notifier function # not calling twice or more for the same failure. # # After reported a masakari notification, sent_notification flag will # need to be reset when there is a QEMU LIFECYCLE event like: # STARTED_BOOTED # SUSPENDED_PAUSED import eventlet import libvirt import libvirtmod_qemu import logging import re import socket import time import traceback # Machine module contains the state machine logic, state and transition events from automaton import machines from dogpile.cache.api import NoValue from lxml import etree from oslo_config import cfg from oslo_utils import timeutils from masakarimonitors.ha import masakari from masakarimonitors.introspectiveinstancemonitor import cache from masakarimonitors.objects import event_constants as ec from masakarimonitors import utils CONF = cfg.CONF ICONF = cfg.CONF.introspectiveinstancemonitor # The VM QEMU Quest Agent states # # discovery = initial state of VM, # remains in this state until it is determined that # the VM has a qemu-agent interface enabling intrusive-instance-monitoring # # healthy = no failure event is detected # # error = An error is recorded on every audit cycle where # the VM is in the error state # # reported = a transient state # to keep track of reporting of notification # # Transitions # # Representation of a transition managed by a ``Machine`` instance. # source (str): Source state of the transition. # dest (str): Destination state of the transition. # trigger (str): The type of triggering event # that advances to the next state in the sequence. def action_on_enter(new_state, triggered_event): pass def action_on_exit(old_state, triggered_event): pass STATE_SPACE = [ { 'name': 'discovery', 'next_states': { 'guest_pingable': 'healthy', 'guest_not_pingable': 'discovery', }, 'on_enter': action_on_enter, 'on_exit': action_on_exit, }, { 'name': 'healthy', 'next_states': { 'guest_pingable': 'healthy', 'guest_not_pingable': 'error', }, 'on_enter': action_on_enter, 'on_exit': action_on_exit, }, { 'name': 'error', 'next_states': { 'report': 'reported', 'guest_pingable': 'healthy', 'guest_not_pingable': 'error', }, 'on_enter': action_on_enter, 'on_exit': action_on_exit, }, { 'name': 'reported', 'next_states': { 'guest_pingable': 'discovery', 'guest_not_pingable': 'reported', }, 'on_enter': action_on_enter, 'on_exit': action_on_exit, }, ] # Journal representation of a managed ``Machine`` instance. class Journal(machines.FiniteMachine): # Factory pattern to create a Journal object @classmethod def factory(cls, domain_uuid): jo = cls.build(STATE_SPACE) jo.default_start_state = 'discovery' jo.initialize() jo.failedCount = 0 # Conditions to reset sent_notification jo.sent_notification = False jo.domain_uuid = domain_uuid jo.lastUsed = time.time() LOG.debug(str(domain_uuid) + ':Journal:__init__:') return jo def resetState(self): self.default_start_state = 'discovery' self.initialize() self.failedCount = 0 self.sent_notification = False LOG.debug(str(self.domain_uuid) + '__resetState__:') def processEvent(self, event): self.process_event(event) self.lastUsed = time.time() def getFailedCount(self): return self.failedCount def incrementFailedCount(self): if (self.current_state == 'error'): self.failedCount += 1 def resetFailedCount(self): self.failedCount = 0 def setSentNotification(self, boolean): self.sent_notification = boolean def getSentNotification(self): return self.sent_notification # libvirt state verbose dictionary STATES = { libvirt.VIR_DOMAIN_NOSTATE: 'no state', libvirt.VIR_DOMAIN_RUNNING: 'running', libvirt.VIR_DOMAIN_BLOCKED: 'blocked on resource', libvirt.VIR_DOMAIN_PAUSED: 'paused by user', libvirt.VIR_DOMAIN_SHUTDOWN: 'being shut down', libvirt.VIR_DOMAIN_SHUTOFF: 'shut off', libvirt.VIR_DOMAIN_CRASHED: 'crashed', } LOG = logging.getLogger(__name__) def get_function_name(): return traceback.extract_stack(None, 2)[0][2] # To reset journal object by domain uuid def resetJournal(domain_uuid): # To get the VM Journal object from the dictionary # :param domain: QEMU domain UUID dict = cache.get_cache_region() jo = None if type(dict.get(domain_uuid)) is NoValue: jo = Journal.factory(domain_uuid) dict.set(domain_uuid, jo) else: jo = dict.get(domain_uuid) jo.resetState() # Qemu guest agent is used to check VM status # The checking pre-conditions are as follows: # - VM is running # - VM has Qemu guest agent installed # # then status is determined by # - VM is guest-agent-pingable or not # # Note: checkGuests function is called by the scheduler class QemuGuestAgent(object): def __init__(self): super(QemuGuestAgent, self).__init__() self.notifier = masakari.SendNotification() # _thresholdsCrossing # # We only issue a notification # to masakari-engine if the VM is 'already' in the error state # when there are consecutive guest_ping failures. # Suggested value for guest_monitoring_failure_threshold >= 3 # # Note: When operators are trying to gracefully shutdown VM, # QEMU may take time to powering-off. # E.g. When you do or # you may see that QEMU is active but monitoring may fail # due to VM is still "powering-off" # Status | Task State | Power State # ACTIVE | powering-off | Running def _thresholdsCrossing(self, domain): if (((self.getVmFsm(domain.UUIDString()).current_state) == 'error') and (self._getJournalObject(domain.UUIDString()).getFailedCount() > ICONF.guest_monitoring_failure_threshold)): LOG.debug('_thresholdsCrossing:' + domain.UUIDString()) LOG.debug(self._getJournalObject( domain.UUIDString()).getFailedCount()) return True else: return False def _masakari_notifier(self, domain_uuid): if self._getJournalObject(domain_uuid).getSentNotification(): LOG.debug('notifier.send_notification Skipped:' + domain_uuid) else: hostname = socket.gethostname() noticeType = ec.EventConstants.TYPE_VM current_time = timeutils.utcnow() event = { 'notification': { 'type': noticeType, 'hostname': hostname, 'generated_time': current_time, 'payload': { 'event': 'QEMU_GUEST_AGENT_ERROR', 'instance_uuid': domain_uuid, 'vir_domain_event': 'STOPPED_FAILED' } } } try: self.notifier.send_notification(CONF.callback.retry_max, CONF.callback.retry_interval, event) self._getJournalObject(domain_uuid).processEvent('report') self._getJournalObject(domain_uuid).setSentNotification(True) except Exception: LOG.warn('Exception :' + domain_uuid + ' @ ' + get_function_name()) pass def _qemuAgentGuestPing(self, domain, timeout, flags=0): def _no_heartbeat(domain_uuid): # Also advance the FSM self.getVmFsm(domain_uuid).processEvent('guest_not_pingable') self._getJournalObject(domain_uuid).incrementFailedCount() def _with_heartbeat(domain_uuid): #The order matters as we want to decrease the counter first self._getJournalObject(domain_uuid).resetFailedCount() self.getVmFsm(domain_uuid).processEvent('guest_pingable') def _record(result): if result is None: LOG.debug(domain.UUIDString() + '\tqemu-ga_guest-ping is not responding.') if self._thresholdsCrossing(domain): self._masakari_notifier(domain.UUIDString()) _no_heartbeat(domain.UUIDString()) else: _with_heartbeat(domain.UUIDString()) """Send a Guest Agent ping to domain """ # must pass domain._o to the c library as virDomainPtr ret = libvirtmod_qemu.virDomainQemuAgentCommand(domain._o, '{"execute": "guest-ping"}', timeout, flags) _record(ret) def _getJournalObject(self, domain_uuid): """Function: To get the dictionary :param domain: QEMU domain :return: the journal object referred by domain_uuid """ dict = cache.get_cache_region() if type(dict.get(domain_uuid)) is NoValue: jo = Journal.factory(domain_uuid) dict.set(domain_uuid, jo) return jo else: return dict.get(domain_uuid) def getVmFsm(self, domain_uuid): """Function: To get the VM Finite State Machine from the dictionary :param domain: QEMU domain :return: FSM object """ dict = cache.get_cache_region() if type(dict.get(domain_uuid)) is NoValue: jo = Journal.factory(domain_uuid) dict.set(domain_uuid, jo) return jo else: return dict.get(domain_uuid) def _hasQemuGuestAgent(self, domain): """Function: To check whether the VM has an QEMU Guest Agent by examining the qemu.guest_agent sock First check if libvirt is running or not, then sock :param domain: QEMU domain :return: true or false """ def qemuGuestAgentPathMatch(path): SOCK = ICONF.qemu_guest_agent_sock_path return re.match('%s' % SOCK, path) state, reason = domain.state() # First check if libvirt is running or not if state != libvirt.VIR_DOMAIN_RUNNING: return False xmlDesc = domain.XMLDesc() tree = etree.fromstring(xmlDesc) ''' Example
''' try: source = tree.find("devices/channel/source") if (source is not None): mode = source.get('mode') path = source.get('path') # There should be a bind for a sock file for qemu guest_agent if (qemuGuestAgentPathMatch(path) and mode == 'bind'): return True except Exception: pass return False def checkGuests(self): """Function: Check QEMU Guests Condition: VM under intrusive monitoring must have QEMU agent client configured, installed and qemu "guest-agent-pingable". """ try: conn = libvirt.open(None) # LIBVIRT_DEFAULT_URI ids = conn.listDomainsID() running = map(conn.lookupByID, ids) columns = 3 for row in map(None, *[iter(running)] * columns): for domain in row: if domain: try: if self._hasQemuGuestAgent(domain): @utils.synchronized(domain.UUIDString()) def do_qemuAgentGuestPing(domain, timeout): self._qemuAgentGuestPing(domain, timeout) do_qemuAgentGuestPing(domain, ICONF.guest_monitoring_timeout) except libvirt.libvirtError as le: LOG.warn(le) continue except Exception as e: LOG.warn(e) pass def reschedule(action, sleep_time=1): """Eventlet Sleep for the specified number of seconds. :param sleep_time: seconds to sleep; if None, no sleep; """ LOG.debug('At reschedule') if sleep_time is not None: LOG.debug('Action %s sleep for %s seconds' % ( action.id, sleep_time)) eventlet.sleep(sleep_time) def sleep(sleep_time): """Interface for sleeping.""" eventlet.sleep(sleep_time) masakari-monitors-9.0.0/masakarimonitors/introspectiveinstancemonitor/README.rst0000664000175000017500000000520713656750014030322 0ustar zuulzuul00000000000000========================================= masakarimonitors-introspectiveinstancemonitor ========================================= Introspective instance monitor for Masakari ---------------------------------------- - masakarimonitors-introspectiveinstancemonitor, provides Virtual Machine High Availability (VMHA) service for OpenStack clouds by automatically detecting the system-level failure events via QEMU Guest Agent. If it detects VM heartbeat failure events, it sends notifications to the masakari-api. - Based on the QEMU Guest Agent, masakarimonitors-introspectiveinstancemonitor aims to provide access to a system-level agent via standard qemu-ga protocol How does it work? ---------------------------------------- - libvirt and QEMU Guest Agent are used as the underlying protocol for messaging to and from VM. - The host-side qemu-agent sockets are used to detemine whether VMs are configured with QEMU Guest Agent. - qemu-guest-ping is used as the monitoring heartbeat. - For the future release, we can pass through arbitrary guest agent commands to check the health of the applications inside a VM. QEMU Guest Agent Installation notes ---------------------------------------- - Set image property: hw_qemu_guest_agent=yes. - This tells NOVA to setup the virtual serial interface thru QEMU to VM - e.g. $ openstack image create --public --disk-format qcow2 --container-format bare --file ~ubuntu/xenial-server-cloudimg-amd64-disk1.img --public --property hw_qemu_guest_agent=yes xenial-server-cloudimg * Inside VM:: $ sudo apt-get install qemu-guest-agent $ sudo systemctl start qemu-guest-agent $ ubuntu@test:~$ ps -ef | fgrep qemu $ ... /usr/sbin/qemu-ga --daemonize -m virtio-serial -p /dev/virtio-ports/org.qemu.guest_agent.0 $ ubuntu@test:~$ ls /dev/virtio-ports/ $ org.qemu.guest_agent.0 Configure masakarimonitors-introspectiveinstancemonitor ---------------------------------------------- #. Clone masakari using:: $ git clone https://github.com/openstack/masakari-monitors.git #. Create masakarimonitors directory in /etc/. #. Run setup.py from masakari-monitors:: $ sudo python setup.py install #. Copy masakarimonitors.conf and process_list.yaml files from masakari-monitors/etc/ to /etc/masakarimonitors folder and make necessary changes to the masakarimonitors.conf and process_list.yaml files. To generate the sample masakarimonitors.conf file, run the following command from the top level of the masakari-monitors directory:: $ tox -egenconfig #. To run masakari-introspectiveinstancemonitor simply use following binary:: $ masakari-introspectiveinstancemonitor masakari-monitors-9.0.0/masakarimonitors/introspectiveinstancemonitor/scheduler.py0000664000175000017500000000616313656750014031165 0ustar zuulzuul00000000000000# Copyright (c) 2018 WindRiver Systems # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import eventlet import logging from oslo_config import cfg from oslo_service import threadgroup from masakarimonitors.introspectiveinstancemonitor import qemu_utils LOG = logging.getLogger(__name__) CONF = cfg.CONF class ThreadGroupManager(object): """Thread group manager.""" def init_qemu_ga(self): self.qemuGA = qemu_utils.QemuGuestAgent() LOG.debug('Started QemuGuestAgent') def __init__(self): self.init_qemu_ga() super(ThreadGroupManager, self).__init__() self.threads = {} self.group = threadgroup.ThreadGroup() # Create dummy service task, because when there is nothing queued # on self.tg the process exits self.add_timer( CONF.introspectiveinstancemonitor.guest_monitoring_interval, self._service_task) def _service_task(self): # This is used to trigger periodic monitoring tasks self.qemuGA.checkGuests() def start(self, func, *args, **kwargs): """Run the given method in a sub-thread.""" LOG.debug('add_thread') return self.group.add_thread(func, *args, **kwargs) def add_timer(self, interval, func, *args, **kwargs): """Define a periodic task to be run in the thread group. The task will be executed in a separate green thread. Interval is from cfg.CONF.periodic_interval """ LOG.debug('group.add_timer') self.group.add_timer(interval, func, *args, **kwargs) def stop_timers(self): self.group.stop_timers() def stop(self, graceful=False): """Stop any active threads belong to this threadgroup.""" # Try to stop all threads gracefully self.group.stop(graceful) self.group.wait() # Wait for link()ed functions (i.e. lock release) threads = self.group.threads[:] links_done = dict((th, False) for th in threads) def mark_done(gt, th): links_done[th] = True for th in threads: th.link(mark_done, th) while not all(links_done.values()): eventlet.sleep() def reschedule(action, sleep_time=1): """Eventlet Sleep for the specified number of seconds. :param sleep_time: seconds to sleep; if None, no sleep; """ LOG.debug('At reschedule') if sleep_time is not None: LOG.debug('Action %s sleep for %s seconds' % ( action.id, sleep_time)) eventlet.sleep(sleep_time) def sleep(sleep_time): """Interface for sleeping.""" LOG.debug('sleep') eventlet.sleep(sleep_time) masakari-monitors-9.0.0/masakarimonitors/__init__.py0000664000175000017500000000124113656750014022703 0ustar zuulzuul00000000000000# -*- coding: utf-8 -*- # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import pbr.version __version__ = pbr.version.VersionInfo( 'masakari-monitors').version_string() masakari-monitors-9.0.0/masakarimonitors/processmonitor/0000775000175000017500000000000013656750123023663 5ustar zuulzuul00000000000000masakari-monitors-9.0.0/masakarimonitors/processmonitor/__init__.py0000664000175000017500000000000013656750014025761 0ustar zuulzuul00000000000000masakari-monitors-9.0.0/masakarimonitors/processmonitor/process.py0000664000175000017500000000623213656750014025715 0ustar zuulzuul00000000000000# Copyright(c) 2016 Nippon Telegraph and Telephone Corporation # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. import yaml import eventlet from oslo_log import log as oslo_logging import masakarimonitors.conf from masakarimonitors import manager from masakarimonitors.processmonitor.process_handler import handle_process LOG = oslo_logging.getLogger(__name__) CONF = masakarimonitors.conf.CONF class ProcessmonitorManager(manager.Manager): """Manages the masakari-processmonitor.""" def __init__(self, *args, **kwargs): super(ProcessmonitorManager, self).__init__( service_name="processmonitor", *args, **kwargs) self.process_handler = handle_process.HandleProcess() def _load_process_list(self): try: process_list = yaml.load(open(CONF.process.process_list_path)) LOG.debug("Loaded process list. %s" % process_list) return process_list except yaml.YAMLError as e: LOG.exception("YAMLError caught: %s", e) return except Exception as e: LOG.exception("Exception caught: %s", e) return def stop(self): self.running = False def main(self): """Main method.""" try: # Load process list. process_list = self._load_process_list() if process_list is None: LOG.error("Failed to load process list file.") return # Set process_list object to the process handler. self.process_handler.set_process_list(process_list) # Initial start of processes. self.process_handler.start_processes() self.running = True while self.running: # Monitor processes. down_process_list = self.process_handler.monitor_processes() if len(down_process_list) != 0: # Restart down processes. self.process_handler.restart_processes(down_process_list) else: # Since no down process, clear the restart_failure_list self.process_handler.restart_failure_list[:] = [] # Reload process list and set to the process handler. process_list = self._load_process_list() if process_list is None: LOG.error("Failed to reload process list file.") break self.process_handler.set_process_list(process_list) eventlet.greenthread.sleep(CONF.process.check_interval) except Exception as e: LOG.exception("Exception caught: %s", e) return return masakari-monitors-9.0.0/masakarimonitors/processmonitor/typescript0000664000175000017500000000000013656750014026001 0ustar zuulzuul00000000000000masakari-monitors-9.0.0/masakarimonitors/processmonitor/process_handler/0000775000175000017500000000000013656750123027036 5ustar zuulzuul00000000000000masakari-monitors-9.0.0/masakarimonitors/processmonitor/process_handler/__init__.py0000664000175000017500000000000013656750014031134 0ustar zuulzuul00000000000000masakari-monitors-9.0.0/masakarimonitors/processmonitor/process_handler/handle_process.py0000664000175000017500000001660313656750014032406 0ustar zuulzuul00000000000000# Copyright(c) 2016 Nippon Telegraph and Telephone Corporation # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. import socket import eventlet from oslo_log import log as oslo_logging from oslo_utils import timeutils import masakarimonitors.conf from masakarimonitors.ha import masakari from masakarimonitors.objects import event_constants as ec from masakarimonitors import utils LOG = oslo_logging.getLogger(__name__) CONF = masakarimonitors.conf.CONF class HandleProcess(object): """Handle process.""" def __init__(self): self.process_list = None self.restart_failure_list = [] self.notifier = masakari.SendNotification() def set_process_list(self, process_list): """Set process list object. :param process_list: process list object """ self.process_list = process_list def _execute_cmd(self, cmd_str, run_as_root): # Split command string and delete empty elements. command = cmd_str.split(' ') command = filter(lambda x: x != '', command) try: # Execute start command. out, err = utils.execute(*command, run_as_root=run_as_root) if out: msg = ("CMD '%s' output stdout: %s") % (cmd_str, out) LOG.info("%s", msg) if err: msg = ("CMD '%s' output stderr: %s") % (cmd_str, err) LOG.warning("%s", msg) return 1 except Exception as e: msg = ("CMD '%s' raised exception: %s") % (cmd_str, e) LOG.error("%s", e) return 1 return 0 def start_processes(self): """Initial start of processes. This method starts the processes using start command written in the process list. """ for process in self.process_list: cmd_str = process['start_command'] pre_cmd_str = process['pre_start_command'] post_cmd_str = process['post_start_command'] # Execute pre start command. if pre_cmd_str: ret = self._execute_cmd(pre_cmd_str, process['run_as_root']) if ret != 0: continue # Execute start command. LOG.info("Start of process with executing command: %s", cmd_str) self._execute_cmd(cmd_str, process['run_as_root']) # Execute post start command. if post_cmd_str: ret = self._execute_cmd(post_cmd_str, process['run_as_root']) def monitor_processes(self): """Monitor processes. This method monitors the processes using process name written in the process list. :returns: List of down process """ down_process_list = [] for process in self.process_list: process_name = process['process_name'] try: # Execute monitoring command. out, err = utils.execute('ps', '-ef', run_as_root=False) if process_name in out: LOG.debug("Process '%s' is found." % process_name) else: # Append down_process_list. down_process_list.append(process) LOG.warning("Process '%s' is not found.", process_name) except Exception as e: LOG.error("Monitoring command raised exception: %s", e) return down_process_list def _make_event(self, process_name): hostname = socket.gethostname() current_time = timeutils.utcnow() event = { 'notification': { 'type': ec.EventConstants.TYPE_PROCESS, 'hostname': hostname, 'generated_time': current_time, 'payload': { 'event': ec.EventConstants.EVENT_STOPPED, 'process_name': process_name } } } return event def restart_processes(self, down_process_list): """Restart processes. This method restarts the processes using restart command written in the process list. :param down_process_list: down process list object """ tmp_restart_failure_list = [] for down_process in down_process_list: # The process which failed to restart previously doesn't restart. if down_process['process_name'] in self.restart_failure_list: msg = "Process '%s' doesn't be restarted because it failed" \ " to restart previously." % down_process['process_name'] LOG.warning("%s", msg) tmp_restart_failure_list.append(down_process['process_name']) continue cmd_str = down_process['restart_command'] pre_cmd_str = down_process['pre_restart_command'] post_cmd_str = down_process['post_restart_command'] LOG.info("Restart of process with executing command: %s", cmd_str) for retries in range(0, CONF.process.restart_retries + 1): # Execute pre start command. if pre_cmd_str: ret = self._execute_cmd(pre_cmd_str, down_process['run_as_root']) if ret != 0: # Failed to restart process. eventlet.greenthread.sleep( CONF.process.restart_interval) continue # Execute start command. ret = self._execute_cmd(cmd_str, down_process['run_as_root']) if ret != 0: # Failed to restart process. eventlet.greenthread.sleep(CONF.process.restart_interval) continue # Execute post start command. if post_cmd_str: ret = self._execute_cmd(post_cmd_str, down_process['run_as_root']) if ret != 0: # Failed to restart process. eventlet.greenthread.sleep( CONF.process.restart_interval) continue # Succeeded in restarting process. break if retries == CONF.process.restart_retries: # Send a notification. event = self._make_event(down_process['process_name']) self.notifier.send_notification( CONF.process.api_retry_max, CONF.process.api_retry_interval, event) # Add the process name which failed to restart to the # failure list. tmp_restart_failure_list.append(down_process['process_name']) # Replace the old restart_failure_list with new one self.restart_failure_list = tmp_restart_failure_list[:] masakari-monitors-9.0.0/masakarimonitors/privsep.py0000664000175000017500000000156413656750014022644 0ustar zuulzuul00000000000000# Copyright(c) 2016 Nippon Telegraph and Telephone Corporation # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. from oslo_privsep import capabilities as c from oslo_privsep import priv_context monitors_priv = priv_context.PrivContext( "masakarimonitors", cfg_section="masakarimonitors_privileged", pypath=__name__ + ".monitors_priv", capabilities=[c.CAP_NET_ADMIN], ) masakari-monitors-9.0.0/masakarimonitors/version.py0000664000175000017500000000461713656750014022643 0ustar zuulzuul00000000000000# Copyright(c) 2016 Nippon Telegraph and Telephone Corporation # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. from pbr import version as pbr_version MONITORS_VENDOR = "OpenStack Foundation" MONITORS_PRODUCT = "OpenStack Masakari Monitors" MONITORS_PACKAGE = None # OS distro package version suffix loaded = False version_info = pbr_version.VersionInfo('masakari-monitors') version_string = version_info.version_string def _load_config(): # Don't load in global context, since we can't assume # these modules are accessible when distutils uses # this module from six.moves import configparser from oslo_config import cfg from oslo_log import log as logging global loaded, MONITORS_VENDOR, MONITORS_PRODUCT, MONITORS_PACKAGE if loaded: return loaded = True cfgfile = cfg.CONF.find_file("release") if cfgfile is None: return try: cfg = configparser.RawConfigParser() cfg.read(cfgfile) if cfg.has_option("Masakarimonitors", "vendor"): MONITORS_VENDOR = cfg.get("Masakarimonitors", "vendor") if cfg.has_option("Masakarimonitors", "product"): MONITORS_PRODUCT = cfg.get("Masakarimonitors", "product") if cfg.has_option("Masakarimonitors", "package"): MONITORS_PACKAGE = cfg.get("Masakarimonitors", "package") except Exception as ex: LOG = logging.getLogger(__name__) LOG.error("Failed to load %(cfgfile)s: %(ex)s", {'cfgfile': cfgfile, 'ex': ex}) def vendor_string(): _load_config() return MONITORS_VENDOR def product_string(): _load_config() return MONITORS_PRODUCT def package_string(): _load_config() return MONITORS_PACKAGE def version_string_with_package(): if package_string() is None: return version_info.version_string() else: return "%s-%s" % (version_info.version_string(), package_string()) masakari-monitors-9.0.0/masakarimonitors/common/0000775000175000017500000000000013656750123022065 5ustar zuulzuul00000000000000masakari-monitors-9.0.0/masakarimonitors/common/__init__.py0000664000175000017500000000000013656750014024163 0ustar zuulzuul00000000000000masakari-monitors-9.0.0/masakarimonitors/common/config.py0000664000175000017500000000147513656750014023712 0ustar zuulzuul00000000000000# Copyright(c) 2016 Nippon Telegraph and Telephone Corporation # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. from oslo_config import cfg from oslo_middleware import cors def set_middleware_defaults(): """Update default configuration options for oslo.middleware.""" # CORS Defaults cfg.set_defaults(cors.CORS_OPTS) masakari-monitors-9.0.0/masakarimonitors/hostmonitor/0000775000175000017500000000000013656750123023162 5ustar zuulzuul00000000000000masakari-monitors-9.0.0/masakarimonitors/hostmonitor/__init__.py0000664000175000017500000000000013656750014025260 0ustar zuulzuul00000000000000masakari-monitors-9.0.0/masakarimonitors/hostmonitor/host.py0000664000175000017500000000375013656750014024515 0ustar zuulzuul00000000000000# Copyright(c) 2016 Nippon Telegraph and Telephone Corporation # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. import os from stevedore import driver from oslo_log import log as oslo_logging import masakarimonitors.conf from masakarimonitors import manager LOG = oslo_logging.getLogger(__name__) CONF = masakarimonitors.conf.CONF class HostmonitorManager(manager.Manager): """Manages the masakari-hostmonitor.""" def __init__(self, *args, **kwargs): super(HostmonitorManager, self).__init__( service_name="hostmonitor", *args, **kwargs) self.driver = None def init_host(self): """Initialization for hostmonitor.""" try: # Determine dynamic load driver from configuration. driver_name = CONF.host.monitoring_driver # Load the driver to global. self.driver = driver.DriverManager( namespace='hostmonitor.driver', name=driver_name, invoke_on_load=True, invoke_args=(), ) except Exception as e: LOG.exception( "Exception caught during initializing hostmonitor: %s", e) os._exit(1) def stop(self): self.driver.driver.stop() def main(self): """Main method.""" try: # Call the host monitoring driver. self.driver.driver.monitor_hosts() except Exception as e: LOG.exception("Exception caught: %s", e) return masakari-monitors-9.0.0/masakarimonitors/hostmonitor/host_handler/0000775000175000017500000000000013656750123025634 5ustar zuulzuul00000000000000masakari-monitors-9.0.0/masakarimonitors/hostmonitor/host_handler/__init__.py0000664000175000017500000000000013656750014027732 0ustar zuulzuul00000000000000masakari-monitors-9.0.0/masakarimonitors/hostmonitor/host_handler/hold_host_status.py0000664000175000017500000000265613656750014031604 0ustar zuulzuul00000000000000# Copyright(c) 2016 Nippon Telegraph and Telephone Corporation # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. class HostHoldStatus(object): """Hold host status. This class holds the host status. """ def __init__(self): # host_status is dictionary like {'hostname': 'online'}. self.host_status = {} def set_host_status(self, node_state_tag): """Setter method. This method set host status by node_state tag of cib xml. :params node_state_tag: node_state tag of cib xml. """ self.host_status[node_state_tag.get('uname')] = \ node_state_tag.get('crmd') def get_host_status(self, hostname): """Getter method. This method returns the requested host status. host status is 'online' or 'offline'. :params hostname: Hostname to get status. :returns: Host status. """ return self.host_status.get(hostname) masakari-monitors-9.0.0/masakarimonitors/hostmonitor/host_handler/driver.py0000664000175000017500000000163613656750014027506 0ustar zuulzuul00000000000000# Copyright(c) 2016 Nippon Telegraph and Telephone Corporation # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. import abc import six @six.add_metaclass(abc.ABCMeta) class DriverBase(object): """Driver Base class. This class is base of monitoring hosts. """ def __init__(self): pass @abc.abstractmethod def monitor_hosts(self): """Must override monitor_hosts method.""" pass masakari-monitors-9.0.0/masakarimonitors/hostmonitor/host_handler/parse_cib_xml.py0000664000175000017500000001434413656750014031022 0ustar zuulzuul00000000000000# Copyright(c) 2016 Nippon Telegraph and Telephone Corporation # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. from xml.etree import ElementTree from oslo_log import log as oslo_logging LOG = oslo_logging.getLogger(__name__) class ParseCibXml(object): """ParseCibXml class This class parses the cib xml. """ def __init__(self): self.cib_tag = None def set_cib_xml(self, cib_xml): """Set xml.etree.ElementTree.Element object. This method recieves string of cib xml, and convert it to xml.etree.ElementTree.Element object. :params cib_xml: String of cib xml """ # Convert xml.etree.ElementTree.Element object. self.cib_tag = ElementTree.fromstring(cib_xml) def have_quorum(self): """Returns if cluster has quorum or not. :returns: 0 on no-quorum, 1 if cluster has quorum. """ return int(self.cib_tag.get('have-quorum')) def _get_status_tag(self): # status tag exists in the cib tag. child_list = self.cib_tag.getchildren() for child in child_list: if child.tag == 'status': return child return None def _get_node_states(self, status_tag): node_state_tag_list = [] # node_state tag exists in the status tag. child_list = status_tag.getchildren() for child in child_list: if child.tag == 'node_state': node_state_tag_list.append(child) return node_state_tag_list def get_node_state_tag_list(self): """Get node_state tag list. This method gets node_state tag list from cib xml. :returns: node_state tag list """ # Get status tag. status_tag = self._get_status_tag() if status_tag is None: LOG.error("Cib xml doesn't have status tag.") return [] # Get node_state tag list. node_state_tag_list = self._get_node_states(status_tag) if len(node_state_tag_list) == 0: LOG.error("Cib xml doesn't have node_state tag.") return node_state_tag_list def _parse_instance_attributes_tag(self, instance_attributes_tag, hostname): # Parse nvpair tag under the instance_attributes tag. is_target_ipmi = False ipmi_values = {} nvpair_tag_list = instance_attributes_tag.getchildren() for nvpair_tag in nvpair_tag_list: if nvpair_tag.get('name') == 'hostname' and \ nvpair_tag.get('value') == hostname: is_target_ipmi = True elif nvpair_tag.get('name') == 'ipaddr': ipmi_values['ipaddr'] = nvpair_tag.get('value') elif nvpair_tag.get('name') == 'userid': ipmi_values['userid'] = nvpair_tag.get('value') elif nvpair_tag.get('name') == 'passwd': ipmi_values['passwd'] = nvpair_tag.get('value') elif nvpair_tag.get('name') == 'interface': ipmi_values['interface'] = nvpair_tag.get('value') if is_target_ipmi is True: return ipmi_values else: return None def _parse_primitive_tag(self, primitive_tag, hostname): if primitive_tag.get('type') != 'external/ipmi': return None # Parse instance_attributes tag under the primitive tag. child_list = primitive_tag.getchildren() for child in child_list: if child.tag == 'instance_attributes': ipmi_values = self._parse_instance_attributes_tag( child, hostname) if ipmi_values is not None: return ipmi_values return None def _parse_group_tag(self, group_tag, hostname): # Parse primitive tag under the group tag. child_list = group_tag.getchildren() for child in child_list: if child.tag == 'primitive': ipmi_values = self._parse_primitive_tag(child, hostname) if ipmi_values is not None: return ipmi_values return None def get_stonith_ipmi_params(self, hostname): """Get stonith ipmi params from cib xml. This method gets params of ipmi resource agent(RA) which is set on resources tag. The resources tag exsists under the configuration tag. And it is assumed that ipmi RA belongs to some resource group. :params hostname: hostname :returns: Dictionary of ipmi RA's params. They are ipaddr, userid, passwd and interface. """ # Get configuration tag from cib tag. configuration_tag = None child_list = self.cib_tag.getchildren() for child in child_list: if child.tag == 'configuration': configuration_tag = child break if configuration_tag is None: LOG.error("Cib xml doesn't have configuration tag.") return None # Get resources tag from configuration tag. resources_tag = None child_list = configuration_tag.getchildren() for child in child_list: if child.tag == 'resources': resources_tag = child break if resources_tag is None: LOG.error("Cib xml doesn't have resources tag.") return None # They are set at nvpair tag which exists under the # instance_attributes of primitive of group tag. ipmi_values = None child_list = resources_tag.getchildren() for child in child_list: if child.tag == 'group': ipmi_values = self._parse_group_tag(child, hostname) if ipmi_values is not None: break return ipmi_values masakari-monitors-9.0.0/masakarimonitors/hostmonitor/host_handler/handle_host.py0000664000175000017500000003754213656750014030510 0ustar zuulzuul00000000000000# Copyright(c) 2016 Nippon Telegraph and Telephone Corporation # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. import socket import eventlet from oslo_log import log as oslo_logging from oslo_utils import timeutils import masakarimonitors.conf from masakarimonitors.ha import masakari import masakarimonitors.hostmonitor.host_handler.driver as driver from masakarimonitors.hostmonitor.host_handler import hold_host_status from masakarimonitors.hostmonitor.host_handler import parse_cib_xml from masakarimonitors.hostmonitor.host_handler import parse_crmmon_xml from masakarimonitors.objects import event_constants as ec from masakarimonitors import utils LOG = oslo_logging.getLogger(__name__) CONF = masakarimonitors.conf.CONF class CibSchemaCompliantTag(dict): """Create a dict which has the same attributes as a cib node tag. Given a crm node tag convert it to a dict with corresponding cib tag attributes. """ def __init__(self, crmon_entry): self['uname'] = crmon_entry.get('name') online = crmon_entry.get('online') self['crmd'] = 'online' if online == 'true' else 'offline' class HandleHost(driver.DriverBase): """Handle hosts. This class handles the host status. """ def __init__(self): super(HandleHost, self).__init__() self.my_hostname = socket.gethostname() self.xml_parser = parse_cib_xml.ParseCibXml() self.crmmon_xml_parser = parse_crmmon_xml.ParseCrmMonXml() self.status_holder = hold_host_status.HostHoldStatus() self.notifier = masakari.SendNotification() def _check_pacemaker_services(self, target_service): try: cmd_str = 'systemctl status ' + target_service command = cmd_str.split(' ') # Execute command. out, err = utils.execute(*command, run_as_root=True) if err: raise Exception return True except Exception: return False def _check_hb_line(self): """Check whether the corosync communication is normal. :returns: 0 if normal, 1 if abnormal, 2 if configuration file is wrong or neither pacemaker nor pacemaker-remote is running. """ # Check whether the pacemaker services is normal. corosync_status = self._check_pacemaker_services('corosync') pacemaker_status = self._check_pacemaker_services('pacemaker') pacemaker_remote_status = self._check_pacemaker_services( 'pacemaker_remote') if corosync_status is False or pacemaker_status is False: if pacemaker_remote_status is False: LOG.error( "Neither pacemaker nor pacemaker-remote is running.") return 2 else: LOG.info("Works on pacemaker-remote.") return 0 # Check whether the neccesary parameters are set. if CONF.host.corosync_multicast_interfaces is None or \ CONF.host.corosync_multicast_ports is None: msg = ("corosync_multicast_interfaces or " "corosync_multicast_ports is not set.") LOG.error("%s", msg) return 2 # Check whether the corosync communication is normal. corosync_multicast_interfaces = \ CONF.host.corosync_multicast_interfaces.split(',') corosync_multicast_ports = \ CONF.host.corosync_multicast_ports.split(',') if len(corosync_multicast_interfaces) != len(corosync_multicast_ports): msg = ("Incorrect parameters corosync_multicast_interfaces or " "corosync_multicast_ports.") LOG.error("%s", msg) return 2 is_nic_normal = False for num in range(0, len(corosync_multicast_interfaces)): cmd_str = ("timeout %s tcpdump -n -c 1 -p -i %s port %s") \ % (CONF.host.tcpdump_timeout, corosync_multicast_interfaces[num], corosync_multicast_ports[num]) command = cmd_str.split(' ') try: # Execute tcpdump command. out, err = utils.execute(*command, run_as_root=True) # If command doesn't raise exception, nic is normal. msg = ("Corosync communication using '%s' is normal.") \ % corosync_multicast_interfaces[num] LOG.info("%s", msg) is_nic_normal = True break except Exception: msg = ("Corosync communication using '%s' is failed.") \ % corosync_multicast_interfaces[num] LOG.warning("%s", msg) if is_nic_normal is False: LOG.error("Corosync communication is failed.") return 1 return 0 def _check_host_status_by_crmadmin(self): try: # Execute crmadmin command. out, err = utils.execute('crmadmin', '-S', self.my_hostname, run_as_root=True) if err: msg = ("crmadmin command output stderr: %s") % err raise Exception(msg) # If own host is stable status, crmadmin outputs # 'S_IDLE' or 'S_NOT_DC' if 'S_IDLE' in out or 'S_NOT_DC' in out: return 0 else: raise Exception( "crmadmin command output unexpected host status.") except Exception as e: LOG.warning("Exception caught: %s", e) LOG.warning("'%s' is unstable state on cluster.", self.my_hostname) return 1 def _get_cib_xml(self): try: # Execute cibadmin command. out, err = utils.execute('cibadmin', '--query', run_as_root=True) if err: msg = ("cibadmin command output stderr: %s") % err raise Exception(msg) except Exception as e: LOG.warning("Exception caught: %s", e) return return out def _get_crmmon_xml(self): """Get summary of cluster's current state in XML format.""" try: # Execute crm_mon command. out, err = utils.execute('crm_mon', '-X', run_as_root=True) if err: msg = ("crmmon command output stderr: %s") % err raise Exception(msg) except Exception as e: LOG.warning("Exception caught: %s", e) return return out def _is_poweroff(self, hostname): ipmi_values = self.xml_parser.get_stonith_ipmi_params(hostname) if ipmi_values is None: LOG.error("Failed to get params of ipmi RA.") return False cmd_str = ("timeout %s ipmitool -U %s -P %s -I %s -H %s " "power status") \ % (str(CONF.host.ipmi_timeout), ipmi_values['userid'], ipmi_values['passwd'], ipmi_values['interface'], ipmi_values['ipaddr']) command = cmd_str.split(' ') retry_count = 0 while True: try: # Execute ipmitool command. out, err = utils.execute(*command, run_as_root=False) if err: msg = ("ipmitool command output stderr: %s") % err raise Exception(msg) msg = ("ipmitool command output stdout: %s") % out if 'Power is off' in out: LOG.info("%s", msg) return True else: raise Exception(msg) except Exception as e: if retry_count < CONF.host.ipmi_retry_max: LOG.warning("Retry executing ipmitool command. (%s)", e) retry_count = retry_count + 1 eventlet.greenthread.sleep(CONF.host.ipmi_retry_interval) else: LOG.error("Exception caught: %s", e) return False def _make_event(self, hostname, current_status): if current_status == 'online': # Set values that host has started. event_type = ec.EventConstants.EVENT_STARTED cluster_status = current_status.upper() host_status = ec.EventConstants.HOST_STATUS_NORMAL else: # Set values that host has stopped. event_type = ec.EventConstants.EVENT_STOPPED cluster_status = current_status.upper() if not CONF.host.disable_ipmi_check: if self._is_poweroff(hostname): # Set value that host status is normal. host_status = ec.EventConstants.HOST_STATUS_NORMAL else: # Set value that host status is unknown. host_status = ec.EventConstants.HOST_STATUS_UNKNOWN else: # Set value that host status is normal. host_status = ec.EventConstants.HOST_STATUS_NORMAL current_time = timeutils.utcnow() event = { 'notification': { 'type': ec.EventConstants.TYPE_COMPUTE_HOST, 'hostname': hostname, 'generated_time': current_time, 'payload': { 'event': event_type, 'cluster_status': cluster_status, 'host_status': host_status } } } return event def _check_if_status_changed(self, node_state_tag_list): # Check if host status changed. for node_state_tag in node_state_tag_list: # hostmonitor doesn't monitor itself. if node_state_tag.get('uname') == self.my_hostname: continue # Get current status and old status. current_status = node_state_tag.get('crmd') old_status = self.status_holder.get_host_status( node_state_tag.get('uname')) # If old_status is None, This is first get of host status. if old_status is None: msg = ("Recognized '%s' as a new member of cluster." " Host status is '%s'.") \ % (node_state_tag.get('uname'), current_status) LOG.info("%s", msg) self.status_holder.set_host_status(node_state_tag) continue # Output host status. msg = ("'%s' is '%s'.") % (node_state_tag.get('uname'), current_status) LOG.info("%s", msg) # If host status changed, send a notification. if current_status != old_status: if current_status != 'online' and current_status != 'offline': # If current_status is not 'online' or 'offline', # hostmonitor doesn't send a notification. msg = ("Since host status is '%s'," " hostmonitor doesn't send a notification.") \ % current_status LOG.info("%s", msg) else: event = self._make_event(node_state_tag.get('uname'), current_status) # Send a notification. self.notifier.send_notification( CONF.host.api_retry_max, CONF.host.api_retry_interval, event) # Update host status. self.status_holder.set_host_status(node_state_tag) def _check_host_status_by_crm_mon(self): crmmon_xml = self._get_crmmon_xml() if crmmon_xml is None: # crm_mon command failure. return 1 # Set to the ParseCrmMonXml object. self.crmmon_xml_parser.set_crmmon_xml(crmmon_xml) # Get node_state tag list. node_state_tag_list = self.crmmon_xml_parser.get_node_state_tag_list() if len(node_state_tag_list) == 0: # If crmmon xml doesn't have node_state tag, # it is an unexpected result. raise Exception( "Failed to get nodes tag from crm_mon xml.") node_state_tag_list = [CibSchemaCompliantTag(n) for n in node_state_tag_list if n.get('type') == 'remote'] # Check if status changed. self._check_if_status_changed(node_state_tag_list) return 0 def _check_host_status_by_cibadmin(self): # Get xml of cib info. cib_xml = self._get_cib_xml() if cib_xml is None: # cibadmin command failure. return 1 # Set to the ParseCibXml object. self.xml_parser.set_cib_xml(cib_xml) # Check if pacemaker cluster have quorum. if self.xml_parser.have_quorum() == 0: msg = "Pacemaker cluster doesn't have quorum." LOG.warning("%s", msg) # Get node_state tag list. node_state_tag_list = self.xml_parser.get_node_state_tag_list() if len(node_state_tag_list) == 0: # If cib xml doesn't have node_state tag, # it is an unexpected result. raise Exception( "Failed to get node_state tag from cib xml.") # Check if status changed. self._check_if_status_changed(node_state_tag_list) return 0 def stop(self): self.running = False def monitor_hosts(self): """Host monitoring main method. This method monitors hosts. """ try: self.running = True while self.running: # Check whether corosync communication between hosts # is normal. ret = self._check_hb_line() if ret == 1: # Because my host may be fenced by stonith due to split # brain condition, sleep for a certain time. eventlet.greenthread.sleep(CONF.host.stonith_wait) elif ret == 2: LOG.warning("hostmonitor skips monitoring hosts.") eventlet.greenthread.sleep(CONF.host.monitoring_interval) continue # Check the host status is stable or unstable by crmadmin. # It only checks when this process runs on the full cluster # stack of corosync. pacemaker_remote_status = self._check_pacemaker_services( 'pacemaker_remote') if pacemaker_remote_status is False: if self._check_host_status_by_crmadmin() != 0: LOG.warning("hostmonitor skips monitoring hosts.") eventlet.greenthread.sleep( CONF.host.monitoring_interval) continue # Check the host status is online or offline. if CONF.host.restrict_to_remotes: status_func = self._check_host_status_by_crm_mon else: status_func = self._check_host_status_by_cibadmin if status_func() != 0: LOG.warning("hostmonitor skips monitoring hosts.") eventlet.greenthread.sleep(CONF.host.monitoring_interval) continue eventlet.greenthread.sleep(CONF.host.monitoring_interval) except Exception as e: LOG.exception("Exception caught: %s", e) return masakari-monitors-9.0.0/masakarimonitors/hostmonitor/host_handler/parse_crmmon_xml.py0000664000175000017500000000467213656750014031563 0ustar zuulzuul00000000000000# Copyright(c) 2019 Canonical Ltd # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. from xml.etree import ElementTree from oslo_log import log as oslo_logging LOG = oslo_logging.getLogger(__name__) class ParseCrmMonXml(object): """ParseCrmMonXml class This class parses the crmmon xml. """ def __init__(self): self.crmmon_tag = None def set_crmmon_xml(self, crmmon_xml): """Set xml.etree.ElementTree.Element object. This method receives string of crmmon xml, and convert it to xml.etree.ElementTree.Element object. :params crmmon_xml: String of crmmon xml """ # Convert xml.etree.ElementTree.Element object. self.crmmon_tag = ElementTree.fromstring(crmmon_xml) def _get_nodes(self): # status tag exists in the crmmon tag. if self.crmmon_tag is None: return None child_list = self.crmmon_tag.getchildren() for child in child_list: if child.tag == 'nodes': return child return None def _get_node_states(self, nodes_tag): node_state_tag_list = [] # node_state tag exists in the status tag. child_list = nodes_tag.getchildren() for child in child_list: if child.tag == 'node': node_state_tag_list.append(child) return node_state_tag_list def get_node_state_tag_list(self): """Get node_state tag list. This method gets node_state tag list from crmmon xml. :returns: node_state tag list """ # Get status tag. nodes_tag = self._get_nodes() if nodes_tag is None: LOG.error("crm_mon xml doesn't have nodes tag.") return [] # Get node_state tag list. node_state_tag_list = self._get_node_states(nodes_tag) if len(node_state_tag_list) == 0: LOG.error("crm_mon xml doesn't have online tag.") return node_state_tag_list masakari-monitors-9.0.0/masakarimonitors/config.py0000664000175000017500000000221713656750014022415 0ustar zuulzuul00000000000000# Copyright(c) 2016 Nippon Telegraph and Telephone Corporation # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. from oslo_log import log import masakarimonitors.conf from masakarimonitors import version CONF = masakarimonitors.conf.CONF def parse_args(argv, default_config_files=None): log.register_options(CONF) # We use the oslo.log default log levels which includes suds=INFO # and add only the extra levels that Masakari needs log.set_defaults(default_log_levels=log.get_default_log_levels()) CONF(argv[1:], project='masakarimonitors', version=version.version_string(), default_config_files=default_config_files) masakari-monitors-9.0.0/masakarimonitors/hacking/0000775000175000017500000000000013656750123022201 5ustar zuulzuul00000000000000masakari-monitors-9.0.0/masakarimonitors/hacking/__init__.py0000664000175000017500000000000013656750014024277 0ustar zuulzuul00000000000000masakari-monitors-9.0.0/masakarimonitors/hacking/checks.py0000664000175000017500000000703513656750014024017 0ustar zuulzuul00000000000000# Copyright (c) 2017, NTT Data # All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import re from hacking import core """ Guidelines for writing new hacking checks - Use only for masakarimonitors specific tests. OpenStack general tests should be submitted to the common 'hacking' module. - Pick numbers in the range M3xx. Find the current test with the highest allocated number and then pick the next value. - Keep the test method code in the source file ordered based on the M3xx value. - List the new rule in the top level HACKING.rst file - Add test cases for each new rule to masakarimonitors/tests/unit/ test_hacking.py """ UNDERSCORE_IMPORT_FILES = [] translated_log = re.compile( r"(.)*LOG\.(audit|error|info|critical|exception)" r"\(\s*_\(\s*('|\")") string_translation = re.compile(r"[^_]*_\(\s*('|\")") underscore_import_check = re.compile(r"(.)*import _(.)*") underscore_import_check_multi = re.compile(r"(.)*i18n\s+import(.)* _, (.)*") # We need this for cases where they have created their own _ function. custom_underscore_check = re.compile(r"(.)*_\s*=\s*(.)*") log_translation = re.compile( r"(.)*LOG\." r"(audit|debug|error|info|warn|warning|critical|exception)" r"\(" r"(_|_LC|_LE|_LI|_LW)" r"\(") yield_not_followed_by_space = re.compile(r"^\s*yield(?:\(|{|\[|\"|').*$") @core.flake8ext def check_explicit_underscore_import(logical_line, filename): """Check for explicit import of the _ function We need to ensure that any files that are using the _() function to translate logs are explicitly importing the _ function. We can't trust unit test to catch whether the import has been added so we need to check for it here. """ # Build a list of the files that have _ imported. No further # checking needed once it is found. for file in UNDERSCORE_IMPORT_FILES: if file in filename: return if (underscore_import_check.match(logical_line) or underscore_import_check_multi.match(logical_line) or custom_underscore_check.match(logical_line)): UNDERSCORE_IMPORT_FILES.append(filename) elif(translated_log.match(logical_line) or string_translation.match(logical_line)): yield (0, "M301: Found use of _() without explicit import of _ !") @core.flake8ext def no_translate_logs(logical_line): """Check that logging doesn't translate messages M302 """ if log_translation.match(logical_line): yield (0, "M302 Don't translate log messages!") @core.flake8ext def yield_followed_by_space(logical_line): """Yield should be followed by a space. Yield should be followed by a space to clarify that yield is not a function. Adding a space may force the developer to rethink if there are unnecessary parentheses in the written code. Not correct: yield(x), yield(a, b) Correct: yield x, yield (a, b), yield a, b M303 """ if yield_not_followed_by_space.match(logical_line): yield (0, "M303: Yield keyword should be followed by a space.") masakari-monitors-9.0.0/masakarimonitors/tests/0000775000175000017500000000000013656750123021737 5ustar zuulzuul00000000000000masakari-monitors-9.0.0/masakarimonitors/tests/__init__.py0000664000175000017500000000000013656750014024035 0ustar zuulzuul00000000000000masakari-monitors-9.0.0/masakarimonitors/tests/unit/0000775000175000017500000000000013656750123022716 5ustar zuulzuul00000000000000masakari-monitors-9.0.0/masakarimonitors/tests/unit/introspectiveinstancemonitor/0000775000175000017500000000000013656750123030751 5ustar zuulzuul00000000000000masakari-monitors-9.0.0/masakarimonitors/tests/unit/introspectiveinstancemonitor/__init__.py0000775000175000017500000000000013656750014033052 0ustar zuulzuul00000000000000././@LongLink0000000000000000000000000000015100000000000011212 Lustar 00000000000000masakari-monitors-9.0.0/masakarimonitors/tests/unit/introspectiveinstancemonitor/test_monitor_manager.pymasakari-monitors-9.0.0/masakarimonitors/tests/unit/introspectiveinstancemonitor/test_monitor_manage0000664000175000017500000000257013656750014034735 0ustar zuulzuul00000000000000# Copyright(c) 2018 WindRiver Systems # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. import eventlet import libvirt import testtools from unittest import mock from masakarimonitors.introspectiveinstancemonitor import instance eventlet.monkey_patch(os=False) class TestMonitorManager(testtools.TestCase): def setUp(self): super(TestMonitorManager, self).setUp() @mock.patch.object(libvirt, 'virEventRunDefaultImpl') def test_vir_event_loop_native_run(self, mock_virEventRunDefaultImpl): mock_virEventRunDefaultImpl.side_effect = Exception("Test exception.") obj = instance.IntrospectiveInstanceMonitorManager() exception_flag = False try: obj._vir_event_loop_native_run() except Exception: exception_flag = True self.assertTrue(exception_flag) mock_virEventRunDefaultImpl.assert_called_once() masakari-monitors-9.0.0/masakarimonitors/tests/unit/introspectiveinstancemonitor/test_qemu_utils.py0000775000175000017500000000767513656750014034572 0ustar zuulzuul00000000000000# Copyright(c) 2018 Nippon Telegraph and Telephone Corporation # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. import libvirt import testtools from unittest import mock import uuid from masakarimonitors.introspectiveinstancemonitor import instance as object from masakarimonitors.introspectiveinstancemonitor import qemu_utils class TestQemuUtils(testtools.TestCase): def setup(self): super(TestQemuUtils, self).setUp() @mock.patch.object(qemu_utils.libvirt, 'virDomain') def test_getVmFsm(self, mock_domain): """To test the state machines Initial stage should be dicovery and will advance to a healthy stage after enough pingable events. Also, it should reach an error stage from healthy after a pingable event failure. """ reference = qemu_utils.QemuGuestAgent() mock_domain.UUID.return_value = uuid.uuid4() reference.getVmFsm(mock_domain) self.assertEqual(reference.getVmFsm(mock_domain).current_state, 'discovery') reference.getVmFsm(mock_domain).process_event('guest_not_pingable') self.assertEqual(reference.getVmFsm(mock_domain).current_state, 'discovery') reference.getVmFsm(mock_domain).process_event('guest_pingable') self.assertEqual(reference.getVmFsm(mock_domain).current_state, 'healthy') reference.getVmFsm(mock_domain).process_event('guest_pingable') self.assertEqual(reference.getVmFsm(mock_domain).current_state, 'healthy') reference.getVmFsm(mock_domain).process_event('guest_not_pingable') self.assertEqual(reference.getVmFsm(mock_domain).current_state, 'error') @mock.patch.object(qemu_utils.libvirt, 'virDomain') def test_hasQemuGuestAgent(self, mock_domain): mock_domain.UUID.return_value = 'testuuid' mock_domain.state.return_value = libvirt.VIR_DOMAIN_RUNNING, 'reason' mock_domain.XMLDesc.return_value = """ """ obj = qemu_utils.QemuGuestAgent() self.assertFalse(obj._hasQemuGuestAgent(mock_domain)) mock_domain.XMLDesc.return_value = """
""" obj = qemu_utils.QemuGuestAgent() self.assertTrue(obj._hasQemuGuestAgent(mock_domain)) @mock.patch.object(qemu_utils, 'resetJournal') def test_resetJournal(self, mock_resetJournal): mock_resetJournal.return_value = None obj = object.IntrospectiveInstanceMonitorManager() event_id = 0 domain_uuid = uuid.uuid4() event_type = libvirt.VIR_DOMAIN_EVENT_STARTED detail = libvirt.VIR_DOMAIN_EVENT_STARTED_BOOTED obj._reset_journal(event_id, event_type, detail, domain_uuid) mock_resetJournal.assert_called_once_with(domain_uuid) masakari-monitors-9.0.0/masakarimonitors/tests/unit/__init__.py0000664000175000017500000000000013656750014025014 0ustar zuulzuul00000000000000masakari-monitors-9.0.0/masakarimonitors/tests/unit/processmonitor/0000775000175000017500000000000013656750123026004 5ustar zuulzuul00000000000000masakari-monitors-9.0.0/masakarimonitors/tests/unit/processmonitor/__init__.py0000664000175000017500000000000013656750014030102 0ustar zuulzuul00000000000000masakari-monitors-9.0.0/masakarimonitors/tests/unit/processmonitor/test_process.py0000664000175000017500000001445013656750014031076 0ustar zuulzuul00000000000000# Copyright(c) 2016 Nippon Telegraph and Telephone Corporation # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. import testtools from unittest import mock import yaml import eventlet from masakarimonitors.processmonitor import process as processmonitor_manager from masakarimonitors.processmonitor.process_handler import handle_process eventlet.monkey_patch(os=False) MOCK_PROCESS_LIST = [ { 'process_name': 'mock_process_name_A', 'start_command': 'mock_start_command', 'pre_start_command': 'mock_pre_start_command', 'post_start_command': 'mock_post_start_command', 'restart_command': 'mock_restart_command', 'pre_restart_command': 'mock_pre_restart_command', 'post_restart_command': 'mock_post_restart_command', 'run_as_root': True }, { 'id': 2, 'process_name': 'mock_process_name_B', 'start_command': 'mock_start_command', 'pre_start_command': 'mock_pre_start_command', 'post_start_command': 'mock_post_start_command', 'restart_command': 'mock_restart_command', 'pre_restart_command': 'mock_pre_restart_command', 'post_restart_command': 'mock_post_restart_command', 'run_as_root': True }, ] MOCK_DOWN_PROCESS_LIST = [ { 'process_name': 'mock_process_name_A', 'start_command': 'mock_start_command', 'pre_start_command': 'mock_pre_start_command', 'post_start_command': 'mock_post_start_command', 'restart_command': 'mock_restart_command', 'pre_restart_command': 'mock_pre_restart_command', 'post_restart_command': 'mock_post_restart_command', 'run_as_root': True }, ] class TestProcessmonitorManager(testtools.TestCase): def setUp(self): super(TestProcessmonitorManager, self).setUp() def _get_mock_process_list(self, call_count): if call_count == 0: return MOCK_PROCESS_LIST else: return @mock.patch.object(eventlet.greenthread, 'sleep') @mock.patch.object(handle_process.HandleProcess, 'restart_processes') @mock.patch.object(handle_process.HandleProcess, 'monitor_processes') @mock.patch.object(handle_process.HandleProcess, 'start_processes') @mock.patch.object(handle_process.HandleProcess, 'set_process_list') @mock.patch.object(yaml, 'load') @mock.patch('six.moves.builtins.open') def test_main(self, mock_file, mock_load, mock_set_process_list, mock_start_processes, mock_monitor_processes, mock_restart_processes, mock_sleep): mock_load.side_effect = [self._get_mock_process_list(0), self._get_mock_process_list(0), self._get_mock_process_list(1)] mock_set_process_list.return_value = None mock_start_processes.return_value = None mock_monitor_processes.side_effect = [MOCK_DOWN_PROCESS_LIST, []] mock_restart_processes.return_value = None mock_sleep.return_value = None obj = processmonitor_manager.ProcessmonitorManager() obj.main() mock_set_process_list.assert_called_with(MOCK_PROCESS_LIST) mock_start_processes.assert_called_once_with() self.assertEqual(2, mock_monitor_processes.call_count) mock_restart_processes.assert_called_once_with(MOCK_DOWN_PROCESS_LIST) @mock.patch.object(handle_process.HandleProcess, 'restart_processes') @mock.patch.object(handle_process.HandleProcess, 'monitor_processes') @mock.patch.object(handle_process.HandleProcess, 'start_processes') @mock.patch.object(handle_process.HandleProcess, 'set_process_list') @mock.patch.object(yaml, 'load') @mock.patch('six.moves.builtins.open') def test_main_exception(self, mock_file, mock_load, mock_set_process_list, mock_start_processes, mock_monitor_processes, mock_restart_processes): mock_load.return_value = self._get_mock_process_list(0) mock_set_process_list.return_value = None mock_start_processes.side_effect = Exception("Test exception.") obj = processmonitor_manager.ProcessmonitorManager() obj.main() mock_set_process_list.assert_called_once_with(MOCK_PROCESS_LIST) mock_start_processes.assert_called_once_with() mock_monitor_processes.assert_not_called() mock_restart_processes.assert_not_called() @mock.patch.object(handle_process.HandleProcess, 'set_process_list') @mock.patch.object(yaml, 'load') @mock.patch('six.moves.builtins.open') def test_load_process_list_yaml_error(self, mock_file, mock_load, mock_set_process_list): mock_load.side_effect = yaml.YAMLError obj = processmonitor_manager.ProcessmonitorManager() obj.main() mock_set_process_list.assert_not_called() @mock.patch.object(handle_process.HandleProcess, 'set_process_list') @mock.patch.object(yaml, 'load') @mock.patch('six.moves.builtins.open') def test_load_process_list_exception(self, mock_file, mock_load, mock_set_process_list): mock_load.side_effect = Exception("Test exception.") obj = processmonitor_manager.ProcessmonitorManager() obj.main() mock_set_process_list.assert_not_called() def test_stop(self): obj = processmonitor_manager.ProcessmonitorManager() obj.stop() self.assertFalse(obj.running) masakari-monitors-9.0.0/masakarimonitors/tests/unit/processmonitor/process_handler/0000775000175000017500000000000013656750123031157 5ustar zuulzuul00000000000000masakari-monitors-9.0.0/masakarimonitors/tests/unit/processmonitor/process_handler/__init__.py0000664000175000017500000000000013656750014033255 0ustar zuulzuul00000000000000././@LongLink0000000000000000000000000000015200000000000011213 Lustar 00000000000000masakari-monitors-9.0.0/masakarimonitors/tests/unit/processmonitor/process_handler/test_handle_process.pymasakari-monitors-9.0.0/masakarimonitors/tests/unit/processmonitor/process_handler/test_handle_proce0000664000175000017500000003545713656750014034601 0ustar zuulzuul00000000000000# Copyright(c) 2016 Nippon Telegraph and Telephone Corporation # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. import socket import testtools from unittest import mock import eventlet from oslo_utils import timeutils import masakarimonitors.conf from masakarimonitors.ha import masakari from masakarimonitors.objects import event_constants as ec from masakarimonitors.processmonitor.process_handler import handle_process from masakarimonitors import utils CONF = masakarimonitors.conf.CONF eventlet.monkey_patch(os=False) MOCK_PROCESS_LIST = [ { 'id': 1, 'process_name': 'mock_process_name_A', 'start_command': 'mock_start_command', 'pre_start_command': 'mock_pre_start_command', 'post_start_command': 'mock_post_start_command', 'restart_command': 'mock_restart_command', 'pre_restart_command': 'mock_pre_restart_command', 'post_restart_command': 'mock_post_restart_command', 'run_as_root': True }, ] MOCK_DOWN_PROCESS_LIST = [ { 'id': 1, 'process_name': 'mock_process_name_A', 'start_command': 'mock_start_command', 'pre_start_command': 'mock_pre_start_command', 'post_start_command': 'mock_post_start_command', 'restart_command': 'mock_restart_command', 'pre_restart_command': 'mock_pre_restart_command', 'post_restart_command': 'mock_post_restart_command', 'run_as_root': True }, ] PS_RESULT = "\n" \ "UID PID PPID C STIME TTY TIME CMD\n" \ "root 11187 1 0 18:52 ? 00:00:00 mock_process_name_A\n" class TestHandleProcess(testtools.TestCase): def setUp(self): super(TestHandleProcess, self).setUp() def test_set_process_list(self): process_list = MOCK_PROCESS_LIST obj = handle_process.HandleProcess() obj.set_process_list(process_list) self.assertEqual(process_list, obj.process_list) @mock.patch.object(utils, 'execute') def test_start_processes(self, mock_execute): process_list = MOCK_PROCESS_LIST obj = handle_process.HandleProcess() obj.set_process_list(process_list) mock_execute.side_effect = [('test_stdout', ''), ('test_stdout', ''), ('test_stdout', 'test_stderr')] obj.start_processes() mock_execute.assert_any_call( MOCK_PROCESS_LIST[0].get('pre_start_command'), run_as_root=MOCK_PROCESS_LIST[0].get('run_as_root')) mock_execute.assert_any_call( MOCK_PROCESS_LIST[0].get('start_command'), run_as_root=MOCK_PROCESS_LIST[0].get('run_as_root')) mock_execute.assert_any_call( MOCK_PROCESS_LIST[0].get('post_start_command'), run_as_root=MOCK_PROCESS_LIST[0].get('run_as_root')) @mock.patch.object(utils, 'execute') def test_start_processes_pre_cmd_fail(self, mock_execute): process_list = MOCK_PROCESS_LIST obj = handle_process.HandleProcess() obj.set_process_list(process_list) mock_execute.return_value = ('test_stdout', 'test_stderr') obj.start_processes() mock_execute.assert_called_once_with( MOCK_PROCESS_LIST[0].get('pre_start_command'), run_as_root=MOCK_PROCESS_LIST[0].get('run_as_root')) @mock.patch.object(utils, 'execute') def test_monitor_processes(self, mock_execute): process_list = MOCK_PROCESS_LIST obj = handle_process.HandleProcess() obj.set_process_list(process_list) mock_execute.return_value = (PS_RESULT, '') down_process_list = obj.monitor_processes() self.assertEqual([], down_process_list) mock_execute.assert_called_once_with( 'ps', '-ef', run_as_root=False) @mock.patch.object(utils, 'execute') def test_monitor_processes_not_found(self, mock_execute): process_list = MOCK_PROCESS_LIST obj = handle_process.HandleProcess() obj.set_process_list(process_list) mock_execute.return_value = ('', '') down_process_list = obj.monitor_processes() self.assertEqual(MOCK_PROCESS_LIST, down_process_list) mock_execute.assert_called_once_with( 'ps', '-ef', run_as_root=False) @mock.patch.object(utils, 'execute') def test_monitor_processes_exception(self, mock_execute): process_list = MOCK_PROCESS_LIST obj = handle_process.HandleProcess() obj.set_process_list(process_list) mock_execute.side_effect = Exception("Test exception.") down_process_list = obj.monitor_processes() self.assertEqual([], down_process_list) @mock.patch.object(utils, 'execute') def test_restart_processes(self, mock_execute): process_list = MOCK_PROCESS_LIST obj = handle_process.HandleProcess() obj.set_process_list(process_list) down_process_list = MOCK_DOWN_PROCESS_LIST mock_execute.side_effect = [('test_stdout', ''), ('test_stdout', ''), ('test_stdout', '')] obj.restart_processes(down_process_list) mock_execute.assert_any_call( down_process_list[0].get('pre_restart_command'), run_as_root=down_process_list[0].get('run_as_root')) mock_execute.assert_any_call( down_process_list[0].get('restart_command'), run_as_root=down_process_list[0].get('run_as_root')) mock_execute.assert_any_call( down_process_list[0].get('post_restart_command'), run_as_root=down_process_list[0].get('run_as_root')) self.assertEqual([], obj.restart_failure_list) @mock.patch.object(utils, 'execute') def test_restart_processes_failed_to_restart_previously( self, mock_execute): process_list = MOCK_PROCESS_LIST obj = handle_process.HandleProcess() obj.set_process_list(process_list) restart_failure_list = [MOCK_DOWN_PROCESS_LIST[0].get('process_name')] obj.restart_failure_list = restart_failure_list down_process_list = MOCK_DOWN_PROCESS_LIST obj.restart_processes(down_process_list) self.assertEqual(restart_failure_list, obj.restart_failure_list) mock_execute.assert_not_called() @mock.patch.object(masakari.SendNotification, 'send_notification') @mock.patch.object(timeutils, 'utcnow') @mock.patch.object(eventlet.greenthread, 'sleep') @mock.patch.object(utils, 'execute') def test_restart_processes_pre_restart_command_retry_over( self, mock_execute, mock_sleep, mock_utcnow, mock_send_notification): process_list = MOCK_PROCESS_LIST obj = handle_process.HandleProcess() obj.set_process_list(process_list) down_process_list = MOCK_DOWN_PROCESS_LIST mock_execute.side_effect = [('test_stdout', 'test_stderr'), ('test_stdout', 'test_stderr'), ('test_stdout', 'test_stderr'), ('test_stdout', 'test_stderr')] mock_sleep.return_value = None current_time = timeutils.utcnow() mock_utcnow.return_value = current_time mock_send_notification.return_value = None obj.restart_processes(down_process_list) pre_execute_count = CONF.process.restart_retries + 1 self.assertEqual(pre_execute_count, mock_execute.call_count) for var in range(0, mock_execute.call_count): args, kwargs = mock_execute.call_args_list[var] self.assertEqual( (down_process_list[0].get('pre_restart_command'),), args) self.assertEqual({'run_as_root': True}, kwargs) event = { 'notification': { 'type': ec.EventConstants.TYPE_PROCESS, 'hostname': socket.gethostname(), 'generated_time': current_time, 'payload': { 'event': ec.EventConstants.EVENT_STOPPED, 'process_name': down_process_list[0].get('process_name') } } } mock_send_notification.assert_called_once_with( CONF.process.api_retry_max, CONF.process.api_retry_interval, event) self.assertEqual( [down_process_list[0].get('process_name')], obj.restart_failure_list) @mock.patch.object(masakari.SendNotification, 'send_notification') @mock.patch.object(timeutils, 'utcnow') @mock.patch.object(eventlet.greenthread, 'sleep') @mock.patch.object(utils, 'execute') def test_restart_processes_restart_command_retry_over( self, mock_execute, mock_sleep, mock_utcnow, mock_send_notification): process_list = MOCK_PROCESS_LIST obj = handle_process.HandleProcess() obj.set_process_list(process_list) down_process_list = MOCK_DOWN_PROCESS_LIST mock_execute.side_effect = [('test_stdout', ''), ('test_stdout', 'test_stderr'), ('test_stdout', ''), ('test_stdout', 'test_stderr'), ('test_stdout', ''), ('test_stdout', 'test_stderr'), ('test_stdout', ''), ('test_stdout', 'test_stderr')] mock_sleep.return_value = None current_time = timeutils.utcnow() mock_utcnow.return_value = current_time mock_send_notification.return_value = None obj.restart_processes(down_process_list) pre_execute_count = CONF.process.restart_retries + 1 execute_count = CONF.process.restart_retries + 1 total_execute_count = pre_execute_count + execute_count self.assertEqual(total_execute_count, mock_execute.call_count) for var in range(0, mock_execute.call_count): # Execute order of restart_command is the second. execute_order = 2 if (var + 1) % execute_order == 0: args, kwargs = mock_execute.call_args_list[var] self.assertEqual( (down_process_list[0].get('restart_command'),), args) self.assertEqual({'run_as_root': True}, kwargs) event = { 'notification': { 'type': ec.EventConstants.TYPE_PROCESS, 'hostname': socket.gethostname(), 'generated_time': current_time, 'payload': { 'event': ec.EventConstants.EVENT_STOPPED, 'process_name': down_process_list[0].get('process_name') } } } mock_send_notification.assert_called_once_with( CONF.process.api_retry_max, CONF.process.api_retry_interval, event) self.assertEqual( [down_process_list[0].get('process_name')], obj.restart_failure_list) @mock.patch.object(masakari.SendNotification, 'send_notification') @mock.patch.object(timeutils, 'utcnow') @mock.patch.object(eventlet.greenthread, 'sleep') @mock.patch.object(utils, 'execute') def test_restart_processes_post_restart_command_retry_over( self, mock_execute, mock_sleep, mock_utcnow, mock_send_notification): process_list = MOCK_PROCESS_LIST obj = handle_process.HandleProcess() obj.set_process_list(process_list) down_process_list = MOCK_DOWN_PROCESS_LIST mock_execute.side_effect = [('test_stdout', ''), ('test_stdout', ''), ('test_stdout', 'test_stderr'), ('test_stdout', ''), ('test_stdout', ''), ('test_stdout', 'test_stderr'), ('test_stdout', ''), ('test_stdout', ''), ('test_stdout', 'test_stderr'), ('test_stdout', ''), ('test_stdout', ''), ('test_stdout', 'test_stderr')] mock_sleep.return_value = None current_time = timeutils.utcnow() mock_utcnow.return_value = current_time mock_send_notification.return_value = None obj.restart_processes(down_process_list) pre_execute_count = CONF.process.restart_retries + 1 execute_count = CONF.process.restart_retries + 1 post_execute_count = CONF.process.restart_retries + 1 total_execute_count = \ pre_execute_count + execute_count + post_execute_count self.assertEqual(total_execute_count, mock_execute.call_count) for var in range(0, mock_execute.call_count): # Execute order of restart_command is the third. execute_order = 3 if (var + 1) % execute_order == 0: args, kwargs = mock_execute.call_args_list[var] self.assertEqual( (down_process_list[0].get('post_restart_command'),), args) self.assertEqual({'run_as_root': True}, kwargs) event = { 'notification': { 'type': ec.EventConstants.TYPE_PROCESS, 'hostname': socket.gethostname(), 'generated_time': current_time, 'payload': { 'event': ec.EventConstants.EVENT_STOPPED, 'process_name': down_process_list[0].get('process_name') } } } mock_send_notification.assert_called_once_with( CONF.process.api_retry_max, CONF.process.api_retry_interval, event) self.assertEqual( [down_process_list[0].get('process_name')], obj.restart_failure_list) @mock.patch.object(utils, 'execute') def test_execute_cmd_exception(self, mock_execute): mock_execute.side_effect = Exception("Test exception.") obj = handle_process.HandleProcess() ret = obj._execute_cmd(MOCK_PROCESS_LIST[0].get('start_command'), MOCK_PROCESS_LIST[0].get('run_as_root')) self.assertEqual(ret, 1) masakari-monitors-9.0.0/masakarimonitors/tests/unit/test_hacking.py0000664000175000017500000001440413656750014025735 0ustar zuulzuul00000000000000# Copyright 2017 NTT Data. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import textwrap from unittest import mock import pycodestyle import testtools from masakarimonitors.hacking import checks class HackingTestCase(testtools.TestCase): """This class tests the hacking checks in masakarimonitors.hacking.checks by passing strings to the check methods like the pycodestyle/flake8 parser would. The parser loops over each line in the file and then passes the parameters to the check method. The parameter names in the check method dictate what type of object is passed to the check method. The parameter types are:: logical_line: A processed line with the following modifications: - Multi-line statements converted to a single line. - Stripped left and right. - Contents of strings replaced with "xxx" of same length. - Comments removed. physical_line: Raw line of text from the input file. lines: a list of the raw lines from the input file tokens: the tokens that contribute to this logical line line_number: line number in the input file total_lines: number of lines in the input file blank_lines: blank lines before this one indent_char: indentation character in this file (" " or "\t") indent_level: indentation (with tabs expanded to multiples of 8) previous_indent_level: indentation on previous line previous_logical: previous logical line filename: Path of the file being run through pycodestyle When running a test on a check method the return will be False/None if there is no violation in the sample input. If there is an error a tuple is returned with a position in the line, and a message. So to check the result just assertTrue if the check is expected to fail and assertFalse if it should pass. """ # We are patching pycodestyle so that only the check under test is actually # installed. @mock.patch('pycodestyle._checks', {'physical_line': {}, 'logical_line': {}, 'tree': {}}) def _run_check(self, code, checker, filename=None): pycodestyle.register_check(checker) lines = textwrap.dedent(code).strip().splitlines(True) checker = pycodestyle.Checker(filename=filename, lines=lines) checker.check_all() checker.report._deferred_print.sort() return checker.report._deferred_print def _assert_has_errors(self, code, checker, expected_errors=None, filename=None): actual_errors = [e[:3] for e in self._run_check(code, checker, filename)] self.assertEqual(expected_errors or [], actual_errors) def _assert_has_no_errors(self, code, checker, filename=None): self._assert_has_errors(code, checker, filename=filename) def test_check_explicit_underscore_import(self): self.assertEqual(len(list(checks.check_explicit_underscore_import( "LOG.info(_('My info message'))", "masakarimonitors/tests/other_files.py"))), 1) self.assertEqual(len(list(checks.check_explicit_underscore_import( "msg = _('My message')", "masakarimonitors/tests/other_files.py"))), 1) self.assertEqual(len(list(checks.check_explicit_underscore_import( "from masakarimonitors.i18n import _", "masakarimonitors/tests/other_files.py"))), 0) self.assertEqual(len(list(checks.check_explicit_underscore_import( "LOG.info(_('My info message'))", "masakarimonitors/tests/other_files.py"))), 0) self.assertEqual(len(list(checks.check_explicit_underscore_import( "msg = _('My message')", "masakarimonitors/tests/other_files.py"))), 0) self.assertEqual(len(list(checks.check_explicit_underscore_import( "from masakarimonitors.i18n import _", "masakarimonitors/tests/other_files2.py"))), 0) self.assertEqual(len(list(checks.check_explicit_underscore_import( "msg = _('My message')", "masakarimonitors/tests/other_files2.py"))), 0) self.assertEqual(len(list(checks.check_explicit_underscore_import( "_ = translations.ugettext", "masakarimonitors/tests/other_files3.py"))), 0) self.assertEqual(len(list(checks.check_explicit_underscore_import( "msg = _('My message')", "masakarimonitors/tests/other_files3.py"))), 0) def test_no_translate_logs(self): self.assertEqual(1, len(list(checks.no_translate_logs( "LOG.error(_LE('foo'))")))) self.assertEqual(0, len(list(checks.no_translate_logs( "LOG.debug('foo')")))) self.assertEqual(1, len(list(checks.no_translate_logs( "LOG.info(_LI('foo'))")))) self.assertEqual(1, len(list(checks.no_translate_logs( "LOG.warning(_LW('foo'))")))) self.assertEqual(1, len(list(checks.no_translate_logs( "LOG.critical(_LC('foo'))")))) def test_yield_followed_by_space(self): code = """ yield(x, y) yield{"type": "test"} yield[a, b, c] yield"test" yield'test' """ errors = [(x + 1, 0, 'M303') for x in range(5)] self._assert_has_errors(code, checks.yield_followed_by_space, expected_errors=errors) code = """ yield x yield (x, y) yield {"type": "test"} yield [a, b, c] yield "test" yield 'test' yieldx_func(a, b) """ self._assert_has_no_errors(code, checks.yield_followed_by_space) masakari-monitors-9.0.0/masakarimonitors/tests/unit/hostmonitor/0000775000175000017500000000000013656750123025303 5ustar zuulzuul00000000000000masakari-monitors-9.0.0/masakarimonitors/tests/unit/hostmonitor/__init__.py0000664000175000017500000000000013656750014027401 0ustar zuulzuul00000000000000masakari-monitors-9.0.0/masakarimonitors/tests/unit/hostmonitor/test_host.py0000664000175000017500000000726613656750014027703 0ustar zuulzuul00000000000000# Copyright(c) 2017 Nippon Telegraph and Telephone Corporation # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. import os import testtools from unittest import mock from stevedore import driver from masakarimonitors.hostmonitor import host class TestHostmonitorManager(testtools.TestCase): def setUp(self): super(TestHostmonitorManager, self).setUp() @mock.patch.object(driver, 'DriverManager') def test_init_host(self, mock_DriverManager): mock_driver = mock.Mock() mock_DriverManager.return_value = mock_driver host_manager = host.HostmonitorManager() host_manager.init_host() mock_DriverManager.assert_called_once_with( namespace='hostmonitor.driver', name='default', invoke_on_load=True, invoke_args=(), ) @mock.patch.object(os, '_exit') @mock.patch.object(driver, 'DriverManager') def test_init_host_exception(self, mock_DriverManager, mock_exit): mock_DriverManager.side_effect = Exception("Test exception.") mock_exit.return_value = None host_manager = host.HostmonitorManager() host_manager.init_host() mock_DriverManager.assert_called_once_with( namespace='hostmonitor.driver', name='default', invoke_on_load=True, invoke_args=(), ) mock_exit.assert_called_once_with(1) @mock.patch.object(driver, 'DriverManager') def test_stop(self, mock_DriverManager): mock_driver = mock.Mock() mock_DriverManager.return_value = mock_driver host_manager = host.HostmonitorManager() host_manager.init_host() host_manager.stop() mock_DriverManager.assert_called_once_with( namespace='hostmonitor.driver', name='default', invoke_on_load=True, invoke_args=(), ) mock_driver.driver.stop.assert_called_once_with() @mock.patch.object(driver, 'DriverManager') def test_main(self, mock_DriverManager): mock_driver = mock.Mock() mock_DriverManager.return_value = mock_driver host_manager = host.HostmonitorManager() host_manager.init_host() ret = host_manager.main() mock_DriverManager.assert_called_once_with( namespace='hostmonitor.driver', name='default', invoke_on_load=True, invoke_args=(), ) mock_driver.driver.monitor_hosts.assert_called_once_with() self.assertIsNone(ret) @mock.patch.object(driver, 'DriverManager') def test_main_exception(self, mock_DriverManager): mock_driver = mock.Mock() mock_DriverManager.return_value = mock_driver mock_driver.driver.monitor_hosts.side_effect = \ Exception("Test exception.") host_manager = host.HostmonitorManager() host_manager.init_host() ret = host_manager.main() mock_DriverManager.assert_called_once_with( namespace='hostmonitor.driver', name='default', invoke_on_load=True, invoke_args=(), ) mock_driver.driver.monitor_hosts.assert_called_once_with() self.assertIsNone(ret) masakari-monitors-9.0.0/masakarimonitors/tests/unit/hostmonitor/host_handler/0000775000175000017500000000000013656750123027755 5ustar zuulzuul00000000000000masakari-monitors-9.0.0/masakarimonitors/tests/unit/hostmonitor/host_handler/__init__.py0000664000175000017500000000000013656750014032053 0ustar zuulzuul00000000000000././@LongLink0000000000000000000000000000014600000000000011216 Lustar 00000000000000masakari-monitors-9.0.0/masakarimonitors/tests/unit/hostmonitor/host_handler/test_parse_crmmon_xml.pymasakari-monitors-9.0.0/masakarimonitors/tests/unit/hostmonitor/host_handler/test_parse_crmmon_xml.p0000664000175000017500000000524513656750014034547 0ustar zuulzuul00000000000000# Copyright(c) 2019 Canonical Ltd # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. import testtools from masakarimonitors.hostmonitor.host_handler import parse_crmmon_xml CRMMON_XML = '' \ '' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ '' CRMMON_NONODES_XML = '' \ '' \ ' ' \ ' ' \ '' CRMMON_NONODES_TAG_XML = '' \ '' \ '' class TestParseCrmMonXml(testtools.TestCase): def setUp(self): super(TestParseCrmMonXml, self).setUp() def test_set_crmmon_xml(self): obj = parse_crmmon_xml.ParseCrmMonXml() obj.set_crmmon_xml(CRMMON_XML) def test_get_node_state_tag_list(self): obj = parse_crmmon_xml.ParseCrmMonXml() obj.set_crmmon_xml(CRMMON_XML) node_state_tag_list = obj.get_node_state_tag_list() expected = { 'node-1': 'true', 'node-2': 'false', 'node-3': 'true'} for node_state_tag in node_state_tag_list: self.assertEqual( expected[node_state_tag.get('name')], node_state_tag.get('online')) def test_get_node_state_tag_list_unset(self): obj = parse_crmmon_xml.ParseCrmMonXml() self.assertEqual(obj.get_node_state_tag_list(), []) def test_get_node_state_tag_list_nonodes(self): obj = parse_crmmon_xml.ParseCrmMonXml() obj.set_crmmon_xml(CRMMON_NONODES_XML) self.assertEqual(obj.get_node_state_tag_list(), []) def test_get_node_state_tag_list_nonodes_tag(self): obj = parse_crmmon_xml.ParseCrmMonXml() obj.set_crmmon_xml(CRMMON_NONODES_TAG_XML) self.assertEqual(obj.get_node_state_tag_list(), []) masakari-monitors-9.0.0/masakarimonitors/tests/unit/hostmonitor/host_handler/test_parse_cib_xml.py0000664000175000017500000001351013656750014034174 0ustar zuulzuul00000000000000# Copyright(c) 2016 Nippon Telegraph and Telephone Corporation # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. import testtools from unittest import mock from xml.etree import ElementTree import eventlet from masakarimonitors.hostmonitor.host_handler import parse_cib_xml eventlet.monkey_patch(os=False) CIB_XML = '' \ ' ' \ ' test' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ '' CIB_TAG = ElementTree.fromstring(CIB_XML) class TestParseCibXml(testtools.TestCase): def setUp(self): super(TestParseCibXml, self).setUp() @mock.patch.object(ElementTree, 'fromstring') def test_set_cib_xml(self, mock_fromstring): obj = parse_cib_xml.ParseCibXml() mock_fromstring.return_value = CIB_TAG obj.set_cib_xml(CIB_XML) def test_have_quorum(self): obj = parse_cib_xml.ParseCibXml() obj.set_cib_xml(CIB_XML) self.assertEqual(1, obj.have_quorum()) def test_get_node_state_tag_list(self): obj = parse_cib_xml.ParseCibXml() obj.set_cib_xml(CIB_XML) node_state_tag_list = obj.get_node_state_tag_list() for node_state_tag in node_state_tag_list: self.assertEqual('online', node_state_tag.get('crmd')) def test_get_stonith_ipmi_params(self): obj = parse_cib_xml.ParseCibXml() obj.set_cib_xml(CIB_XML) ipmi_values = obj.get_stonith_ipmi_params('compute-node') self.assertEqual('192.168.10.21', ipmi_values['ipaddr']) self.assertEqual('admin', ipmi_values['userid']) self.assertEqual('password', ipmi_values['passwd']) self.assertEqual('lanplus', ipmi_values['interface']) masakari-monitors-9.0.0/masakarimonitors/tests/unit/hostmonitor/host_handler/test_handle_host.py0000664000175000017500000010617113656750014033663 0ustar zuulzuul00000000000000# Copyright(c) 2016 Nippon Telegraph and Telephone Corporation # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. import socket import testtools from unittest import mock from xml.etree import ElementTree import eventlet from oslo_utils import timeutils import masakarimonitors.conf from masakarimonitors.ha import masakari from masakarimonitors.hostmonitor.host_handler import handle_host from masakarimonitors.hostmonitor.host_handler import hold_host_status from masakarimonitors.hostmonitor.host_handler import parse_cib_xml from masakarimonitors.hostmonitor.host_handler import parse_crmmon_xml from masakarimonitors.objects import event_constants as ec from masakarimonitors import utils eventlet.monkey_patch(os=False) CONF = masakarimonitors.conf.CONF STATUS_TAG_XML = ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' \ ' ' CRMMON_NODES_TAG_XML = """ """ class TestCibSchemaCompliantTag(testtools.TestCase): def setUp(self): super(TestCibSchemaCompliantTag, self).setUp() def test_init_offline(self): tag = handle_host.CibSchemaCompliantTag( {'name': 'test1', 'online': 'false'}) self.assertEqual(tag['uname'], 'test1') self.assertEqual(tag['crmd'], 'offline') def test_init_online(self): tag = handle_host.CibSchemaCompliantTag( {'name': 'test1', 'online': 'true'}) self.assertEqual(tag['uname'], 'test1') self.assertEqual(tag['crmd'], 'online') class TestHandleHost(testtools.TestCase): def setUp(self): super(TestHandleHost, self).setUp() @mock.patch.object(utils, 'execute') def test_check_pacemaker_services(self, mock_execute): mock_execute.return_value = ('test_stdout', '') obj = handle_host.HandleHost() ret = obj._check_pacemaker_services('corosync') self.assertTrue(ret) mock_execute.assert_called_once_with( 'systemctl', 'status', 'corosync', run_as_root=True) @mock.patch.object(utils, 'execute') def test_check_pacemaker_services_stderr(self, mock_execute): mock_execute.return_value = ('test_stdout', 'test_stderr') obj = handle_host.HandleHost() ret = obj._check_pacemaker_services('corosync') self.assertFalse(ret) mock_execute.assert_called_once_with( 'systemctl', 'status', 'corosync', run_as_root=True) @mock.patch.object(utils, 'execute') @mock.patch.object(handle_host.HandleHost, '_check_pacemaker_services') def test_check_hb_line( self, mock_check_pacemaker_services, mock_execute): mock_check_pacemaker_services.side_effect = [True, True, False] interfaces = "enp0s8" ports = "5405" CONF.host.corosync_multicast_interfaces = interfaces CONF.host.corosync_multicast_ports = ports mock_execute.return_value = ('', '') obj = handle_host.HandleHost() ret = obj._check_hb_line() self.assertEqual(0, ret) self.assertEqual(3, mock_check_pacemaker_services.call_count) mock_check_pacemaker_services.assert_any_call('corosync') mock_check_pacemaker_services.assert_any_call('pacemaker') mock_check_pacemaker_services.assert_any_call('pacemaker_remote') cmd_str = ("timeout %s tcpdump -n -c 1 -p -i %s port %s") \ % (CONF.host.tcpdump_timeout, interfaces, ports) command = cmd_str.split(' ') mock_execute.assert_called_once_with(*command, run_as_root=True) @mock.patch.object(handle_host.HandleHost, '_check_pacemaker_services') def test_check_hb_line_pacemaker_not_running( self, mock_check_pacemaker_services): mock_check_pacemaker_services.side_effect = [False, False, False] obj = handle_host.HandleHost() ret = obj._check_hb_line() self.assertEqual(2, ret) self.assertEqual(3, mock_check_pacemaker_services.call_count) mock_check_pacemaker_services.assert_any_call('corosync') mock_check_pacemaker_services.assert_any_call('pacemaker') mock_check_pacemaker_services.assert_any_call('pacemaker_remote') @mock.patch.object(handle_host.HandleHost, '_check_pacemaker_services') def test_check_hb_line_work_on_pacemaker_remote( self, mock_check_pacemaker_services): mock_check_pacemaker_services.side_effect = [False, False, True] obj = handle_host.HandleHost() ret = obj._check_hb_line() self.assertEqual(0, ret) self.assertEqual(3, mock_check_pacemaker_services.call_count) mock_check_pacemaker_services.assert_any_call('corosync') mock_check_pacemaker_services.assert_any_call('pacemaker') mock_check_pacemaker_services.assert_any_call('pacemaker_remote') @mock.patch.object(handle_host.HandleHost, '_check_pacemaker_services') def test_check_hb_line_conf_interfaces_is_none( self, mock_check_pacemaker_services): mock_check_pacemaker_services.side_effect = [True, True, False] interfaces = None ports = "5405" CONF.host.corosync_multicast_interfaces = interfaces CONF.host.corosync_multicast_ports = ports obj = handle_host.HandleHost() ret = obj._check_hb_line() self.assertEqual(2, ret) self.assertEqual(3, mock_check_pacemaker_services.call_count) mock_check_pacemaker_services.assert_any_call('corosync') mock_check_pacemaker_services.assert_any_call('pacemaker') mock_check_pacemaker_services.assert_any_call('pacemaker_remote') @mock.patch.object(handle_host.HandleHost, '_check_pacemaker_services') def test_check_hb_line_conf_ports_is_none( self, mock_check_pacemaker_services): mock_check_pacemaker_services.side_effect = [True, True, False] interfaces = "enp0s8" ports = None CONF.host.corosync_multicast_interfaces = interfaces CONF.host.corosync_multicast_ports = ports obj = handle_host.HandleHost() ret = obj._check_hb_line() self.assertEqual(2, ret) self.assertEqual(3, mock_check_pacemaker_services.call_count) mock_check_pacemaker_services.assert_any_call('corosync') mock_check_pacemaker_services.assert_any_call('pacemaker') mock_check_pacemaker_services.assert_any_call('pacemaker_remote') @mock.patch.object(handle_host.HandleHost, '_check_pacemaker_services') def test_check_hb_line_incorrect_interfaces( self, mock_check_pacemaker_services): mock_check_pacemaker_services.side_effect = [True, True, False] interfaces = "enp0s3,enp0s8" ports = "5405" CONF.host.corosync_multicast_interfaces = interfaces CONF.host.corosync_multicast_ports = ports obj = handle_host.HandleHost() ret = obj._check_hb_line() self.assertEqual(2, ret) self.assertEqual(3, mock_check_pacemaker_services.call_count) mock_check_pacemaker_services.assert_any_call('corosync') mock_check_pacemaker_services.assert_any_call('pacemaker') mock_check_pacemaker_services.assert_any_call('pacemaker_remote') @mock.patch.object(handle_host.HandleHost, '_check_pacemaker_services') def test_check_hb_line_incorrect_ports( self, mock_check_pacemaker_services): mock_check_pacemaker_services.side_effect = [True, True, False] interfaces = "enp0s8" ports = "5405,5406" CONF.host.corosync_multicast_interfaces = interfaces CONF.host.corosync_multicast_ports = ports obj = handle_host.HandleHost() ret = obj._check_hb_line() self.assertEqual(2, ret) self.assertEqual(3, mock_check_pacemaker_services.call_count) mock_check_pacemaker_services.assert_any_call('corosync') mock_check_pacemaker_services.assert_any_call('pacemaker') mock_check_pacemaker_services.assert_any_call('pacemaker_remote') @mock.patch.object(utils, 'execute') @mock.patch.object(handle_host.HandleHost, '_check_pacemaker_services') def test_check_hb_line_tcpdump_fail( self, mock_check_pacemaker_services, mock_execute): mock_check_pacemaker_services.side_effect = [True, True, False] interfaces = "enp0s8" ports = "5405" CONF.host.corosync_multicast_interfaces = interfaces CONF.host.corosync_multicast_ports = ports mock_execute.side_effect = Exception("Test exception.") obj = handle_host.HandleHost() ret = obj._check_hb_line() self.assertEqual(1, ret) self.assertEqual(3, mock_check_pacemaker_services.call_count) mock_check_pacemaker_services.assert_any_call('corosync') mock_check_pacemaker_services.assert_any_call('pacemaker') mock_check_pacemaker_services.assert_any_call('pacemaker_remote') cmd_str = ("timeout %s tcpdump -n -c 1 -p -i %s port %s") \ % (CONF.host.tcpdump_timeout, interfaces, ports) command = cmd_str.split(' ') mock_execute.assert_called_once_with(*command, run_as_root=True) @mock.patch.object(utils, 'execute') def test_check_host_status_by_crmadmin_idle(self, mock_execute): my_hostname = socket.gethostname() crmadmin_stdout = "Status of crmd@%s: S_IDLE (ok)" % my_hostname mock_execute.return_value = (crmadmin_stdout, '') obj = handle_host.HandleHost() ret = obj._check_host_status_by_crmadmin() self.assertEqual(0, ret) mock_execute.assert_called_once_with( 'crmadmin', '-S', my_hostname, run_as_root=True) @mock.patch.object(utils, 'execute') def test_check_host_status_by_crmadmin_not_dc(self, mock_execute): my_hostname = socket.gethostname() crmadmin_stdout = "Status of crmd@%s: S_NOT_DC (ok)" % my_hostname mock_execute.return_value = (crmadmin_stdout, '') obj = handle_host.HandleHost() ret = obj._check_host_status_by_crmadmin() self.assertEqual(0, ret) mock_execute.assert_called_once_with( 'crmadmin', '-S', my_hostname, run_as_root=True) @mock.patch.object(utils, 'execute') def test_check_host_status_by_crmadmin_output_stderr(self, mock_execute): my_hostname = socket.gethostname() mock_execute.return_value = ('', 'crmadmin: command not found') obj = handle_host.HandleHost() ret = obj._check_host_status_by_crmadmin() self.assertEqual(1, ret) mock_execute.assert_called_once_with( 'crmadmin', '-S', my_hostname, run_as_root=True) @mock.patch.object(utils, 'execute') def test_check_host_status_by_crmadmin_output_unexpected_message( self, mock_execute): my_hostname = socket.gethostname() mock_execute.return_value = ('Unexpected message.', '') obj = handle_host.HandleHost() ret = obj._check_host_status_by_crmadmin() self.assertEqual(1, ret) mock_execute.assert_called_once_with( 'crmadmin', '-S', my_hostname, run_as_root=True) @mock.patch.object(utils, 'execute') def test_get_cib_xml(self, mock_execute): mock_execute.return_value = ('test_stdout', '') obj = handle_host.HandleHost() ret = obj._get_cib_xml() self.assertEqual('test_stdout', ret) mock_execute.assert_called_once_with( 'cibadmin', '--query', run_as_root=True) @mock.patch.object(utils, 'execute') def test_get_cib_xml_output_stderr(self, mock_execute): mock_execute.return_value = ('test_stdout', 'test_stderr') obj = handle_host.HandleHost() ret = obj._get_cib_xml() self.assertIsNone(ret) mock_execute.assert_called_once_with( 'cibadmin', '--query', run_as_root=True) @mock.patch.object(utils, 'execute') def test_get_crmmon_xml(self, mock_execute): mock_execute.return_value = ('test_stdout', '') obj = handle_host.HandleHost() ret = obj._get_crmmon_xml() self.assertEqual('test_stdout', ret) mock_execute.assert_called_once_with( 'crm_mon', '-X', run_as_root=True) @mock.patch.object(utils, 'execute') def test_get_crmmon_xml_stderr(self, mock_execute): mock_execute.return_value = ('test_stdout', 'test_stderr') obj = handle_host.HandleHost() ret = obj._get_crmmon_xml() self.assertIsNone(ret) mock_execute.assert_called_once_with( 'crm_mon', '-X', run_as_root=True) @mock.patch.object(utils, 'execute') @mock.patch.object(parse_cib_xml.ParseCibXml, 'get_stonith_ipmi_params') def test_is_poweroff(self, mock_get_stonith_ipmi_params, mock_execute): ipmi_values = { 'userid': 'admin', 'passwd': 'password', 'interface': 'lanplus', 'ipaddr': '0.0.0.0' } mock_get_stonith_ipmi_params.return_value = ipmi_values mock_execute.return_value = ('Chassis Power is off', '') obj = handle_host.HandleHost() ret = obj._is_poweroff('test_hostname') self.assertTrue(ret) cmd_str = ("timeout %s ipmitool -U %s -P %s -I %s -H %s " "power status") \ % (str(CONF.host.ipmi_timeout), ipmi_values['userid'], ipmi_values['passwd'], ipmi_values['interface'], ipmi_values['ipaddr']) command = cmd_str.split(' ') mock_execute.assert_called_once_with(*command, run_as_root=False) @mock.patch.object(utils, 'execute') @mock.patch.object(parse_cib_xml.ParseCibXml, 'get_stonith_ipmi_params') def test_is_poweroff_ipmi_values_is_none( self, mock_get_stonith_ipmi_params, mock_execute): mock_get_stonith_ipmi_params.return_value = None mock_execute.return_value = ('Chassis Power is off', '') obj = handle_host.HandleHost() ret = obj._is_poweroff('test_hostname') self.assertFalse(ret) mock_execute.assert_not_called() @mock.patch.object(eventlet.greenthread, 'sleep') @mock.patch.object(utils, 'execute') @mock.patch.object(parse_cib_xml.ParseCibXml, 'get_stonith_ipmi_params') def test_is_poweroff_output_stderr( self, mock_get_stonith_ipmi_params, mock_execute, mock_sleep): ipmi_values = { 'userid': 'admin', 'passwd': 'password', 'interface': 'lanplus', 'ipaddr': '0.0.0.0' } mock_get_stonith_ipmi_params.return_value = ipmi_values mock_execute.return_value = ('Chassis Power is off', 'test_stderr') mock_sleep.return_value = None obj = handle_host.HandleHost() ret = obj._is_poweroff('test_hostname') self.assertFalse(ret) cmd_str = ("timeout %s ipmitool -U %s -P %s -I %s -H %s " "power status") \ % (str(CONF.host.ipmi_timeout), ipmi_values['userid'], ipmi_values['passwd'], ipmi_values['interface'], ipmi_values['ipaddr']) command = cmd_str.split(' ') calls = [mock.call(*command, run_as_root=False), mock.call(*command, run_as_root=False), mock.call(*command, run_as_root=False), mock.call(*command, run_as_root=False)] mock_execute.assert_has_calls(calls) @mock.patch.object(eventlet.greenthread, 'sleep') @mock.patch.object(utils, 'execute') @mock.patch.object(parse_cib_xml.ParseCibXml, 'get_stonith_ipmi_params') def test_is_poweroff_output_unexpected_message( self, mock_get_stonith_ipmi_params, mock_execute, mock_sleep): ipmi_values = { 'userid': 'admin', 'passwd': 'password', 'interface': 'lanplus', 'ipaddr': '0.0.0.0' } mock_get_stonith_ipmi_params.return_value = ipmi_values mock_execute.return_value = ('Unexpected message.', '') mock_sleep.return_value = None obj = handle_host.HandleHost() ret = obj._is_poweroff('test_hostname') self.assertFalse(ret) cmd_str = ("timeout %s ipmitool -U %s -P %s -I %s -H %s " "power status") \ % (str(CONF.host.ipmi_timeout), ipmi_values['userid'], ipmi_values['passwd'], ipmi_values['interface'], ipmi_values['ipaddr']) command = cmd_str.split(' ') calls = [mock.call(*command, run_as_root=False), mock.call(*command, run_as_root=False), mock.call(*command, run_as_root=False), mock.call(*command, run_as_root=False)] mock_execute.assert_has_calls(calls) @mock.patch.object(timeutils, 'utcnow') def test_make_event_online(self, mock_utcnow): current_time = timeutils.utcnow() mock_utcnow.return_value = current_time obj = handle_host.HandleHost() hostname = 'test_hostname' current_status = 'online' ret = obj._make_event(hostname, current_status) event_type = ec.EventConstants.EVENT_STARTED cluster_status = current_status.upper() host_status = ec.EventConstants.HOST_STATUS_NORMAL event = { 'notification': { 'type': ec.EventConstants.TYPE_COMPUTE_HOST, 'hostname': hostname, 'generated_time': current_time, 'payload': { 'event': event_type, 'cluster_status': cluster_status, 'host_status': host_status } } } self.assertEqual(event, ret) @mock.patch.object(timeutils, 'utcnow') @mock.patch.object(handle_host.HandleHost, '_is_poweroff') def test_make_event_offline_disable_ipmi_check_false_poweroff( self, mock_is_poweroff, mock_utcnow): mock_is_poweroff.return_value = True current_time = timeutils.utcnow() mock_utcnow.return_value = current_time CONF.host.disable_ipmi_check = False obj = handle_host.HandleHost() hostname = 'test_hostname' current_status = 'offline' ret = obj._make_event(hostname, current_status) event_type = ec.EventConstants.EVENT_STOPPED cluster_status = current_status.upper() host_status = ec.EventConstants.HOST_STATUS_NORMAL event = { 'notification': { 'type': ec.EventConstants.TYPE_COMPUTE_HOST, 'hostname': hostname, 'generated_time': current_time, 'payload': { 'event': event_type, 'cluster_status': cluster_status, 'host_status': host_status } } } self.assertEqual(event, ret) mock_is_poweroff.assert_called_once_with(hostname) @mock.patch.object(timeutils, 'utcnow') @mock.patch.object(handle_host.HandleHost, '_is_poweroff') def test_make_event_offline_disable_ipmi_check_false_poweron( self, mock_is_poweroff, mock_utcnow): mock_is_poweroff.return_value = False current_time = timeutils.utcnow() mock_utcnow.return_value = current_time CONF.host.disable_ipmi_check = False obj = handle_host.HandleHost() hostname = 'test_hostname' current_status = 'offline' ret = obj._make_event(hostname, current_status) event_type = ec.EventConstants.EVENT_STOPPED cluster_status = current_status.upper() host_status = ec.EventConstants.HOST_STATUS_UNKNOWN event = { 'notification': { 'type': ec.EventConstants.TYPE_COMPUTE_HOST, 'hostname': hostname, 'generated_time': current_time, 'payload': { 'event': event_type, 'cluster_status': cluster_status, 'host_status': host_status } } } self.assertEqual(event, ret) mock_is_poweroff.assert_called_once_with(hostname) @mock.patch.object(timeutils, 'utcnow') def test_make_event_offline_disable_ipmi_check_true(self, mock_utcnow): current_time = timeutils.utcnow() mock_utcnow.return_value = current_time CONF.host.disable_ipmi_check = True obj = handle_host.HandleHost() hostname = 'test_hostname' current_status = 'offline' ret = obj._make_event(hostname, current_status) event_type = ec.EventConstants.EVENT_STOPPED cluster_status = current_status.upper() host_status = ec.EventConstants.HOST_STATUS_NORMAL event = { 'notification': { 'type': ec.EventConstants.TYPE_COMPUTE_HOST, 'hostname': hostname, 'generated_time': current_time, 'payload': { 'event': event_type, 'cluster_status': cluster_status, 'host_status': host_status } } } self.assertEqual(event, ret) @mock.patch.object(masakari.SendNotification, 'send_notification') @mock.patch.object(handle_host.HandleHost, '_make_event') @mock.patch.object(hold_host_status.HostHoldStatus, 'set_host_status') @mock.patch.object(hold_host_status.HostHoldStatus, 'get_host_status') @mock.patch.object(socket, 'gethostname') def test_check_if_status_changed( self, mock_gethostname, mock_get_host_status, mock_set_host_status, mock_make_event, mock_send_notification): mock_gethostname.return_value = 'node1' mock_get_host_status.side_effect = \ [None, 'online', 'online', 'online'] mock_set_host_status.return_value = None test_event = {'notification': 'test'} mock_make_event.return_value = test_event status_tag = ElementTree.fromstring(STATUS_TAG_XML) node_state_tag_list = status_tag.getchildren() obj = handle_host.HandleHost() obj._check_if_status_changed(node_state_tag_list) node_state_node2 = node_state_tag_list[1] node_state_node3 = node_state_tag_list[2] node_state_node4 = node_state_tag_list[3] node_state_node5 = node_state_tag_list[4] node2 = node_state_node2.get('uname') node3 = node_state_node3.get('uname') node4 = node_state_node4.get('uname') node5 = node_state_node5.get('uname') calls_get_host_status = [mock.call(node2), mock.call(node3), mock.call(node4), mock.call(node5)] calls_set_host_status = [mock.call(node_state_node2), mock.call(node_state_node3), mock.call(node_state_node4), mock.call(node_state_node5)] mock_get_host_status.assert_has_calls(calls_get_host_status) mock_set_host_status.assert_has_calls(calls_set_host_status) mock_make_event.assert_called_once_with(node4, 'offline') mock_send_notification.assert_called_once_with( CONF.host.api_retry_max, CONF.host.api_retry_interval, test_event) @mock.patch.object(handle_host.HandleHost, '_check_if_status_changed') @mock.patch.object(parse_crmmon_xml.ParseCrmMonXml, 'get_node_state_tag_list') @mock.patch.object(parse_crmmon_xml.ParseCrmMonXml, 'set_crmmon_xml') @mock.patch.object(handle_host.HandleHost, '_get_crmmon_xml') def test_check_host_status_by_crm_mon( self, mock_get_crmmon_xml, mock_set_crmmon_xml, mock_get_node_state_tag_list, mock_check_if_status_changed): mock_get_crmmon_xml.return_value = CRMMON_NODES_TAG_XML mock_set_crmmon_xml.return_value = None status_tag = ElementTree.fromstring(CRMMON_NODES_TAG_XML) node_state_tag_list = status_tag.getchildren() mock_get_node_state_tag_list.return_value = node_state_tag_list mock_check_if_status_changed.return_value = None obj = handle_host.HandleHost() ret = obj._check_host_status_by_crm_mon() self.assertEqual(0, ret) mock_get_node_state_tag_list.assert_called_once_with() mock_set_crmmon_xml.assert_called_once_with(CRMMON_NODES_TAG_XML) mock_get_node_state_tag_list.assert_called_once_with() mock_check_if_status_changed.assert_called_once_with( [ {'uname': 'remote1', 'crmd': 'online'}, {'uname': 'remote2', 'crmd': 'online'}, {'uname': 'remote3', 'crmd': 'online'}]) @mock.patch.object(parse_crmmon_xml.ParseCrmMonXml, 'get_node_state_tag_list') @mock.patch.object(parse_crmmon_xml.ParseCrmMonXml, 'set_crmmon_xml') @mock.patch.object(handle_host.HandleHost, '_get_crmmon_xml') def test_check_host_status_by_crm_mon_not_have_node_state_tag( self, mock_get_crmmon_xml, mock_set_crmmon_xml, mock_get_node_state_tag_list): mock_get_crmmon_xml.return_value = CRMMON_NODES_TAG_XML mock_set_crmmon_xml.return_value = None mock_get_node_state_tag_list.return_value = [] obj = handle_host.HandleHost() self.assertRaisesRegexp( Exception, "Failed to get nodes tag from crm_mon xml.", obj._check_host_status_by_crm_mon) mock_get_crmmon_xml.assert_called_once_with() mock_set_crmmon_xml.assert_called_once_with(CRMMON_NODES_TAG_XML) mock_get_node_state_tag_list.assert_called_once_with() @mock.patch.object(handle_host.HandleHost, '_get_crmmon_xml') def test_check_host_status_by_crm_mon_xml_is_None( self, mock_get_crmmon_xml): mock_get_crmmon_xml.return_value = None obj = handle_host.HandleHost() ret = obj._check_host_status_by_crm_mon() self.assertEqual(1, ret) mock_get_crmmon_xml.assert_called_once_with() @mock.patch.object(handle_host.HandleHost, '_check_if_status_changed') @mock.patch.object(parse_cib_xml.ParseCibXml, 'get_node_state_tag_list') @mock.patch.object(parse_cib_xml.ParseCibXml, 'have_quorum') @mock.patch.object(parse_cib_xml.ParseCibXml, 'set_cib_xml') @mock.patch.object(handle_host.HandleHost, '_get_cib_xml') def test_check_host_status_by_cibadmin( self, mock_get_cib_xml, mock_set_cib_xml, mock_have_quorum, mock_get_node_state_tag_list, mock_check_if_status_changed): mock_get_cib_xml.return_value = STATUS_TAG_XML mock_set_cib_xml.return_value = None mock_have_quorum.return_value = 1 status_tag = ElementTree.fromstring(STATUS_TAG_XML) node_state_tag_list = status_tag.getchildren() mock_get_node_state_tag_list.return_value = node_state_tag_list mock_check_if_status_changed.return_value = None obj = handle_host.HandleHost() ret = obj._check_host_status_by_cibadmin() self.assertEqual(0, ret) mock_get_cib_xml.assert_called_once_with() mock_set_cib_xml.assert_called_once_with(STATUS_TAG_XML) mock_have_quorum.assert_called_once_with() mock_get_node_state_tag_list.assert_called_once_with() mock_check_if_status_changed.assert_called_once_with( node_state_tag_list) @mock.patch.object(handle_host.HandleHost, '_check_if_status_changed') @mock.patch.object(parse_cib_xml.ParseCibXml, 'get_node_state_tag_list') @mock.patch.object(parse_cib_xml.ParseCibXml, 'have_quorum') @mock.patch.object(parse_cib_xml.ParseCibXml, 'set_cib_xml') @mock.patch.object(handle_host.HandleHost, '_get_cib_xml') def test_check_host_status_by_cibadmin_no_quorum( self, mock_get_cib_xml, mock_set_cib_xml, mock_have_quorum, mock_get_node_state_tag_list, mock_check_if_status_changed): mock_get_cib_xml.return_value = STATUS_TAG_XML mock_set_cib_xml.return_value = None mock_have_quorum.return_value = 0 status_tag = ElementTree.fromstring(STATUS_TAG_XML) node_state_tag_list = status_tag.getchildren() mock_get_node_state_tag_list.return_value = node_state_tag_list mock_check_if_status_changed.return_value = None obj = handle_host.HandleHost() ret = obj._check_host_status_by_cibadmin() self.assertEqual(0, ret) mock_get_cib_xml.assert_called_once_with() mock_set_cib_xml.assert_called_once_with(STATUS_TAG_XML) mock_have_quorum.assert_called_once_with() mock_get_node_state_tag_list.assert_called_once_with() mock_check_if_status_changed.assert_called_once_with( node_state_tag_list) @mock.patch.object(handle_host.HandleHost, '_get_cib_xml') def test_check_host_status_by_cibadmin_cib_xml_is_None( self, mock_get_cib_xml): mock_get_cib_xml.return_value = None obj = handle_host.HandleHost() ret = obj._check_host_status_by_cibadmin() self.assertEqual(1, ret) mock_get_cib_xml.assert_called_once_with() @mock.patch.object(parse_cib_xml.ParseCibXml, 'get_node_state_tag_list') @mock.patch.object(parse_cib_xml.ParseCibXml, 'have_quorum') @mock.patch.object(parse_cib_xml.ParseCibXml, 'set_cib_xml') @mock.patch.object(handle_host.HandleHost, '_get_cib_xml') def test_check_host_status_by_cibadmin_not_have_node_state_tag( self, mock_get_cib_xml, mock_set_cib_xml, mock_have_quorum, mock_get_node_state_tag_list): mock_get_cib_xml.return_value = STATUS_TAG_XML mock_set_cib_xml.return_value = None mock_have_quorum.return_value = 1 mock_get_node_state_tag_list.return_value = [] obj = handle_host.HandleHost() self.assertRaisesRegexp( Exception, "Failed to get node_state tag from cib xml.", obj._check_host_status_by_cibadmin) mock_get_cib_xml.assert_called_once_with() mock_set_cib_xml.assert_called_once_with(STATUS_TAG_XML) mock_have_quorum.assert_called_once_with() mock_get_node_state_tag_list.assert_called_once_with() def test_stop(self): obj = handle_host.HandleHost() obj.stop() self.assertFalse(obj.running) @mock.patch.object(eventlet.greenthread, 'sleep') @mock.patch.object(handle_host.HandleHost, '_check_host_status_by_cibadmin') @mock.patch.object(handle_host.HandleHost, '_check_host_status_by_crmadmin') @mock.patch.object(handle_host.HandleHost, '_check_pacemaker_services') @mock.patch.object(handle_host.HandleHost, '_check_hb_line') def test_monitor_hosts(self, mock_check_hb_line, mock_check_pacemaker_services, mock_check_host_status_by_cibadmin, mock_check_host_status_by_crmadmin, mock_sleep): mock_check_hb_line.side_effect = \ [0, 1, 2, 0, Exception("Test exception.")] mock_check_pacemaker_services.side_effect = [True, False, False] mock_check_host_status_by_cibadmin.side_effect = [0, 1] mock_check_host_status_by_crmadmin.side_effect = [0, 1] mock_sleep.return_value = None obj = handle_host.HandleHost() obj.monitor_hosts() self.assertEqual(5, mock_check_hb_line.call_count) self.assertEqual(3, mock_check_pacemaker_services.call_count) mock_check_pacemaker_services.assert_called_with('pacemaker_remote') self.assertEqual(2, mock_check_host_status_by_cibadmin.call_count) self.assertEqual(2, mock_check_host_status_by_crmadmin.call_count) @mock.patch.object(eventlet.greenthread, 'sleep') @mock.patch.object(handle_host.HandleHost, '_check_host_status_by_crm_mon') @mock.patch.object(handle_host.HandleHost, '_check_pacemaker_services') @mock.patch.object(handle_host.HandleHost, '_check_hb_line') def test_monitor_hosts_remotes_only(self, mock_check_hb_line, mock_check_pacemaker_services, mock_check_host_status_by_crm_mon, mock_sleep): CONF.host.restrict_to_remotes = True mock_check_hb_line.side_effect = \ [0, Exception("Test exception.")] mock_check_pacemaker_services.return_value = True mock_check_host_status_by_crm_mon.side_effect = 0 mock_sleep.return_value = None obj = handle_host.HandleHost() obj.monitor_hosts() self.assertEqual(1, mock_check_hb_line.call_count) self.assertEqual(1, mock_check_pacemaker_services.call_count) mock_check_pacemaker_services.assert_called_with('pacemaker_remote') self.assertEqual(1, mock_check_host_status_by_crm_mon.call_count) mock_check_host_status_by_crm_mon.assert_called_once_with() ././@LongLink0000000000000000000000000000014600000000000011216 Lustar 00000000000000masakari-monitors-9.0.0/masakarimonitors/tests/unit/hostmonitor/host_handler/test_hold_host_status.pymasakari-monitors-9.0.0/masakarimonitors/tests/unit/hostmonitor/host_handler/test_hold_host_status.p0000664000175000017500000000274213656750014034567 0ustar zuulzuul00000000000000# Copyright(c) 2016 Nippon Telegraph and Telephone Corporation # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. import testtools from xml.etree import ElementTree import eventlet from masakarimonitors.hostmonitor.host_handler import hold_host_status eventlet.monkey_patch(os=False) NODE_STATE_XML = '' \ ' ' \ '' NODE_STATE_TAG = ElementTree.fromstring(NODE_STATE_XML) class TestHostHoldStatus(testtools.TestCase): def setUp(self): super(TestHostHoldStatus, self).setUp() def test_set_host_status(self): obj = hold_host_status.HostHoldStatus() obj.set_host_status(NODE_STATE_TAG) self.assertEqual('online', obj.get_host_status('masakari-node')) def test_get_host_status(self): obj = hold_host_status.HostHoldStatus() obj.set_host_status(NODE_STATE_TAG) self.assertEqual('online', obj.get_host_status('masakari-node')) masakari-monitors-9.0.0/masakarimonitors/tests/unit/ha/0000775000175000017500000000000013656750123023306 5ustar zuulzuul00000000000000masakari-monitors-9.0.0/masakarimonitors/tests/unit/ha/__init__.py0000664000175000017500000000000013656750014025404 0ustar zuulzuul00000000000000masakari-monitors-9.0.0/masakarimonitors/tests/unit/ha/test_masakari.py0000664000175000017500000001355713656750014026521 0ustar zuulzuul00000000000000# Copyright(c) 2017 Nippon Telegraph and Telephone Corporation # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. import testtools from unittest import mock import uuid import eventlet from keystoneauth1.identity.generic import password as ks_password from keystoneauth1 import session as ks_session from openstack import connection from openstack import exceptions from oslo_utils import timeutils from masakarimonitors.ha import masakari from masakarimonitors.objects import event_constants as ec PROFILE_TYPE = "ha" PROFILE_NAME = "masakari" class FakeResponse(object): def __init__(self, status_code=200, headers=None): self.status_code = status_code self.headers = { 'content-type': 'application/json', 'x-openstack-request-id': uuid.uuid4().hex, } class TestSendNotification(testtools.TestCase): def setUp(self): super(TestSendNotification, self).setUp() self.api_retry_max = 3 self.api_retry_interval = 1 self.event = { 'notification': { 'type': ec.EventConstants.TYPE_COMPUTE_HOST, 'hostname': 'compute-node1', 'generated_time': timeutils.utcnow(), 'payload': { 'event': ec.EventConstants.EVENT_STOPPED, 'cluster_status': 'OFFLINE', 'host_status': ec.EventConstants.HOST_STATUS_NORMAL } } } @mock.patch.object(connection, 'Connection') @mock.patch.object(ks_session, 'Session') @mock.patch.object(ks_password, 'Password') def test_send_notification( self, mock_password, mock_session, mock_connection): mock_conn = mock.Mock() mock_conn.instance_ha.return_value = mock.Mock() mock_conn.instance_ha.create_notification.return_value = mock.Mock() mock_connection.return_value = mock_conn notifier = masakari.SendNotification() notifier.send_notification( self.api_retry_max, self.api_retry_interval, self.event) mock_conn.instance_ha.create_notification.assert_called_once_with( type=self.event['notification']['type'], hostname=self.event['notification']['hostname'], generated_time=self.event['notification']['generated_time'], payload=self.event['notification']['payload']) @mock.patch.object(connection, 'Connection') @mock.patch.object(ks_session, 'Session') @mock.patch.object(ks_password, 'Password') def test_send_notification_409_error( self, mock_password, mock_session, mock_connection): mock_conn = mock.Mock() mock_conn.instance_ha.return_value = mock.Mock() mock_conn.instance_ha.create_notification.return_value = mock.Mock() mock_connection.return_value = mock_conn # TODO(samP): Remove attribute check and else case if # openstacksdk is bumped up from '>=0.9.19' to '>=0.10.0' # in global-requirements. if hasattr(exceptions.HttpException(), 'status_code'): response = FakeResponse(status_code=409) status_ex = exceptions.HttpException(response=response) else: status_ex = exceptions.HttpException(http_status=409) mock_conn.instance_ha.create_notification.side_effect = status_ex notifier = masakari.SendNotification() notifier.send_notification( self.api_retry_max, self.api_retry_interval, self.event) mock_conn.instance_ha.create_notification.assert_called_once_with( type=self.event['notification']['type'], hostname=self.event['notification']['hostname'], generated_time=self.event['notification']['generated_time'], payload=self.event['notification']['payload']) @mock.patch.object(eventlet.greenthread, 'sleep') @mock.patch.object(connection, 'Connection') @mock.patch.object(ks_session, 'Session') @mock.patch.object(ks_password, 'Password') def test_send_notification_500_error( self, mock_password, mock_session, mock_connection, mock_sleep): mock_conn = mock.Mock() mock_conn.instance_ha.return_value = mock.Mock() mock_conn.instance_ha.create_notification.return_value = mock.Mock() mock_connection.return_value = mock_conn # TODO(samP): Remove attribute check and else case if # openstacksdk is bumped up from '>=0.9.19' to '>=0.10.0' # in global-requirements. if hasattr(exceptions.HttpException(), 'status_code'): response = FakeResponse(status_code=500) status_ex = exceptions.HttpException(response=response) else: status_ex = exceptions.HttpException(http_status=500) mock_conn.instance_ha.create_notification.side_effect = status_ex mock_sleep.return_value = None notifier = masakari.SendNotification() notifier.send_notification( self.api_retry_max, self.api_retry_interval, self.event) mock_conn.instance_ha.create_notification.assert_called_with( type=self.event['notification']['type'], hostname=self.event['notification']['hostname'], generated_time=self.event['notification']['generated_time'], payload=self.event['notification']['payload']) self.assertEqual(self.api_retry_max + 1, mock_conn.instance_ha.create_notification.call_count) masakari-monitors-9.0.0/masakarimonitors/tests/unit/instancemonitor/0000775000175000017500000000000013656750123026132 5ustar zuulzuul00000000000000masakari-monitors-9.0.0/masakarimonitors/tests/unit/instancemonitor/libvirt_handler/0000775000175000017500000000000013656750123031302 5ustar zuulzuul00000000000000masakari-monitors-9.0.0/masakarimonitors/tests/unit/instancemonitor/libvirt_handler/__init__.py0000664000175000017500000000000013656750014033400 0ustar zuulzuul00000000000000././@LongLink0000000000000000000000000000015000000000000011211 Lustar 00000000000000masakari-monitors-9.0.0/masakarimonitors/tests/unit/instancemonitor/libvirt_handler/test_eventfilter.pymasakari-monitors-9.0.0/masakarimonitors/tests/unit/instancemonitor/libvirt_handler/test_eventfilter0000664000175000017500000001332113656750014034612 0ustar zuulzuul00000000000000# Copyright(c) 2016 Nippon Telegraph and Telephone Corporation # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. import socket import testtools import threading from unittest import mock import uuid import eventlet from oslo_utils import excutils from oslo_utils import timeutils from masakarimonitors.instancemonitor.libvirt_handler import callback from masakarimonitors.instancemonitor.libvirt_handler import eventfilter from masakarimonitors.instancemonitor.libvirt_handler \ import eventfilter_table as evft from masakarimonitors.objects import event_constants as ec eventlet.monkey_patch(os=False) class TestEventFilter(testtools.TestCase): def setUp(self): super(TestEventFilter, self).setUp() @mock.patch.object(excutils, 'save_and_reraise_exception') @mock.patch.object(callback.Callback, 'libvirt_event_callback') @mock.patch.object(timeutils, 'utcnow') def test_vir_event_filter(self, mock_utcnow, mock_libvirt_event_callback, mock_save_and_reraise_exception): current_time = timeutils.utcnow() mock_utcnow.return_value = current_time mock_libvirt_event_callback.return_value = None mock_save_and_reraise_exception.return_value = None obj = eventfilter.EventFilter() eventID = 0 eventType = 5 detail = 5 uuID = uuid.uuid4() obj.vir_event_filter(eventID, eventType, detail, uuID) mock_libvirt_event_callback.assert_called_once_with( evft.eventID_dic[eventID], evft.detail_dic[eventID][eventType][detail], uuID, ec.EventConstants.TYPE_VM, socket.gethostname(), current_time) mock_save_and_reraise_exception.assert_not_called() @mock.patch.object(excutils, 'save_and_reraise_exception') @mock.patch.object(callback.Callback, 'libvirt_event_callback') def test_vir_event_filter_unmatched(self, mock_libvirt_event_callback, mock_save_and_reraise_exception): mock_libvirt_event_callback.return_value = None mock_save_and_reraise_exception.return_value = None obj = eventfilter.EventFilter() eventID = 0 eventType = 5 detail = 2 uuID = uuid.uuid4() obj.vir_event_filter(eventID, eventType, detail, uuID) mock_libvirt_event_callback.assert_not_called() mock_save_and_reraise_exception.assert_not_called() @mock.patch.object(excutils, 'save_and_reraise_exception') @mock.patch.object(callback.Callback, 'libvirt_event_callback') def test_vir_event_filter_key_error(self, mock_libvirt_event_callback, mock_save_and_reraise_exception): mock_libvirt_event_callback.return_value = None mock_save_and_reraise_exception.return_value = None obj = eventfilter.EventFilter() eventID = 0 eventType = 0 detail = 0 uuID = uuid.uuid4() obj.vir_event_filter(eventID, eventType, detail, uuID) mock_libvirt_event_callback.assert_not_called() mock_save_and_reraise_exception.assert_not_called() @mock.patch.object(excutils, 'save_and_reraise_exception') @mock.patch.object(callback.Callback, 'libvirt_event_callback') @mock.patch.object(threading, 'Thread') def test_vir_event_filter_type_error(self, mock_Thread, mock_libvirt_event_callback, mock_save_and_reraise_exception): mock_Thread.side_effect = TypeError("Threading exception.") mock_libvirt_event_callback.return_value = None mock_save_and_reraise_exception.return_value = None obj = eventfilter.EventFilter() eventID = 0 eventType = 5 detail = 5 uuID = uuid.uuid4() obj.vir_event_filter(eventID, eventType, detail, uuID) mock_libvirt_event_callback.assert_not_called() mock_save_and_reraise_exception.assert_not_called() @mock.patch.object(excutils, 'save_and_reraise_exception') @mock.patch.object(callback.Callback, 'libvirt_event_callback') @mock.patch.object(threading, 'Thread') def test_vir_event_filter_index_error(self, mock_Thread, mock_libvirt_event_callback, mock_save_and_reraise_exception): mock_Thread.side_effect = IndexError("Threading exception.") mock_libvirt_event_callback.return_value = None mock_save_and_reraise_exception.return_value = None obj = eventfilter.EventFilter() eventID = 0 eventType = 5 detail = 5 uuID = uuid.uuid4() obj.vir_event_filter(eventID, eventType, detail, uuID) mock_libvirt_event_callback.assert_not_called() mock_save_and_reraise_exception.assert_not_called() @mock.patch.object(callback.Callback, 'libvirt_event_callback') @mock.patch.object(threading, 'Thread') def test_vir_event_filter_other_exception(self, mock_Thread, mock_libvirt_event_callback): mock_Thread.side_effect = NameError("Threading exception.") mock_libvirt_event_callback.return_value = None obj = eventfilter.EventFilter() eventID = 0 eventType = 5 detail = 5 uuID = uuid.uuid4() self.assertRaises(NameError, obj.vir_event_filter, eventID, eventType, detail, uuID) mock_libvirt_event_callback.assert_not_called() masakari-monitors-9.0.0/masakarimonitors/tests/unit/instancemonitor/libvirt_handler/test_callback.py0000664000175000017500000000411213656750014034444 0ustar zuulzuul00000000000000# Copyright(c) 2016 Nippon Telegraph and Telephone Corporation # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. import socket import testtools from unittest import mock import uuid import eventlet from oslo_utils import timeutils from masakarimonitors.ha import masakari from masakarimonitors.instancemonitor.libvirt_handler import callback from masakarimonitors.objects import event_constants as ec eventlet.monkey_patch(os=False) class TestCallback(testtools.TestCase): def setUp(self): super(TestCallback, self).setUp() @mock.patch.object(masakari.SendNotification, 'send_notification') def test_libvirt_event_callback(self, mock_send_notification): mock_send_notification.return_value = None obj = callback.Callback() event_id = 0 details = 5 domain_uuid = uuid.uuid4() notice_type = ec.EventConstants.TYPE_VM hostname = socket.gethostname() current_time = timeutils.utcnow() obj.libvirt_event_callback(event_id, details, domain_uuid, notice_type, hostname, current_time) retry_max = 12 retry_interval = 10 event = { 'notification': { 'type': notice_type, 'hostname': hostname, 'generated_time': current_time, 'payload': { 'event': event_id, 'instance_uuid': domain_uuid, 'vir_domain_event': details } } } mock_send_notification.assert_called_once_with( retry_max, retry_interval, event) masakari-monitors-9.0.0/masakarimonitors/tests/unit/instancemonitor/__init__.py0000664000175000017500000000000013656750014030230 0ustar zuulzuul00000000000000masakari-monitors-9.0.0/masakarimonitors/tests/unit/instancemonitor/test_instance.py0000664000175000017500000002320313656750014031346 0ustar zuulzuul00000000000000# Copyright(c) 2016 Nippon Telegraph and Telephone Corporation # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. import libvirt import testtools import threading import time from unittest import mock import uuid import eventlet from masakarimonitors.instancemonitor import instance from masakarimonitors.instancemonitor.libvirt_handler import eventfilter eventlet.monkey_patch(os=False) class TestInstancemonitorManager(testtools.TestCase): def setUp(self): super(TestInstancemonitorManager, self).setUp() def _make_callback_params(self): mock_conn = mock.Mock() mock_dom = mock.Mock() test_uuid = uuid.uuid4() mock_dom.UUIDString.return_value = test_uuid mock_opaque = mock.Mock() return mock_conn, mock_dom, mock_opaque, test_uuid @mock.patch.object(libvirt, 'virEventRunDefaultImpl') def test_vir_event_loop_native_run(self, mock_virEventRunDefaultImpl): mock_virEventRunDefaultImpl.side_effect = Exception("Test exception.") obj = instance.InstancemonitorManager() exception_flag = False try: obj._vir_event_loop_native_run() except Exception: exception_flag = True self.assertTrue(exception_flag) mock_virEventRunDefaultImpl.assert_called_once() @mock.patch.object(eventfilter.EventFilter, 'vir_event_filter') def test_my_domain_event_callback(self, mock_vir_event_filter): mock_vir_event_filter.return_value = None mock_conn, mock_dom, mock_opaque, test_uuid = \ self._make_callback_params() obj = instance.InstancemonitorManager() obj._my_domain_event_callback(mock_conn, mock_dom, 0, 1, mock_opaque) mock_vir_event_filter.assert_called_once_with( libvirt.VIR_DOMAIN_EVENT_ID_LIFECYCLE, 0, 1, test_uuid) @mock.patch.object(eventfilter.EventFilter, 'vir_event_filter') def test_my_domain_event_reboot_callback(self, mock_vir_event_filter): mock_vir_event_filter.return_value = None mock_conn, mock_dom, mock_opaque, test_uuid = \ self._make_callback_params() obj = instance.InstancemonitorManager() obj._my_domain_event_reboot_callback(mock_conn, mock_dom, mock_opaque) mock_vir_event_filter.assert_called_once_with( libvirt.VIR_DOMAIN_EVENT_ID_REBOOT, -1, -1, test_uuid) @mock.patch.object(eventfilter.EventFilter, 'vir_event_filter') def test_my_domain_event_rtc_change_callback(self, mock_vir_event_filter): mock_vir_event_filter.return_value = None mock_conn, mock_dom, mock_opaque, test_uuid = \ self._make_callback_params() utcoffset = "" obj = instance.InstancemonitorManager() obj._my_domain_event_rtc_change_callback( mock_conn, mock_dom, utcoffset, mock_opaque) mock_vir_event_filter.assert_called_once_with( libvirt.VIR_DOMAIN_EVENT_ID_RTC_CHANGE, -1, -1, test_uuid) @mock.patch.object(eventfilter.EventFilter, 'vir_event_filter') def test_my_domain_event_watchdog_callback(self, mock_vir_event_filter): mock_vir_event_filter.return_value = None mock_conn, mock_dom, mock_opaque, test_uuid = \ self._make_callback_params() action = 0 obj = instance.InstancemonitorManager() obj._my_domain_event_watchdog_callback( mock_conn, mock_dom, action, mock_opaque) mock_vir_event_filter.assert_called_once_with( libvirt.VIR_DOMAIN_EVENT_ID_WATCHDOG, action, -1, test_uuid) @mock.patch.object(eventfilter.EventFilter, 'vir_event_filter') def test_my_domain_event_io_error_callback(self, mock_vir_event_filter): mock_vir_event_filter.return_value = None mock_conn, mock_dom, mock_opaque, test_uuid = \ self._make_callback_params() srcpath = "" devalias = "" action = 0 obj = instance.InstancemonitorManager() obj._my_domain_event_io_error_callback( mock_conn, mock_dom, srcpath, devalias, action, mock_opaque) mock_vir_event_filter.assert_called_once_with( libvirt.VIR_DOMAIN_EVENT_ID_IO_ERROR, action, -1, test_uuid) @mock.patch.object(eventfilter.EventFilter, 'vir_event_filter') def test_my_domain_event_graphics_callback(self, mock_vir_event_filter): mock_vir_event_filter.return_value = None mock_conn, mock_dom, mock_opaque, test_uuid = \ self._make_callback_params() phase = 0 localAddr = "" remoteAddr = "" authScheme = "" subject = "" obj = instance.InstancemonitorManager() obj._my_domain_event_graphics_callback( mock_conn, mock_dom, phase, localAddr, remoteAddr, authScheme, subject, mock_opaque) mock_vir_event_filter.assert_called_once_with( libvirt.VIR_DOMAIN_EVENT_ID_GRAPHICS, -1, phase, test_uuid) @mock.patch.object(eventfilter.EventFilter, 'vir_event_filter') def test_my_domain_event_disk_change_callback( self, mock_vir_event_filter): mock_vir_event_filter.return_value = None mock_conn, mock_dom, mock_opaque, test_uuid = \ self._make_callback_params() oldSrcPath = "" newSrcPath = "" devAlias = "" reason = "" obj = instance.InstancemonitorManager() obj._my_domain_event_disk_change_callback( mock_conn, mock_dom, oldSrcPath, newSrcPath, devAlias, reason, mock_opaque) mock_vir_event_filter.assert_called_once_with( libvirt.VIR_DOMAIN_EVENT_ID_DISK_CHANGE, -1, -1, test_uuid) @mock.patch.object(eventfilter.EventFilter, 'vir_event_filter') def test_my_domain_event_io_error_reason_callback( self, mock_vir_event_filter): mock_vir_event_filter.return_value = None mock_conn, mock_dom, mock_opaque, test_uuid = \ self._make_callback_params() srcPath = "" devAlias = "" action = "" reason = "" obj = instance.InstancemonitorManager() obj._my_domain_event_io_error_reason_callback( mock_conn, mock_dom, srcPath, devAlias, action, reason, mock_opaque) mock_vir_event_filter.assert_called_once_with( libvirt.VIR_DOMAIN_EVENT_ID_IO_ERROR_REASON, -1, -1, test_uuid) @mock.patch.object(eventfilter.EventFilter, 'vir_event_filter') def test_my_domain_event_generic_callback(self, mock_vir_event_filter): mock_vir_event_filter.return_value = None mock_conn, mock_dom, mock_opaque, test_uuid = \ self._make_callback_params() obj = instance.InstancemonitorManager() obj._my_domain_event_generic_callback( mock_conn, mock_dom, mock_opaque) mock_vir_event_filter.assert_called_once_with( libvirt.VIR_DOMAIN_EVENT_ID_CONTROL_ERROR, -1, -1, test_uuid) def test_err_handler(self): obj = instance.InstancemonitorManager() obj._err_handler("Test context.", ('err0.', 'err1', 'err2')) def test_stop(self): obj = instance.InstancemonitorManager() obj.stop() self.assertFalse(obj.running) @mock.patch.object(time, 'sleep') @mock.patch.object(eventlet.greenthread, 'sleep') @mock.patch.object(libvirt, 'openReadOnly') @mock.patch.object(threading, 'Thread') @mock.patch.object(libvirt, 'virEventRegisterDefaultImpl') def test_main(self, mock_virEventRegisterDefaultImpl, mock_Thread, mock_openReadOnly, mock_greenthread_sleep, mock_time_sleep): mock_virEventRegisterDefaultImpl.return_value = None mock_event_loop_thread = mock.Mock(return_value=None) mock_Thread.return_value = mock_event_loop_thread mock_vc = mock.Mock() mock_openReadOnly.return_value = mock_vc mock_vc.domainEventRegisterAny.side_effect = \ [0, 0, 0, 0, 0, 0, 0, 0, 0] mock_vc.setKeepAlive.return_value = None mock_vc.isAlive.side_effect = [1, 0] mock_vc.domainEventDeregisterAny.side_effect = \ [None, None, None, None, None, None, None, None, Exception("Test exception.")] mock_vc.close.return_value = None mock_greenthread_sleep.return_value = None mock_time_sleep.side_effect = Exception("Test exception.") obj = instance.InstancemonitorManager() exception_flag = False try: obj.main() except Exception: exception_flag = True handlers_count = 9 self.assertTrue(exception_flag) mock_virEventRegisterDefaultImpl.assert_called_once() mock_event_loop_thread.setDaemon.assert_called_once_with(True) mock_event_loop_thread.start.assert_called_once() mock_openReadOnly.assert_called_once_with("qemu:///system") self.assertEqual( handlers_count, mock_vc.domainEventRegisterAny.call_count) mock_vc.setKeepAlive.assert_called_once_with(5, 3) self.assertEqual(2, mock_vc.isAlive.call_count) self.assertEqual( handlers_count, mock_vc.domainEventDeregisterAny.call_count) mock_vc.close.assert_called_once() masakari-monitors-9.0.0/masakarimonitors/tests/test_masakarimonitors.py0000664000175000017500000000141313656750014026731 0ustar zuulzuul00000000000000# Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. """ test_masakarimonitors ---------------------------------- Tests for `masakarimonitors` module. """ from masakarimonitors.tests import base class TestMasakarimonitors(base.TestCase): def test_something(self): pass masakari-monitors-9.0.0/masakarimonitors/tests/base.py0000664000175000017500000000140113656750014023216 0ustar zuulzuul00000000000000# Copyright 2010-2011 OpenStack Foundation # Copyright (c) 2013 Hewlett-Packard Development Company, L.P. # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. from oslotest import base class TestCase(base.BaseTestCase): """Test case base class for all unit tests.""" masakari-monitors-9.0.0/masakarimonitors/manager.py0000664000175000017500000000674413656750014022573 0ustar zuulzuul00000000000000# Copyright(c) 2016 Nippon Telegraph and Telephone Corporation # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. """Base Manager class. Managers are responsible for a certain aspect of the system. It is a logical grouping of code relating to a portion of the system. In general other components should be using the manager to make changes to the components that it is responsible for. We have adopted a basic strategy of Smart managers and dumb data, which means rather than attaching methods to data objects, components should call manager methods that act on the data. Methods on managers that can be executed locally should be called directly. If a particular method must execute on a remote host, this should be done via rpc to the service that wraps the manager. Managers should be responsible for most of the db access, and non-implementation specific data. Anything implementation specific that can't be generalized should be done by the Driver. Managers will often provide methods for initial setup of a host or periodic tasks to a wrapping service. This module provides Manager, a base class for managers. """ from oslo_service import periodic_task import masakarimonitors.conf CONF = masakarimonitors.conf.CONF class PeriodicTasks(periodic_task.PeriodicTasks): def __init__(self): super(PeriodicTasks, self).__init__(CONF) class Manager(PeriodicTasks): def __init__(self, host=None, service_name='undefined'): if not host: host = CONF.hostname self.host = host self.service_name = service_name super(Manager, self).__init__() def periodic_tasks(self, context, raise_on_error=False): """Tasks to be run at a periodic interval.""" return self.run_periodic_tasks(context, raise_on_error=raise_on_error) def init_host(self): """Hook to do additional manager initialization when one requests the service be started. This is called before any service record is created. Child classes should override this method. """ pass def cleanup_host(self): """Hook to do cleanup work when the service shuts down. Child classes should override this method. """ pass def pre_start_hook(self): """Hook to provide the manager the ability to do additional start-up work before any RPC queues/consumers are created. This is called after other initialization has succeeded and a service record is created. Child classes should override this method. """ pass def post_start_hook(self): """Hook to provide the manager the ability to do additional start-up work immediately after a service creates RPC consumers and starts 'running'. Child classes should override this method. """ pass def reset(self): """Hook called on SIGHUP to signal the manager to re-read any dynamic configuration or do any reconfiguration tasks. """ pass masakari-monitors-9.0.0/masakarimonitors/ha/0000775000175000017500000000000013656750123021165 5ustar zuulzuul00000000000000masakari-monitors-9.0.0/masakarimonitors/ha/__init__.py0000664000175000017500000000000013656750014023263 0ustar zuulzuul00000000000000masakari-monitors-9.0.0/masakarimonitors/ha/masakari.py0000664000175000017500000000727613656750014023342 0ustar zuulzuul00000000000000# Copyright(c) 2016 Nippon Telegraph and Telephone Corporation # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. import eventlet from keystoneauth1.identity.generic import password as ks_password from keystoneauth1 import session as ks_session from openstack import connection from openstack import exceptions from oslo_log import log as oslo_logging import masakarimonitors.conf LOG = oslo_logging.getLogger(__name__) CONF = masakarimonitors.conf.CONF class SendNotification(object): def _make_client(self): auth = ks_password.Password( auth_url=CONF.api.auth_url, username=CONF.api.username, password=CONF.api.password, user_domain_id=CONF.api.user_domain_id, project_name=CONF.api.project_name, project_domain_id=CONF.api.project_domain_id) session = ks_session.Session(auth=auth) conn = connection.Connection(session=session, interface=CONF.api.api_interface, region_name=CONF.api.region) return conn.instance_ha def send_notification(self, api_retry_max, api_retry_interval, event): """Send a notification. This method sends a notification to the masakari-api. :param api_retry_max: Number of retries when the notification processing is error. :param api_retry_interval: Trial interval of time of the notification processing is error. :param event: dictionary of event that included in notification. """ LOG.info("Send a notification. %s", event) # Get client. client = self._make_client() # Send a notification. retry_count = 0 while True: try: response = client.create_notification( type=event['notification']['type'], hostname=event['notification']['hostname'], generated_time=event['notification']['generated_time'], payload=event['notification']['payload']) LOG.info("Response: %s", response) break except Exception as e: if isinstance(e, exceptions.HttpException): # TODO(samP): Remove attribute check and else case if # openstacksdk is bumped up from '>=0.9.19' to '>=0.10.0' # in global-requirements. if hasattr(e, 'status_code'): is_status_409 = e.status_code == 409 else: is_status_409 = e.http_status == 409 if is_status_409: msg = ("Stop retrying to send a notification because " "same notification have been already sent.") LOG.info("%s", msg) break if retry_count < api_retry_max: LOG.warning("Retry sending a notification. (%s)", e) retry_count = retry_count + 1 eventlet.greenthread.sleep(api_retry_interval) else: LOG.exception("Exception caught: %s", e) break masakari-monitors-9.0.0/masakarimonitors/i18n.py0000664000175000017500000000165713656750014021736 0ustar zuulzuul00000000000000# Copyright 2016 NTT DATA # # Licensed under the Apache License, Version 2.0 (the "License"); you may # not use this file except in compliance with the License. You may obtain # a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, WITHOUT # WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the # License for the specific language governing permissions and limitations # under the License. import oslo_i18n DOMAIN = 'masakarimonitors' _translators = oslo_i18n.TranslatorFactory(domain=DOMAIN) # The primary translation function using the well-known name "_" _ = _translators.primary def translate(value, user_locale): return oslo_i18n.translate(value, user_locale) def get_available_languages(): return oslo_i18n.get_available_languages(DOMAIN) masakari-monitors-9.0.0/masakarimonitors/service.py0000664000175000017500000000766113656750014022620 0ustar zuulzuul00000000000000# Copyright(c) 2016 Nippon Telegraph and Telephone Corporation # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. """Generic Node base class for all workers that run on hosts.""" import os import sys from oslo_log import log as logging from oslo_service import service from oslo_utils import importutils import masakarimonitors.conf from masakarimonitors.i18n import _ from masakarimonitors import utils LOG = logging.getLogger(__name__) CONF = masakarimonitors.conf.CONF class Service(service.Service): """Service object for binaries running on hosts. A service takes a manager. """ def __init__(self, host, binary, manager): super(Service, self).__init__() self.host = host self.binary = binary self.manager_class_name = manager manager_class = importutils.import_class(self.manager_class_name) self.manager = manager_class(host=self.host) def __repr__(self): return "<%(cls_name)s: host=%(host)s, binary=%(binary)s, " \ "manager_class_name=%(manager)s>" %\ { 'cls_name': self.__class__.__name__, 'host': self.host, 'binary': self.binary, 'manager': self.manager_class_name } def start(self): LOG.info('Starting %s', self.binary) self.basic_config_check() self.manager.init_host() self.manager.main() def __getattr__(self, key): manager = self.__dict__.get('manager', None) return getattr(manager, key) @classmethod def create(cls, host=None, binary=None, manager=None): """Instantiates class and passes back application object. :param host: defaults to CONF.hostname :param binary: defaults to basename of executable :param manager: defaults to CONF._manager """ if not host: host = CONF.hostname if not binary: binary = os.path.basename(sys.argv[0]) if not manager: manager_cls = ('%s_manager' % binary.rpartition('masakarimonitors-')[2]) manager = CONF.get(manager_cls, None) service_obj = cls(host, binary, manager) return service_obj def kill(self): """Destroy the service object in the datastore. NOTE: Although this method is not used anywhere else than tests, it is convenient to have it here, so the tests might easily and in clean way stop and remove the service_ref. """ self.stop() def stop(self): LOG.info('Stopping %s', self.binary) self.manager.stop() super(Service, self).stop() def basic_config_check(self): """Perform basic config checks before starting processing.""" # Make sure the tempdir exists and is writable try: with utils.tempdir(): pass except Exception as e: LOG.error('Temporary directory is invalid: %s', e) sys.exit(1) def reset(self): self.manager.reset() # NOTE: the global launcher is to maintain the existing # functionality of calling service.serve + # service.wait _launcher = None def serve(server, workers=None): global _launcher if _launcher: raise RuntimeError(_('serve() can only be called once')) _launcher = service.launch(CONF, server, workers=workers) def wait(): _launcher.wait() masakari-monitors-9.0.0/masakarimonitors/objects/0000775000175000017500000000000013656750123022226 5ustar zuulzuul00000000000000masakari-monitors-9.0.0/masakarimonitors/objects/__init__.py0000664000175000017500000000000013656750014024324 0ustar zuulzuul00000000000000masakari-monitors-9.0.0/masakarimonitors/objects/event_constants.py0000664000175000017500000000202613656750014026014 0ustar zuulzuul00000000000000# Copyright(c) 2016 Nippon Telegraph and Telephone Corporation # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. class EventConstants(object): # Define event types. TYPE_PROCESS = "PROCESS" TYPE_COMPUTE_HOST = "COMPUTE_HOST" TYPE_VM = "VM" # Define event details. EVENT_STARTED = "STARTED" EVENT_STOPPED = "STOPPED" # Define host status. HOST_STATUS_NORMAL = "NORMAL" HOST_STATUS_UNKNOWN = "UNKNOWN" # Define cluster status. CLUSTER_STATUS_ONLINE = "ONLINE" CLUSTER_STATUS_OFFLINE = "OFFLINE" masakari-monitors-9.0.0/masakarimonitors/instancemonitor/0000775000175000017500000000000013656750123024011 5ustar zuulzuul00000000000000masakari-monitors-9.0.0/masakarimonitors/instancemonitor/libvirt_handler/0000775000175000017500000000000013656750123027161 5ustar zuulzuul00000000000000masakari-monitors-9.0.0/masakarimonitors/instancemonitor/libvirt_handler/__init__.py0000664000175000017500000000000013656750014031257 0ustar zuulzuul00000000000000masakari-monitors-9.0.0/masakarimonitors/instancemonitor/libvirt_handler/eventfilter.py0000664000175000017500000000567313656750014032074 0ustar zuulzuul00000000000000# Copyright(c) 2016 Nippon Telegraph and Telephone Corporation # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. import socket import sys import threading from oslo_log import log as oslo_logging from oslo_utils import excutils from oslo_utils import timeutils import masakarimonitors.conf from masakarimonitors.instancemonitor.libvirt_handler import callback from masakarimonitors.instancemonitor.libvirt_handler \ import eventfilter_table as evft from masakarimonitors.objects import event_constants as ec LOG = oslo_logging.getLogger(__name__) CONF = masakarimonitors.conf.CONF class EventFilter(object): """Class of filtering events.""" def __init__(self): self.callback = callback.Callback() def vir_event_filter(self, eventID, eventType, detail, uuID): """Filter events from libvirt. :param eventID: EventID :param eventType: Event type :param detail: Event name :pram uuID: UUID """ noticeType = ec.EventConstants.TYPE_VM hostname = CONF.hostname or socket.gethostname() currentTime = timeutils.utcnow() # All Event Output if debug mode is on. msg = "libvirt Event Received.type = %s \ hostname = %s uuid = %s time = %s eventID = %d eventType = %d \ detail = %d" % ( noticeType, hostname, uuID, currentTime, eventID, eventType, detail) LOG.debug(msg) try: if detail in evft.event_filter_dic[eventID][eventType]: LOG.debug("Event Filter Matched.") eventID_val = evft.eventID_dic[eventID] detail_val = evft.detail_dic[eventID][eventType][detail] # callback Thread Start thread = threading.Thread( target=self.callback.libvirt_event_callback, args=(eventID_val, detail_val, uuID, noticeType, hostname, currentTime) ) thread.start() else: LOG.debug("Event Filter Unmatched.") except KeyError: LOG.debug("virEventFilter KeyError") except IndexError: LOG.debug("virEventFilter IndexError") except TypeError: LOG.debug("virEventFilter TypeError") except Exception: with excutils.save_and_reraise_exception(): LOG.debug("Unexpected error: %s" % sys.exc_info()[0]) masakari-monitors-9.0.0/masakarimonitors/instancemonitor/libvirt_handler/eventfilter_table.py0000664000175000017500000001105713656750014033234 0ustar zuulzuul00000000000000# Copyright(c) 2016 Nippon Telegraph and Telephone Corporation # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. import libvirt # If is not defined internal , -1 is stored. DUMMY = -1 # Enumerate all event that can get. # Comment out events that is not targeted in the callback. event_filter_dic = { libvirt.VIR_DOMAIN_EVENT_ID_LIFECYCLE: { libvirt.VIR_DOMAIN_EVENT_SUSPENDED: ( libvirt.VIR_DOMAIN_EVENT_SUSPENDED_IOERROR, libvirt.VIR_DOMAIN_EVENT_SUSPENDED_WATCHDOG, libvirt.VIR_DOMAIN_EVENT_SUSPENDED_API_ERROR ), libvirt.VIR_DOMAIN_EVENT_STOPPED: ( libvirt.VIR_DOMAIN_EVENT_STOPPED_SHUTDOWN, libvirt.VIR_DOMAIN_EVENT_STOPPED_DESTROYED, libvirt.VIR_DOMAIN_EVENT_STOPPED_FAILED, ), libvirt.VIR_DOMAIN_EVENT_SHUTDOWN: ( libvirt.VIR_DOMAIN_EVENT_SHUTDOWN_FINISHED, ) }, libvirt.VIR_DOMAIN_EVENT_ID_REBOOT: {DUMMY: (DUMMY,)}, libvirt.VIR_DOMAIN_EVENT_ID_WATCHDOG: { libvirt.VIR_DOMAIN_EVENT_WATCHDOG_NONE: (DUMMY,), libvirt.VIR_DOMAIN_EVENT_WATCHDOG_PAUSE: (DUMMY,), libvirt.VIR_DOMAIN_EVENT_WATCHDOG_RESET: (DUMMY,), libvirt.VIR_DOMAIN_EVENT_WATCHDOG_POWEROFF: (DUMMY,), libvirt.VIR_DOMAIN_EVENT_WATCHDOG_SHUTDOWN: (DUMMY,), libvirt.VIR_DOMAIN_EVENT_WATCHDOG_DEBUG: (DUMMY,) }, libvirt.VIR_DOMAIN_EVENT_ID_IO_ERROR: { libvirt.VIR_DOMAIN_EVENT_IO_ERROR_NONE: (DUMMY,), libvirt.VIR_DOMAIN_EVENT_IO_ERROR_PAUSE: (DUMMY,), libvirt.VIR_DOMAIN_EVENT_IO_ERROR_REPORT: (DUMMY,) }, libvirt.VIR_DOMAIN_EVENT_ID_IO_ERROR_REASON: {DUMMY: (DUMMY,)}, libvirt.VIR_DOMAIN_EVENT_ID_CONTROL_ERROR: {DUMMY: (DUMMY,)} } eventID_dic = { libvirt.VIR_DOMAIN_EVENT_ID_LIFECYCLE: 'LIFECYCLE', libvirt.VIR_DOMAIN_EVENT_ID_REBOOT: 'REBOOT', libvirt.VIR_DOMAIN_EVENT_ID_WATCHDOG: 'WATCHDOG', libvirt.VIR_DOMAIN_EVENT_ID_IO_ERROR: 'IO_ERROR', libvirt.VIR_DOMAIN_EVENT_ID_IO_ERROR_REASON: 'IO_ERROR_REASON', libvirt.VIR_DOMAIN_EVENT_ID_CONTROL_ERROR: 'CONTROL_ERROR'} detail_dic = { libvirt.VIR_DOMAIN_EVENT_ID_LIFECYCLE: { libvirt.VIR_DOMAIN_EVENT_SUSPENDED: { libvirt.VIR_DOMAIN_EVENT_SUSPENDED_IOERROR: 'SUSPENDED_IOERROR', libvirt.VIR_DOMAIN_EVENT_SUSPENDED_WATCHDOG: 'SUSPENDED_WATCHDOG', libvirt.VIR_DOMAIN_EVENT_SUSPENDED_API_ERROR: 'SUSPENDED_API_ERROR'}, libvirt.VIR_DOMAIN_EVENT_STOPPED: { libvirt.VIR_DOMAIN_EVENT_STOPPED_SHUTDOWN: 'STOPPED_SHUTDOWN', libvirt.VIR_DOMAIN_EVENT_STOPPED_DESTROYED: 'STOPPED_DESTROYED', libvirt.VIR_DOMAIN_EVENT_STOPPED_FAILED: 'STOPPED_FAILED'}, libvirt.VIR_DOMAIN_EVENT_SHUTDOWN: { libvirt.VIR_DOMAIN_EVENT_SHUTDOWN_FINISHED: 'SHUTDOWN_FINISHED'} }, libvirt.VIR_DOMAIN_EVENT_ID_REBOOT: { DUMMY: { DUMMY: 'UNKNOWN'}}, libvirt.VIR_DOMAIN_EVENT_ID_WATCHDOG: { libvirt.VIR_DOMAIN_EVENT_WATCHDOG_NONE: { DUMMY: 'WATCHDOG_NONE'}, libvirt.VIR_DOMAIN_EVENT_WATCHDOG_PAUSE: { DUMMY: 'WATCHDOG_PAUSE'}, libvirt.VIR_DOMAIN_EVENT_WATCHDOG_RESET: { DUMMY: 'WATCHDOG_RESET'}, libvirt.VIR_DOMAIN_EVENT_WATCHDOG_POWEROFF: { DUMMY: 'WATCHDOG_POWEROFF'}, libvirt.VIR_DOMAIN_EVENT_WATCHDOG_SHUTDOWN: { DUMMY: 'WATCHDOG_SHUTDOWN'}, libvirt.VIR_DOMAIN_EVENT_WATCHDOG_DEBUG: { DUMMY: 'WATCHDOG_DEBUG'}}, libvirt.VIR_DOMAIN_EVENT_ID_IO_ERROR: { libvirt.VIR_DOMAIN_EVENT_IO_ERROR_NONE: { DUMMY: 'IO_ERROR_NONE'}, libvirt.VIR_DOMAIN_EVENT_IO_ERROR_PAUSE: { DUMMY: 'IO_ERROR_PAUSE'}, libvirt.VIR_DOMAIN_EVENT_IO_ERROR_REPORT: { DUMMY: 'IO_ERROR_REPORT'}}, libvirt.VIR_DOMAIN_EVENT_ID_IO_ERROR_REASON: { DUMMY: { DUMMY: 'UNKNOWN'}}, libvirt.VIR_DOMAIN_EVENT_ID_CONTROL_ERROR: { DUMMY: { DUMMY: 'UNKNOWN'}} } masakari-monitors-9.0.0/masakarimonitors/instancemonitor/libvirt_handler/callback.py0000664000175000017500000000560513656750014031274 0ustar zuulzuul00000000000000# Copyright(c) 2016 Nippon Telegraph and Telephone Corporation # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. from oslo_log import log as oslo_logging import masakarimonitors.conf from masakarimonitors.ha import masakari LOG = oslo_logging.getLogger(__name__) CONF = masakarimonitors.conf.CONF class Callback(object): """Class of callback processing.""" def __init__(self): self.notifier = masakari.SendNotification() def libvirt_event_callback(self, event_id, details, domain_uuid, notice_type, hostname, current_time): """Callback method. Callback processing be executed as result of the libvirt event filter. :param event_id: Event ID notify to the callback function :param details: Event code notify to the callback function :param domain_uuid: Uuid notify to the callback function :param notice_type: Notice type notify to the callback function :param hostname: Server host name of the source an event occur notify to the callback function :param current_time: Event occurred time notify to the callback function """ # Output to the syslog. LOG.info("Libvirt Event: type=%(notice_type)s," " hostname=%(hostname)s," " uuid=%(uuid)s, time=%(current_time)s," " event_id=%(event_id)s," " detail=%(detail)s)" % {'notice_type': notice_type, 'hostname': hostname, 'uuid': domain_uuid, 'current_time': current_time, 'event_id': event_id, 'detail': details}) # Set the event to the dictionary. event = { 'notification': { 'type': notice_type, 'hostname': hostname, 'generated_time': current_time, 'payload': { 'event': event_id, 'instance_uuid': domain_uuid, 'vir_domain_event': details } } } self.notifier.send_notification(CONF.callback.retry_max, CONF.callback.retry_interval, event) masakari-monitors-9.0.0/masakarimonitors/instancemonitor/__init__.py0000664000175000017500000000000013656750014026107 0ustar zuulzuul00000000000000masakari-monitors-9.0.0/masakarimonitors/instancemonitor/instance.py0000664000175000017500000001536613656750014026201 0ustar zuulzuul00000000000000# Copyright(c) 2016 Nippon Telegraph and Telephone Corporation # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. import threading import time import eventlet import libvirt from oslo_config import cfg from oslo_log import log as oslo_logging from masakarimonitors.instancemonitor.libvirt_handler import eventfilter from masakarimonitors import manager LOG = oslo_logging.getLogger(__name__) CONF = cfg.CONF class InstancemonitorManager(manager.Manager): """Manages the masakari-instancemonitor.""" def __init__(self, *args, **kwargs): super(InstancemonitorManager, self).__init__( service_name="instancemonitor", *args, **kwargs) self.evf = eventfilter.EventFilter() # This keeps track of what thread is running the event loop, # (if it is run in a background thread) self.event_loop_thread = None def _vir_event_loop_native_run(self): # Directly run the event loop in the current thread while True: libvirt.virEventRunDefaultImpl() def _vir_event_loop_native_start(self): libvirt.virEventRegisterDefaultImpl() self.event_loop_thread = threading.Thread( target=self._vir_event_loop_native_run, name="lib_virt_eventLoop") self.event_loop_thread.setDaemon(True) self.event_loop_thread.start() def _my_domain_event_callback(self, conn, dom, event, detail, opaque): self.evf.vir_event_filter(libvirt.VIR_DOMAIN_EVENT_ID_LIFECYCLE, event, detail, dom.UUIDString()) def _my_domain_event_reboot_callback(self, conn, dom, opaque): self.evf.vir_event_filter(libvirt.VIR_DOMAIN_EVENT_ID_REBOOT, -1, -1, dom.UUIDString()) def _my_domain_event_rtc_change_callback(self, conn, dom, utcoffset, opaque): self.evf.vir_event_filter(libvirt.VIR_DOMAIN_EVENT_ID_RTC_CHANGE, -1, -1, dom.UUIDString()) def _my_domain_event_watchdog_callback(self, conn, dom, action, opaque): self.evf.vir_event_filter(libvirt.VIR_DOMAIN_EVENT_ID_WATCHDOG, action, -1, dom.UUIDString()) def _my_domain_event_io_error_callback(self, conn, dom, srcpath, devalias, action, opaque): self.evf.vir_event_filter(libvirt.VIR_DOMAIN_EVENT_ID_IO_ERROR, action, -1, dom.UUIDString()) def _my_domain_event_graphics_callback(self, conn, dom, phase, localAddr, remoteAddr, authScheme, subject, opaque): self.evf.vir_event_filter(libvirt.VIR_DOMAIN_EVENT_ID_GRAPHICS, -1, phase, dom.UUIDString()) def _my_domain_event_disk_change_callback(self, conn, dom, oldSrcPath, newSrcPath, devAlias, reason, opaque): self.evf.vir_event_filter(libvirt.VIR_DOMAIN_EVENT_ID_DISK_CHANGE, -1, -1, dom.UUIDString()) def _my_domain_event_io_error_reason_callback(self, conn, dom, srcPath, devAlias, action, reason, opaque): self.evf.vir_event_filter(libvirt.VIR_DOMAIN_EVENT_ID_IO_ERROR_REASON, -1, -1, dom.UUIDString()) def _my_domain_event_generic_callback(self, conn, dom, opaque): self.evf.vir_event_filter(libvirt.VIR_DOMAIN_EVENT_ID_CONTROL_ERROR, -1, -1, dom.UUIDString()) def _err_handler(self, ctxt, err): LOG.warning("Error from libvirt : %s", err[2]) def _virt_event(self, uri): # Run a background thread with the event loop self._vir_event_loop_native_start() event_callback_handlers = { libvirt.VIR_DOMAIN_EVENT_ID_LIFECYCLE: self._my_domain_event_callback, libvirt.VIR_DOMAIN_EVENT_ID_REBOOT: self._my_domain_event_reboot_callback, libvirt.VIR_DOMAIN_EVENT_ID_RTC_CHANGE: self._my_domain_event_rtc_change_callback, libvirt.VIR_DOMAIN_EVENT_ID_IO_ERROR: self._my_domain_event_io_error_callback, libvirt.VIR_DOMAIN_EVENT_ID_WATCHDOG: self._my_domain_event_watchdog_callback, libvirt.VIR_DOMAIN_EVENT_ID_GRAPHICS: self._my_domain_event_graphics_callback, libvirt.VIR_DOMAIN_EVENT_ID_DISK_CHANGE: self._my_domain_event_disk_change_callback, libvirt.VIR_DOMAIN_EVENT_ID_IO_ERROR_REASON: self._my_domain_event_io_error_reason_callback, libvirt.VIR_DOMAIN_EVENT_ID_CONTROL_ERROR: self._my_domain_event_generic_callback } # Connect to libvirt - If be disconnected, reprocess. self.running = True while self.running: vc = libvirt.openReadOnly(uri) # Event callback settings callback_ids = [] for event, callback in event_callback_handlers.items(): cid = vc.domainEventRegisterAny(None, event, callback, None) callback_ids.append(cid) # Connection monitoring. vc.setKeepAlive(5, 3) while vc.isAlive() == 1 and self.running: eventlet.greenthread.sleep(1) # If connection between libvirtd was lost, # clear callback connection. LOG.warning("Libvirt Connection Closed Unexpectedly.") for cid in callback_ids: try: vc.domainEventDeregisterAny(cid) except Exception: pass vc.close() del vc time.sleep(3) def stop(self): self.running = False def main(self): """Main method. Set the URI, error handler, and executes event loop processing. """ uri = CONF.libvirt.connection_uri LOG.debug("Using uri:" + uri) # set error handler & do event loop libvirt.registerErrorHandler(self._err_handler, '_virt_event') self._virt_event(uri) masakari-monitors-9.0.0/masakarimonitors/cmd/0000775000175000017500000000000013656750123021340 5ustar zuulzuul00000000000000masakari-monitors-9.0.0/masakarimonitors/cmd/__init__.py0000664000175000017500000000000013656750014023436 0ustar zuulzuul00000000000000masakari-monitors-9.0.0/masakarimonitors/cmd/hostmonitor.py0000664000175000017500000000213513656750014024277 0ustar zuulzuul00000000000000# Copyright(c) 2016 Nippon Telegraph and Telephone Corporation # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. """Starter script for Masakari Host Monitor.""" import sys from oslo_log import log as logging import masakarimonitors.conf from masakarimonitors import config from masakarimonitors import service from masakarimonitors import utils CONF = masakarimonitors.conf.CONF def main(): config.parse_args(sys.argv) logging.setup(CONF, "masakarimonitors") utils.monkey_patch() server = service.Service.create(binary='masakarimonitors-hostmonitor') service.serve(server) service.wait() masakari-monitors-9.0.0/masakarimonitors/cmd/instancemonitor.py0000664000175000017500000000214513656750014025127 0ustar zuulzuul00000000000000# Copyright(c) 2016 Nippon Telegraph and Telephone Corporation # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. """Starter script for Masakari Instance Monitor.""" import sys from oslo_log import log as logging import masakarimonitors.conf from masakarimonitors import config from masakarimonitors import service from masakarimonitors import utils CONF = masakarimonitors.conf.CONF def main(): config.parse_args(sys.argv) logging.setup(CONF, "masakarimonitors") utils.monkey_patch() server = service.Service.create(binary='masakarimonitors-instancemonitor') service.serve(server) service.wait() masakari-monitors-9.0.0/masakarimonitors/cmd/processmonitor.py0000664000175000017500000000214313656750014024777 0ustar zuulzuul00000000000000# Copyright(c) 2016 Nippon Telegraph and Telephone Corporation # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. """Starter script for Masakari Process Monitor.""" import sys from oslo_log import log as logging import masakarimonitors.conf from masakarimonitors import config from masakarimonitors import service from masakarimonitors import utils CONF = masakarimonitors.conf.CONF def main(): config.parse_args(sys.argv) logging.setup(CONF, "masakarimonitors") utils.monkey_patch() server = service.Service.create(binary='masakarimonitors-processmonitor') service.serve(server) service.wait() masakari-monitors-9.0.0/masakarimonitors/cmd/introspectiveinstancemonitor.py0000664000175000017500000000216013656750014027743 0ustar zuulzuul00000000000000# Copyright(c) 2018 WindRiver Systems # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. """Starter script for Masakari Introspective Instance Monitor.""" import sys from oslo_log import log as logging import masakarimonitors.conf from masakarimonitors import config from masakarimonitors import service from masakarimonitors import utils CONF = masakarimonitors.conf.CONF def main(): config.parse_args(sys.argv) logging.setup(CONF, "masakarimonitors") utils.monkey_patch() server = service.Service.create( binary='masakarimonitors-introspectiveinstancemonitor') service.serve(server) service.wait() masakari-monitors-9.0.0/masakarimonitors/conf/0000775000175000017500000000000013656750123021522 5ustar zuulzuul00000000000000masakari-monitors-9.0.0/masakarimonitors/conf/__init__.py0000664000175000017500000000222213656750014023630 0ustar zuulzuul00000000000000# Copyright(c) 2016 Nippon Telegraph and Telephone Corporation # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. from oslo_config import cfg from masakarimonitors.conf import api from masakarimonitors.conf import base from masakarimonitors.conf import host from masakarimonitors.conf import instance from masakarimonitors.conf import introspectiveinstancemonitor from masakarimonitors.conf import process from masakarimonitors.conf import service CONF = cfg.CONF api.register_opts(CONF) base.register_opts(CONF) host.register_opts(CONF) instance.register_opts(CONF) introspectiveinstancemonitor.register_opts(CONF) process.register_opts(CONF) service.register_opts(CONF) masakari-monitors-9.0.0/masakarimonitors/conf/api.py0000664000175000017500000000304013656750014022641 0ustar zuulzuul00000000000000# Copyright(c) 2016 Nippon Telegraph and Telephone Corporation # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. from keystoneauth1 import loading as ks_loading from oslo_config import cfg API_GROUP = 'api' api_group = cfg.OptGroup( API_GROUP, title='Api Options', help=""" Configuration options for sending notifications. """) monitor_api_opts = [ cfg.StrOpt('region', default='RegionOne', help='Region name.'), cfg.StrOpt('api_version', default='v1', help='Masakari API Version.'), cfg.StrOpt('api_interface', default='public', help='Interface of endpoint.'), ] def register_opts(conf): conf.register_group(api_group) conf.register_opts(monitor_api_opts, group=api_group) conf.register_opts(ks_loading.get_auth_plugin_conf_options('password'), group=api_group) def list_opts(): return { api_group: ( monitor_api_opts + ks_loading.get_auth_plugin_conf_options('password')) } masakari-monitors-9.0.0/masakarimonitors/conf/process.py0000664000175000017500000000334713656750014023560 0ustar zuulzuul00000000000000# Copyright(c) 2016 Nippon Telegraph and Telephone Corporation # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. from oslo_config import cfg monitor_process_opts = [ cfg.IntOpt('check_interval', default=5, help='Interval in seconds for checking a process.'), cfg.IntOpt('restart_retries', default=3, help='Number of retries when the failure of restarting a' ' process.'), cfg.IntOpt('restart_interval', default=5, help='Interval in seconds for restarting a process.'), cfg.IntOpt('api_retry_max', default=12, help='Number of retries for send a notification in' ' processmonitor.'), cfg.IntOpt('api_retry_interval', default=10, help='Interval between re-sending a notification in' ' processmonitor(in seconds).'), cfg.StrOpt('process_list_path', default='/etc/masakarimonitors/process_list.yaml', help='The file path of process list.'), ] def register_opts(conf): conf.register_opts(monitor_process_opts, group='process') def list_opts(): return { 'process': monitor_process_opts } masakari-monitors-9.0.0/masakarimonitors/conf/opts.py0000664000175000017500000000570513656750014023067 0ustar zuulzuul00000000000000# Copyright(c) 2016 Nippon Telegraph and Telephone Corporation # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. """ This is the single point of entry to generate the sample configuration file for Masakari Monitors. It collects all the necessary info from the other modules in this package. It is assumed that: * every other module in this package has a 'list_opts' function which return a dict where * the keys are strings which are the group names * the value of each key is a list of config options for that group * the masakari.conf package doesn't have further packages with config options * this module is only used in the context of sample file generation """ import collections import importlib import os import pkgutil LIST_OPTS_FUNC_NAME = "list_opts" def _tupleize(dct): """Take the dict of options and convert to the 2-tuple format.""" return [(key, val) for key, val in dct.items()] def list_opts(): opts = collections.defaultdict(list) module_names = _list_module_names() imported_modules = _import_modules(module_names) _append_config_options(imported_modules, opts) return _tupleize(opts) def _list_module_names(): module_names = [] package_path = os.path.dirname(os.path.abspath(__file__)) for _, modname, ispkg in pkgutil.iter_modules(path=[package_path]): if modname == "opts" or ispkg: continue else: module_names.append(modname) return module_names def _import_modules(module_names): imported_modules = [] for modname in module_names: mod = importlib.import_module("masakarimonitors.conf." + modname) if not hasattr(mod, LIST_OPTS_FUNC_NAME): msg = "The module 'masakarimonitors.conf.%s' should have a" \ " '%s' function which returns the config options." % \ (modname, LIST_OPTS_FUNC_NAME) raise Exception(msg) else: imported_modules.append(mod) return imported_modules def _process_old_opts(configs): """Convert old-style 2-tuple configs to dicts.""" if isinstance(configs, tuple): configs = [configs] return {label: options for label, options in configs} def _append_config_options(imported_modules, config_options): for mod in imported_modules: configs = mod.list_opts() if not isinstance(configs, dict): configs = _process_old_opts(configs) for key, val in configs.items(): config_options[key].extend(val) masakari-monitors-9.0.0/masakarimonitors/conf/host.py0000664000175000017500000000677213656750014023064 0ustar zuulzuul00000000000000# Copyright(c) 2016 Nippon Telegraph and Telephone Corporation # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. from oslo_config import cfg monitor_host_opts = [ cfg.StrOpt('monitoring_driver', default='default', help='Driver that hostmonitor uses for monitoring hosts.'), cfg.IntOpt('monitoring_interval', default=60, help='Monitoring interval(in seconds) of node status.'), cfg.IntOpt('api_retry_max', default=12, help='Number of retries for send a notification in' ' processmonitor.'), cfg.IntOpt('api_retry_interval', default=10, help='Trial interval of time of the notification processing' ' is error(in seconds).'), cfg.BoolOpt('disable_ipmi_check', default=False, help=''' Do not check whether the host is completely down. Possible values: * True: Do not check whether the host is completely down. * False: Do check whether the host is completely down. If ipmi RA is not set in pacemaker, this value should be set True. '''), cfg.BoolOpt('restrict_to_remotes', default=False, help='Only monitor pacemaker-remotes, ignore the status of' ' full cluster members.'), cfg.IntOpt('ipmi_timeout', default=5, help='Timeout value(in seconds) of the ipmitool command.'), cfg.IntOpt('ipmi_retry_max', default=3, help='Number of ipmitool command retries.'), cfg.IntOpt('ipmi_retry_interval', default=10, help='Retry interval(in seconds) of the ipmitool command.'), cfg.IntOpt('stonith_wait', default=30, help='Standby time(in seconds) until activate STONITH.'), cfg.IntOpt('tcpdump_timeout', default=5, help='Timeout value(in seconds) of the tcpdump command when' ' monitors the corosync communication.'), cfg.StrOpt('corosync_multicast_interfaces', help=''' The name of interface that corosync is using for mutual communication between hosts. If there are multiple interfaces, specify them in comma-separated like 'enp0s3,enp0s8'. The number of interfaces you specify must be equal to the number of corosync_multicast_ports values and must be in correct order with relevant ports in corosync_multicast_ports. '''), cfg.StrOpt('corosync_multicast_ports', help=''' The port numbers that corosync is using for mutual communication between hosts. If there are multiple port numbers, specify them in comma-separated like '5405,5406'. The number of port numbers you specify must be equal to the number of corosync_multicast_interfaces values and must be in correct order with relevant interfaces in corosync_multicast_interfaces. '''), ] def register_opts(conf): conf.register_opts(monitor_host_opts, group='host') def list_opts(): return { 'host': monitor_host_opts } masakari-monitors-9.0.0/masakarimonitors/conf/instance.py0000664000175000017500000000261613656750014023704 0ustar zuulzuul00000000000000# Copyright(c) 2016 Nippon Telegraph and Telephone Corporation # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. from oslo_config import cfg monitor_callback_opts = [ cfg.IntOpt('retry_max', default=12, help='Number of retries when the notification processing' ' is error.'), cfg.IntOpt('retry_interval', default=10, help='Trial interval of time of the notification processing' ' is error(in seconds).'), ] libvirt_opts = [ cfg.StrOpt('connection_uri', default='qemu:///system', help='Override the default libvirt URI.') ] def register_opts(conf): conf.register_opts(monitor_callback_opts, group='callback') conf.register_opts(libvirt_opts, group='libvirt') def list_opts(): return { 'callback': monitor_callback_opts, 'libvirt': libvirt_opts } masakari-monitors-9.0.0/masakarimonitors/conf/service.py0000664000175000017500000000402013656750014023527 0ustar zuulzuul00000000000000# Copyright(c) 2016 Nippon Telegraph and Telephone Corporation # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. import socket from oslo_config import cfg service_opts = [ cfg.StrOpt('hostname', default=socket.gethostname(), deprecated_name="host", help=''' Hostname, FQDN or IP address of this host. Must be valid within AMQP key. Possible values: * String with hostname, FQDN or IP address. Default is hostname of this host. '''), cfg.StrOpt('instancemonitor_manager', default='masakarimonitors.instancemonitor.instance' '.InstancemonitorManager', help='Full class name for the Manager for instancemonitor.'), cfg.StrOpt('introspectiveinstancemonitor_manager', default='masakarimonitors.introspectiveinstancemonitor.instance' '.IntrospectiveInstanceMonitorManager', help='Full class name for introspectiveinstancemonitor.'), cfg.StrOpt('processmonitor_manager', default='masakarimonitors.processmonitor.process' '.ProcessmonitorManager', help='Full class name for the Manager for processmonitor.'), cfg.StrOpt('hostmonitor_manager', default='masakarimonitors.hostmonitor.host' '.HostmonitorManager', help='Full class name for the Manager for hostmonitor.'), ] def register_opts(conf): conf.register_opts(service_opts) def list_opts(): return {'DEFAULT': service_opts} masakari-monitors-9.0.0/masakarimonitors/conf/introspectiveinstancemonitor.py0000664000175000017500000000456113656750014030134 0ustar zuulzuul00000000000000# Copyright(c) 2018 WindRiver Systems # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. from oslo_config import cfg # Note: this string is being used for regex parsing later with re module. # # Use Python's raw string notation for regular expressions and # uses the backslash character ('\') to indicate special # forms or to allow special characters to be used without invoking # their special meaning. SOCK = r'/var/lib/libvirt/qemu/org\.qemu\.guest_agent\..*\.instance-.*\.sock' monitor_opts = [ cfg.IntOpt('guest_monitoring_interval', default=10, help=''' Guest monitoring interval of VM status (in seconds). * The value should not be too low as there should not be false negative * for reporting QEMU_GUEST_AGENT failures * VM needs time to do powering-off. * guest_monitoring_interval should be greater than * the time to SHUTDOWN VM gracefully. * e.g. | 565da9ba-3c0c-4087-83ca | iim1 | ACTIVE | powering-off | Running '''), cfg.IntOpt('guest_monitoring_timeout', default=2, help='Guest monitoring timeout (in seconds).'), cfg.IntOpt('guest_monitoring_failure_threshold', default=3, help='Failure threshold before sending notification.'), cfg.StrOpt('qemu_guest_agent_sock_path', default=SOCK, help=r''' * The file path of qemu guest agent sock. * Please use Python raw string notation as regular expressions. e.g. r'/var/lib/libvirt/qemu/org\.qemu\.guest_agent\..*\.instance-.*\.sock' '''), cfg.BoolOpt('callback_paused_event', default=True, help=''' * True: Callback for VM paused events. * False: Do not callback for VM paused events. '''), ] def register_opts(conf): conf.register_opts(monitor_opts, group='introspectiveinstancemonitor') def list_opts(): return { 'introspectiveinstancemonitor': monitor_opts } masakari-monitors-9.0.0/masakarimonitors/conf/base.py0000664000175000017500000000266313656750014023014 0ustar zuulzuul00000000000000# Copyright(c) 2016 Nippon Telegraph and Telephone Corporation # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. from oslo_config import cfg base_options = [ cfg.StrOpt( 'tempdir', help='Explicitly specify the temporary working directory.'), cfg.BoolOpt( 'monkey_patch', default=False, help=""" Determine if monkey patching should be applied. Related options: * ``monkey_patch_modules``: This must have values set for this option to have any effect """), cfg.ListOpt( 'monkey_patch_modules', default=[], help=""" List of modules/decorators to monkey patch. This option allows you to patch a decorator for all functions in specified modules. Related options: * ``monkey_patch``: This must be set to ``True`` for this option to have any effect """), ] def register_opts(conf): conf.register_opts(base_options) def list_opts(): return {'DEFAULT': base_options} masakari-monitors-9.0.0/setup.cfg0000664000175000017500000000426213656750123017037 0ustar zuulzuul00000000000000[metadata] name = masakari-monitors summary = Monitors for Masakari description-file = README.rst author = OpenStack author-email = openstack-dev@lists.openstack.org home-page = http://www.openstack.org/ classifier = Environment :: OpenStack Intended Audience :: Information Technology Intended Audience :: System Administrators License :: OSI Approved :: Apache Software License Operating System :: POSIX :: Linux Programming Language :: Python Programming Language :: Python :: 3 Programming Language :: Python :: 3.6 Programming Language :: Python :: 3.7 [files] packages = masakarimonitors [entry_points] oslo.config.opts = masakarimonitors.conf = masakarimonitors.conf.opts:list_opts oslo.config.opts.defaults = masakarimonitors.introspectiveinstancemonitor = masakarimonitors.common.config:set_middleware_defaults masakarimonitors.instancemonitor = masakarimonitors.common.config:set_middleware_defaults masakarimonitors.processmonitor = masakarimonitors.common.config:set_middleware_defaults masakarimonitors.hostmonitor = masakarimonitors.common.config:set_middleware_defaults console_scripts = masakari-introspectiveinstancemonitor = masakarimonitors.cmd.introspectiveinstancemonitor:main masakari-instancemonitor = masakarimonitors.cmd.instancemonitor:main masakari-processmonitor = masakarimonitors.cmd.processmonitor:main masakari-hostmonitor = masakarimonitors.cmd.hostmonitor:main hostmonitor.driver = simple = masakarimonitors.hostmonitor.host_handler.handle_host:HandleHost default = masakarimonitors.hostmonitor.host_handler.handle_host:HandleHost [build_sphinx] source-dir = doc/source build-dir = doc/build all_files = 1 [upload_sphinx] upload-dir = doc/build/html [compile_catalog] directory = masakarimonitors/locale domain = masakarimonitors [update_catalog] domain = masakarimonitors output_dir = masakarimonitors/locale input_file = masakarimonitors/locale/masakarimonitors.pot [extract_messages] keywords = _ gettext ngettext l_ lazy_gettext mapping_file = babel.cfg output_file = masakarimonitors/locale/masakarimonitors.pot [build_releasenotes] all_files = 1 build-dir = releasenotes/build source-dir = releasenotes/source [egg_info] tag_build = tag_date = 0 masakari-monitors-9.0.0/tox.ini0000664000175000017500000000640613656750014016532 0ustar zuulzuul00000000000000[tox] minversion = 3.1.1 envlist = pep8,py36,py37 skipsdist = True ignore_basepython_conflict = True [testenv] basepython = python3 usedevelop = True install_command = pip install -c{env:UPPER_CONSTRAINTS_FILE:https://releases.openstack.org/constraints/upper/master} {opts} {packages} setenv = VIRTUAL_ENV={envdir} OS_TEST_PATH=./masakarimonitors/tests/unit LANGUAGE=en_US LC_ALL=en_US.utf-8 deps = -r{toxinidir}/test-requirements.txt whitelist_externals = bash find rm env # By default stestr will set concurrency # to ncpu, to specify something else use # the concurrency= option. # call ie: 'tox -epy27 -- --concurrency=4' commands = find . -type f -name "*.pyc" -delete passenv = http_proxy HTTP_PROXY https_proxy HTTPS_PROXY no_proxy NO_PROXY OS_DEBUG GENERATE_HASHES [testenv:py36] basepython = python3.6 commands = {[testenv]commands} stestr run {posargs} [testenv:py37] basepython = python3.7 commands = {[testenv]commands} stestr run {posargs} [testenv:genconfig] commands = oslo-config-generator --config-file=etc/masakarimonitors/masakarimonitors-config-generator.conf [testenv:pep8] commands = flake8 {posargs} [testenv:venv] commands = {posargs} [testenv:cover] setenv = VIRTUAL_ENV={envdir} PYTHON=coverage run --source masakarimonitors --parallel-mode commands = stestr run {posargs} coverage combine coverage html -d cover coverage xml -o cover/coverage.xml [testenv:docs] deps = -r{toxinidir}/doc/requirements.txt commands = python setup.py build_sphinx [testenv:releasenotes] deps = -r{toxinidir}/doc/requirements.txt commands = rm -fr releasenotes/build sphinx-build -a -E -W -d releasenotes/build/doctrees -b html releasenotes/source releasenotes/build/html [testenv:debug] commands = oslo_debug_helper {posargs} [flake8] # E123 - closing bracket does not match indentation of opening bracket's line # E125 - continuation line with same indent as next logical line # E128 - continuation line under-indented for visual indent # E265 - block comment should start with '# ' # H405 - multi line docstring summary not separated with an empty line # W503 - line break before binary operator # W504 - line break after binary operator show-source = True ignore = E123,E125,E128,E265,H405,W503,W504 builtins = _ exclude=.venv,.git,.tox,dist,doc,*lib/python*,*egg,build [hacking] import_exceptions = masakarimonitors.i18n [flake8:local-plugins] extension = M301 = checks:check_explicit_underscore_import M302 = checks:no_translate_logs M303 = checks:yield_followed_by_space paths = ./masakarimonitors/hacking [testenv:lower-constraints] deps = -c{toxinidir}/lower-constraints.txt -r{toxinidir}/test-requirements.txt -r{toxinidir}/requirements.txt [testenv:bindep] # Do not install any requirements. We want this to be fast and work even if # system dependencies are missing, since it's used to tell you what system # dependencies are missing! This also means that bindep must be installed # separately, outside of the requirements files, and develop mode disabled # explicitly to avoid unnecessarily installing the checked-out repo too (this # further relies on "tox.skipsdist = True" above). deps = bindep commands = bindep test usedevelop = False masakari-monitors-9.0.0/ChangeLog0000664000175000017500000001051113656750122016761 0ustar zuulzuul00000000000000CHANGES ======= 9.0.0 ----- * Use unittest.mock instead of third party mock * Update and replace http with https for doc links * Check config file for hostname * Update hacking for Python3 * Use hostname to avoid clash with section * [ussuri][goal] Drop python 2.7 support and testing * Update constraints path to preferred static location * Update master for stable/train * Use crm\_mon for pacemaker-remote deployments 8.0.0 ----- * Add Python 3 Train unit tests * add libvirt-python for libvirt package * Remove deprecated shell scripts * Switch to using stestr * Use template for lower-constraints * OpenDev Migration Patch * Dropping the py35 testing * Switch from oslosphinx to openstackdocstheme * Update master for stable/stein 7.0.0 ----- * Run all jobs by default using python3 * Add line for PyCharm IDE in gitignore file * Removed unnecessary parantheses in yield statements 7.0.0.0b1 --------- * Increment versioning with pbr instruction * import zuul job settings from project-config * fix tox python3 overrides * Pass region\_name and interface parameters during connection initialization * Update reno for stable/rocky 6.0.0 ----- * Remove python-openstackclient from global-requirements 6.0.0.0b3 --------- * add lower-constraints job * Fix stable branch releasenotes * Introspective Instance Monitoring through QEMU Guest Agent 6.0.0.0b2 --------- * Remove dependency on python-masakariclient * Update for new openstacksdk changes to masakari-monitors 6.0.0.0b1 --------- * Updated from global requirements * Updated from global requirements * Update for upcoming openstacksdk changes to masakari-monitors 5.0.0 ----- * Use status\_code instead of http\_status * fix misspelling of 'configuration' * Support specify connection uri for libvirt 4.0.0 ----- * Updated from global requirements * Remove an extra word in process\_list.yaml.sample * Use os-testr and add PY35 support * Add testcases of hostmonitor * Remove log translations * Pass correct parameters to '\_get\_connection' method * Fix typo of hostmonitor * Change the required olso.privsep version * Fix typo of processmonitor * Add testcases of ha and processmonitor * Add testcases of instancemonitor * Change the condition of success or failure of notifying * Fix syntax errors of README.rst * Remove data\_files definition from setup.cfg 3.0.1 ----- * Add README.rst * Add warning messages about deprecation to process/host monitor * Skip notification retry processing when HTTP status is 409 * Fix global name '\_' is not defined * Return user-friendly error message when monkey\_patch enabled * Add hacking check to ensure \_ is imported * Add implement of calling pre and post script * Add implement of sending notification * Add implement of preventing split-brain * Add implement of sending a notification * Add implement of comparing host status * Add implement of monitoring host * Add implement of restarting processes * Add implement of monitoring processes * Add unit test codes of instancemonitor * Add implement of loading hostmonitor driver * Add python hostmonitor only main process * Add common notification sending functions * Add initial start of processes * Loading the process list written in YAML * Add python processmonitor only main process * Add missing packages in requirements.txt * Using ServiceLauncher instead of ProcessLauncher * Change the section name of settings that related keystone * Update to match latest global-requirements * Add stop method to terminate child process * Fix missing translations for log messages * Remove conversion of 'retry\_interval' parameter * Remove unnecessary return and pass statements * Fix unexpected bash error raised by hostmonitor * Add the start scripts of processmonitor and hostmonitor * Refactor: Move domainEventRegisterAny method in a loop * Fix the regular expression for the check state of RA * Change service type and service name * Allow masakari-instancemonitor command anywhere * To allow processmonitor recognize proc.list which exists anywhere * Fix processmonitor's invalid messages * masakari-instancemonitor: fix incorrect module specification 2.0.0 ----- * Remove unnecessary methods * Add hostmonitor * Add processmonitor * Add implementation of notifying * Add implementation of handling events * Add instancemonitor only main process * Add a mechanism to use the oslo libraries * Initial Cookiecutter Commit * Added .gitreview masakari-monitors-9.0.0/.mailmap0000664000175000017500000000013113656750014016625 0ustar zuulzuul00000000000000# Format is: # # masakari-monitors-9.0.0/masakari_monitors.egg-info/0000775000175000017500000000000013656750123022426 5ustar zuulzuul00000000000000masakari-monitors-9.0.0/masakari_monitors.egg-info/entry_points.txt0000664000175000017500000000170013656750122025721 0ustar zuulzuul00000000000000[console_scripts] masakari-hostmonitor = masakarimonitors.cmd.hostmonitor:main masakari-instancemonitor = masakarimonitors.cmd.instancemonitor:main masakari-introspectiveinstancemonitor = masakarimonitors.cmd.introspectiveinstancemonitor:main masakari-processmonitor = masakarimonitors.cmd.processmonitor:main [hostmonitor.driver] default = masakarimonitors.hostmonitor.host_handler.handle_host:HandleHost simple = masakarimonitors.hostmonitor.host_handler.handle_host:HandleHost [oslo.config.opts] masakarimonitors.conf = masakarimonitors.conf.opts:list_opts [oslo.config.opts.defaults] masakarimonitors.hostmonitor = masakarimonitors.common.config:set_middleware_defaults masakarimonitors.instancemonitor = masakarimonitors.common.config:set_middleware_defaults masakarimonitors.introspectiveinstancemonitor = masakarimonitors.common.config:set_middleware_defaults masakarimonitors.processmonitor = masakarimonitors.common.config:set_middleware_defaults masakari-monitors-9.0.0/masakari_monitors.egg-info/dependency_links.txt0000664000175000017500000000000113656750122026473 0ustar zuulzuul00000000000000 masakari-monitors-9.0.0/masakari_monitors.egg-info/requires.txt0000664000175000017500000000046713656750122025034 0ustar zuulzuul00000000000000automaton>=1.9.0 libvirt-python!=4.1.0,>=3.5.0 openstacksdk>=0.13.0 oslo.concurrency>=3.26.0 oslo.config>=5.2.0 lxml!=3.7.0,>=3.4.1 oslo.cache>=1.26.0 oslo.i18n>=3.15.3 oslo.log>=3.36.0 oslo.middleware>=3.31.0 oslo.privsep>=1.23.0 oslo.service!=1.28.1,>=1.24.0 oslo.utils>=3.33.0 pbr!=2.1.0,>=2.0.0 six>=1.10.0 masakari-monitors-9.0.0/masakari_monitors.egg-info/pbr.json0000664000175000017500000000005613656750122024104 0ustar zuulzuul00000000000000{"git_version": "92c934a", "is_release": true}masakari-monitors-9.0.0/masakari_monitors.egg-info/PKG-INFO0000664000175000017500000000567313656750122023535 0ustar zuulzuul00000000000000Metadata-Version: 1.1 Name: masakari-monitors Version: 9.0.0 Summary: Monitors for Masakari Home-page: http://www.openstack.org/ Author: OpenStack Author-email: openstack-dev@lists.openstack.org License: UNKNOWN Description: =============================== masakari-monitors =============================== Monitors for Masakari ===================== Monitors for Masakari provides Virtual Machine High Availability (VMHA) service for OpenStack clouds by automatically detecting the failure events such as VM process down, provisioning process down, and nova-compute host failure. If it detect the events, it sends notifications to the masakari-api. Original version of Masakari: https://github.com/ntt-sic/masakari Tokyo Summit Session: https://www.youtube.com/watch?v=BmjNKceW_9A Monitors for Masakari is distributed under the terms of the Apache License, Version 2.0. The full terms and conditions of this license are detailed in the LICENSE file. * Free software: Apache license * Documentation: https://docs.openstack.org/masakari-monitors * Source: https://git.openstack.org/cgit/openstack/masakari-monitors * Bugs: https://bugs.launchpad.net/masakari-monitors Configure masakari-monitors --------------------------- #. Clone masakari using:: $ git clone https://github.com/openstack/masakari-monitors.git #. Create masakarimonitors directory in /etc/. #. Run setup.py from masakari-monitors:: $ sudo python setup.py install #. Copy masakarimonitors.conf and process_list.yaml files from masakari-monitors/etc/ to /etc/masakarimonitors folder and make necessary changes to the masakarimonitors.conf and process_list.yaml files. To generate the sample masakarimonitors.conf file, run the following command from the top level of the masakari-monitors directory:: $ tox -egenconfig #. To run masakari-processmonitor, masakari-hostmonitor and masakari-instancemonitor simply use following binary:: $ masakari-processmonitor $ masakari-hostmonitor $ masakari-instancemonitor Features -------- * TODO Platform: UNKNOWN Classifier: Environment :: OpenStack Classifier: Intended Audience :: Information Technology Classifier: Intended Audience :: System Administrators Classifier: License :: OSI Approved :: Apache Software License Classifier: Operating System :: POSIX :: Linux Classifier: Programming Language :: Python Classifier: Programming Language :: Python :: 3 Classifier: Programming Language :: Python :: 3.6 Classifier: Programming Language :: Python :: 3.7 masakari-monitors-9.0.0/masakari_monitors.egg-info/top_level.txt0000664000175000017500000000002113656750122025150 0ustar zuulzuul00000000000000masakarimonitors masakari-monitors-9.0.0/masakari_monitors.egg-info/not-zip-safe0000664000175000017500000000000113656750122024653 0ustar zuulzuul00000000000000 masakari-monitors-9.0.0/masakari_monitors.egg-info/SOURCES.txt0000664000175000017500000001276513656750123024325 0ustar zuulzuul00000000000000.coveragerc .mailmap .stestr.conf .zuul.yaml AUTHORS CONTRIBUTING.rst ChangeLog HACKING.rst LICENSE MANIFEST.in README.rst babel.cfg bindep.txt lower-constraints.txt requirements.txt setup.cfg setup.py test-requirements.txt tox.ini doc/requirements.txt doc/source/conf.py doc/source/contributing.rst doc/source/index.rst doc/source/installation.rst doc/source/readme.rst doc/source/usage.rst etc/masakarimonitors/README-masakarimonitors.conf.txt etc/masakarimonitors/hostmonitor.conf.sample etc/masakarimonitors/masakarimonitors-config-generator.conf etc/masakarimonitors/proc.list.sample etc/masakarimonitors/process_list.yaml.sample etc/masakarimonitors/processmonitor.conf.sample masakari_monitors.egg-info/PKG-INFO masakari_monitors.egg-info/SOURCES.txt masakari_monitors.egg-info/dependency_links.txt masakari_monitors.egg-info/entry_points.txt masakari_monitors.egg-info/not-zip-safe masakari_monitors.egg-info/pbr.json masakari_monitors.egg-info/requires.txt masakari_monitors.egg-info/top_level.txt masakarimonitors/__init__.py masakarimonitors/config.py masakarimonitors/i18n.py masakarimonitors/manager.py masakarimonitors/privsep.py masakarimonitors/service.py masakarimonitors/utils.py masakarimonitors/version.py masakarimonitors/cmd/__init__.py masakarimonitors/cmd/hostmonitor.py masakarimonitors/cmd/instancemonitor.py masakarimonitors/cmd/introspectiveinstancemonitor.py masakarimonitors/cmd/processmonitor.py masakarimonitors/common/__init__.py masakarimonitors/common/config.py masakarimonitors/conf/__init__.py masakarimonitors/conf/api.py masakarimonitors/conf/base.py masakarimonitors/conf/host.py masakarimonitors/conf/instance.py masakarimonitors/conf/introspectiveinstancemonitor.py masakarimonitors/conf/opts.py masakarimonitors/conf/process.py masakarimonitors/conf/service.py masakarimonitors/ha/__init__.py masakarimonitors/ha/masakari.py masakarimonitors/hacking/__init__.py masakarimonitors/hacking/checks.py masakarimonitors/hostmonitor/__init__.py masakarimonitors/hostmonitor/host.py masakarimonitors/hostmonitor/host_handler/__init__.py masakarimonitors/hostmonitor/host_handler/driver.py masakarimonitors/hostmonitor/host_handler/handle_host.py masakarimonitors/hostmonitor/host_handler/hold_host_status.py masakarimonitors/hostmonitor/host_handler/parse_cib_xml.py masakarimonitors/hostmonitor/host_handler/parse_crmmon_xml.py masakarimonitors/instancemonitor/__init__.py masakarimonitors/instancemonitor/instance.py masakarimonitors/instancemonitor/libvirt_handler/__init__.py masakarimonitors/instancemonitor/libvirt_handler/callback.py masakarimonitors/instancemonitor/libvirt_handler/eventfilter.py masakarimonitors/instancemonitor/libvirt_handler/eventfilter_table.py masakarimonitors/introspectiveinstancemonitor/README.rst masakarimonitors/introspectiveinstancemonitor/__init__.py masakarimonitors/introspectiveinstancemonitor/cache.py masakarimonitors/introspectiveinstancemonitor/instance.py masakarimonitors/introspectiveinstancemonitor/qemu_utils.py masakarimonitors/introspectiveinstancemonitor/scheduler.py masakarimonitors/objects/__init__.py masakarimonitors/objects/event_constants.py masakarimonitors/processmonitor/__init__.py masakarimonitors/processmonitor/process.py masakarimonitors/processmonitor/typescript masakarimonitors/processmonitor/process_handler/__init__.py masakarimonitors/processmonitor/process_handler/handle_process.py masakarimonitors/tests/__init__.py masakarimonitors/tests/base.py masakarimonitors/tests/test_masakarimonitors.py masakarimonitors/tests/unit/__init__.py masakarimonitors/tests/unit/test_hacking.py masakarimonitors/tests/unit/ha/__init__.py masakarimonitors/tests/unit/ha/test_masakari.py masakarimonitors/tests/unit/hostmonitor/__init__.py masakarimonitors/tests/unit/hostmonitor/test_host.py masakarimonitors/tests/unit/hostmonitor/host_handler/__init__.py masakarimonitors/tests/unit/hostmonitor/host_handler/test_handle_host.py masakarimonitors/tests/unit/hostmonitor/host_handler/test_hold_host_status.py masakarimonitors/tests/unit/hostmonitor/host_handler/test_parse_cib_xml.py masakarimonitors/tests/unit/hostmonitor/host_handler/test_parse_crmmon_xml.py masakarimonitors/tests/unit/instancemonitor/__init__.py masakarimonitors/tests/unit/instancemonitor/test_instance.py masakarimonitors/tests/unit/instancemonitor/libvirt_handler/__init__.py masakarimonitors/tests/unit/instancemonitor/libvirt_handler/test_callback.py masakarimonitors/tests/unit/instancemonitor/libvirt_handler/test_eventfilter.py masakarimonitors/tests/unit/introspectiveinstancemonitor/__init__.py masakarimonitors/tests/unit/introspectiveinstancemonitor/test_monitor_manager.py masakarimonitors/tests/unit/introspectiveinstancemonitor/test_qemu_utils.py masakarimonitors/tests/unit/processmonitor/__init__.py masakarimonitors/tests/unit/processmonitor/test_process.py masakarimonitors/tests/unit/processmonitor/process_handler/__init__.py masakarimonitors/tests/unit/processmonitor/process_handler/test_handle_process.py releasenotes/notes/.placeholder releasenotes/notes/bug-1866660-ef8624f5283b2e5e.yaml releasenotes/notes/drop-py-2-7-b28de816eac45468.yaml releasenotes/notes/introspectiveinstancemonitor-f4bc71f029b61d49.yaml releasenotes/notes/pythonize-monitors-081e74dfaf78fe99.yaml releasenotes/source/conf.py releasenotes/source/index.rst releasenotes/source/ocata.rst releasenotes/source/pike.rst releasenotes/source/queens.rst releasenotes/source/rocky.rst releasenotes/source/stein.rst releasenotes/source/train.rst releasenotes/source/unreleased.rst releasenotes/source/_static/.placeholder releasenotes/source/_templates/.placeholdermasakari-monitors-9.0.0/MANIFEST.in0000664000175000017500000000013613656750014016747 0ustar zuulzuul00000000000000include AUTHORS include ChangeLog exclude .gitignore exclude .gitreview global-exclude *.pyc masakari-monitors-9.0.0/etc/0000775000175000017500000000000013656750123015765 5ustar zuulzuul00000000000000masakari-monitors-9.0.0/etc/masakarimonitors/0000775000175000017500000000000013656750123021350 5ustar zuulzuul00000000000000masakari-monitors-9.0.0/etc/masakarimonitors/proc.list.sample0000775000175000017500000000035713656750014024477 0ustar zuulzuul0000000000000001,/usr/sbin/libvirtd,sudo service libvirt-bin start,sudo service libvirt-bin start,,,, 02,/usr/bin/python /usr/local/bin/masakari-instancemonitor,sudo service masakari-instancemonitor start,sudo service masakari-instancemonitor start,,,, masakari-monitors-9.0.0/etc/masakarimonitors/README-masakarimonitors.conf.txt0000664000175000017500000000022013656750014027344 0ustar zuulzuul00000000000000To generate the sample masakarimonitors.conf file, run the following command from the top level of the masakari directory: tox -egenconfig masakari-monitors-9.0.0/etc/masakarimonitors/masakarimonitors-config-generator.conf0000664000175000017500000000025513656750014031032 0ustar zuulzuul00000000000000[DEFAULT] output_file = etc/masakarimonitors/masakarimonitors.conf.sample wrap_width = 80 namespace = masakarimonitors.conf namespace = oslo.log namespace = oslo.middleware masakari-monitors-9.0.0/etc/masakarimonitors/process_list.yaml.sample0000664000175000017500000000421413656750014026225 0ustar zuulzuul00000000000000# Define the monitoring processes as follows: # process_name: [Name of the process as it in 'ps -ef'.] # start_command: [Start command of the process.] # pre_start_command: [Command which is executed before start_command.] # post_start_command: [Command which is executed after start_command.] # restart_command: [Restart command of the process.] # pre_restart_command: [Command which is executed before restart_command.] # post_restart_command: [Command which is executed after restart_command.] # run_as_root: [Bool value whether to execute commands as root authority.] # # These definitions need to be set according to the environment to use. # Sample of definitions is shown below. - # libvirt-bin process_name: /usr/sbin/libvirtd start_command: systemctl start libvirt-bin pre_start_command: post_start_command: restart_command: systemctl restart libvirt-bin pre_restart_command: post_restart_command: run_as_root: True - # nova-compute process_name: /usr/local/bin/nova-compute start_command: systemctl start nova-compute pre_start_command: post_start_command: restart_command: systemctl restart nova-compute pre_restart_command: post_restart_command: run_as_root: True - # instancemonitor process_name: /usr/bin/python /usr/local/bin/masakari-instancemonitor start_command: systemctl start masakari-instancemonitor pre_start_command: post_start_command: restart_command: systemctl restart masakari-instancemonitor pre_restart_command: post_restart_command: run_as_root: True - # hostmonitor process_name: /usr/bin/python /usr/local/bin/masakari-hostmonitor start_command: systemctl start masakari-hostmonitor pre_start_command: post_start_command: restart_command: systemctl restart masakari-hostmonitor pre_restart_command: post_restart_command: run_as_root: True - # sshd process_name: /usr/sbin/sshd start_command: systemctl start ssh pre_start_command: post_start_command: restart_command: systemctl restart ssh pre_restart_command: post_restart_command: run_as_root: True masakari-monitors-9.0.0/etc/masakarimonitors/hostmonitor.conf.sample0000775000175000017500000000331713656750014026072 0ustar zuulzuul00000000000000# Monitoring interval(in seconds) of node status # Default : 60 MONITOR_INTERVAL=60 # Timeout value(in seconds) when cannot send the notice to resource management. # Default : 10 NOTICE_TIMEOUT=10 # Retry numbers when failed to notify the resource management. # Default : 12 NOTICE_RETRY_COUNT=12 # Retry interval(in seconds) when cannot send the notice to resource management. # Default : 10 NOTICE_RETRY_INTERVAL=10 # Standby time(in seconds) until activate STONITH # Default : 30 STONITH_WAIT=30 # Stonith type ( ipmi(default) / ssh(for test) ) STONITH_TYPE=ssh # Maximum number of child process to start # Default : 3 MAX_CHILD_PROCESS=3 # Timeout value(in seconds) of the tcpdump command when monitor the HB line # Default : 10 TCPDUMP_TIMEOUT=10 # Timeout value(in seconds) of the ipmitool command # Default : 5 IPMI_TIMEOUT=5 # Number of ipmitool command retries # Default : 3 IPMI_RETRY_MAX=3 # Retry interval(in seconds) of the ipmitool command # Default : 10 IPMI_RETRY_INTERVAL=10 # Configuration file path of corosync # Default : /etc/corosync/corosync.conf HA_CONF="/etc/corosync/corosync.conf" # Log level ( info / debug ) # Otherwise, info is set # Default : info LOG_LEVEL="debug" # These value of the order to get the token from the key stone # DOMAIN,ADMIN_USER,ADMIN_PASS,AUTH_URL # Domain name which the project belongs DOMAIN="Default" # The name of a user with administrative privileges ADMIN_USER="admin" # Administrator user's password ADMIN_PASS="admin" # Administrator user's project name PROJECT="demo" # Name of Region REGION="RegionOne" # Address of Keystone AUTH_URL="http://127.0.0.1:5000/" # Pattern of resource group name to ignore from RA count IGNORE_RESOURCE_GROUP_NAME_PATTERN="stonith" masakari-monitors-9.0.0/etc/masakarimonitors/processmonitor.conf.sample0000775000175000017500000000044213656750014026567 0ustar zuulzuul00000000000000PROCESS_CHECK_INTERVAL=5 PROCESS_REBOOT_RETRY=3 REBOOT_INTERVAL=5 MASAKARI_API_SEND_TIMEOUT=10 MASAKARI_API_SEND_RETRY=12 MASAKARI_API_SEND_DELAY=10 LOG_LEVEL="debug" DOMAIN="Default" PROJECT="demo" ADMIN_USER="admin" ADMIN_PASS="admin" AUTH_URL="http://127.0.0.1:5000/" REGION="RegionOne" masakari-monitors-9.0.0/HACKING.rst0000664000175000017500000000071713656750014017014 0ustar zuulzuul00000000000000masakari-monitors Style Commandments =============================================== Read the OpenStack Style Commandments http://docs.openstack.org/developer/hacking/ masakari-monitors Specific Commandments ------------------------------ - [M301] Ensure that the _() function is explicitly imported to ensure proper translations. - [M302] Validate that log messages are not translated. - [M303] Yield must always be followed by a space when yielding a value. masakari-monitors-9.0.0/releasenotes/0000775000175000017500000000000013656750123017703 5ustar zuulzuul00000000000000masakari-monitors-9.0.0/releasenotes/notes/0000775000175000017500000000000013656750123021033 5ustar zuulzuul00000000000000masakari-monitors-9.0.0/releasenotes/notes/.placeholder0000664000175000017500000000000013656750014023303 0ustar zuulzuul00000000000000masakari-monitors-9.0.0/releasenotes/notes/bug-1866660-ef8624f5283b2e5e.yaml0000664000175000017500000000065513656750014025566 0ustar zuulzuul00000000000000--- features: - | A 'hostname' option has been added to the Masakari monitors configuration file which replaces the 'host' option. The option should be used to specify the Hostname, FQDN or IP address of the host. deprecations: - | The 'host' option in the 'DEFAULT' section has been deprecated as it clashes with the name of a section in the same file. It has been replaced by the 'hostname' option. masakari-monitors-9.0.0/releasenotes/notes/introspectiveinstancemonitor-f4bc71f029b61d49.yaml0000664000175000017500000000026113656750014032167 0ustar zuulzuul00000000000000--- features: - | Added Introspective Instance Monitoring through QEMU Guest Agent, in order to detect, report and optionally recover VMs from internal VM faults. masakari-monitors-9.0.0/releasenotes/notes/drop-py-2-7-b28de816eac45468.yaml0000664000175000017500000000034013656750014026035 0ustar zuulzuul00000000000000--- upgrade: - | Python 2.7 support has been dropped. Last release of masakari-monitors to support python 2.7 is OpenStack Train. The minimum version of Python now supported by masakari-monitors is Python 3.6. masakari-monitors-9.0.0/releasenotes/notes/pythonize-monitors-081e74dfaf78fe99.yaml0000664000175000017500000000050013656750014030135 0ustar zuulzuul00000000000000--- features: - Added the masakari-hostmonitor and masakari-processmonitor which were implemented in python. deprecations: - The masakari-hostmonitor and masakari-processmonitor which were implemented in bash shell script are deprecated as of the Ocata release and will be removed in the Queens release. masakari-monitors-9.0.0/releasenotes/source/0000775000175000017500000000000013656750123021203 5ustar zuulzuul00000000000000masakari-monitors-9.0.0/releasenotes/source/train.rst0000664000175000017500000000017613656750014023055 0ustar zuulzuul00000000000000========================== Train Series Release Notes ========================== .. release-notes:: :branch: stable/train masakari-monitors-9.0.0/releasenotes/source/_static/0000775000175000017500000000000013656750123022631 5ustar zuulzuul00000000000000masakari-monitors-9.0.0/releasenotes/source/_static/.placeholder0000664000175000017500000000000013656750014025101 0ustar zuulzuul00000000000000masakari-monitors-9.0.0/releasenotes/source/conf.py0000664000175000017500000002203513656750014022503 0ustar zuulzuul00000000000000# -*- coding: utf-8 -*- # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or # implied. # See the License for the specific language governing permissions and # limitations under the License. # Masakarimonitors Release Notes documentation build configuration file, # created by sphinx-quickstart on Tue Oct 25 17:40:50 2016. # # This file is execfile()d with the current directory set to its # containing dir. # # Note that not all possible configuration values are present in this # autogenerated file. # # All configuration values have a default; values that are commented out # serve to show the default. # If extensions (or modules to document with autodoc) are in another directory, # add these directories to sys.path here. If the directory is relative to the # documentation root, use os.path.abspath to make it absolute, like shown here. # sys.path.insert(0, os.path.abspath('.')) # -- General configuration ------------------------------------------------ # If your documentation needs a minimal Sphinx version, state it here. # needs_sphinx = '1.0' # Add any Sphinx extension module names here, as strings. They can be # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom # ones. extensions = [ 'openstackdocstheme', 'reno.sphinxext', ] # Add any paths that contain templates here, relative to this directory. templates_path = ['_templates'] # The suffix of source filenames. source_suffix = '.rst' # The encoding of source files. # source_encoding = 'utf-8-sig' # The master toctree document. master_doc = 'index' # General information about the project. project = u'masakarimonitors Release Notes' copyright = u'2016, OpenStack Foundation' # The version info for the project you're documenting, acts as replacement for # |version| and |release|, also used in various other places throughout the # built documents. # # The short X.Y version. # The full version, including alpha/beta/rc tags. release = '' # The short X.Y version. version = '' # The language for content autogenerated by Sphinx. Refer to documentation # for a list of supported languages. # language = None # There are two options for replacing |today|: either, you set today to some # non-false value, then it is used: # today = '' # Else, today_fmt is used as the format for a strftime call. # today_fmt = '%B %d, %Y' # List of patterns, relative to source directory, that match files and # directories to ignore when looking for source files. exclude_patterns = [] # The reST default role (used for this markup: `text`) to use for all # documents. # default_role = None # If true, '()' will be appended to :func: etc. cross-reference text. # add_function_parentheses = True # If true, the current module name will be prepended to all description # unit titles (such as .. function::). # add_module_names = True # If true, sectionauthor and moduleauthor directives will be shown in the # output. They are ignored by default. # show_authors = False # The name of the Pygments (syntax highlighting) style to use. pygments_style = 'sphinx' # A list of ignored prefixes for module index sorting. # modindex_common_prefix = [] # If true, keep warnings as "system message" paragraphs in the built documents. # keep_warnings = False # -- Options for HTML output ---------------------------------------------- # The theme to use for HTML and HTML Help pages. See the documentation for # a list of builtin themes. html_theme = 'openstackdocs' repository_name = 'openstack/masakari-monitors' # Theme options are theme-specific and customize the look and feel of a theme # further. For a list of options available for each theme, see the # documentation. # html_theme_options = {} # Add any paths that contain custom themes here, relative to this directory. # html_theme_path = [] # The name for this set of Sphinx documents. If None, it defaults to # " v documentation". # html_title = None # A shorter title for the navigation bar. Default is the same as html_title. # html_short_title = None # The name of an image file (relative to this directory) to place at the top # of the sidebar. # html_logo = None # The name of an image file (within the static path) to use as favicon of the # docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32 # pixels large. # html_favicon = None # Add any paths that contain custom static files (such as style sheets) here, # relative to this directory. They are copied after the builtin static files, # so a file named "default.css" will overwrite the builtin "default.css". html_static_path = ['_static'] # Add any extra paths that contain custom files (such as robots.txt or # .htaccess) here, relative to this directory. These files are copied # directly to the root of the documentation. # html_extra_path = [] # If not '', a 'Last updated on:' timestamp is inserted at every page bottom, # using the given strftime format. # html_last_updated_fmt = '%b %d, %Y' # If true, SmartyPants will be used to convert quotes and dashes to # typographically correct entities. # html_use_smartypants = True # Custom sidebar templates, maps document names to template names. # html_sidebars = {} # Additional templates that should be rendered to pages, maps page names to # template names. # html_additional_pages = {} # If false, no module index is generated. # html_domain_indices = True # If false, no index is generated. # html_use_index = True # If true, the index is split into individual pages for each letter. # html_split_index = False # If true, links to the reST sources are added to the pages. # html_show_sourcelink = True # If true, "Created using Sphinx" is shown in the HTML footer. Default is True. # html_show_sphinx = True # If true, "(C) Copyright ..." is shown in the HTML footer. Default is True. # html_show_copyright = True # If true, an OpenSearch description file will be output, and all pages will # contain a tag referring to it. The value of this option must be the # base URL from which the finished HTML is served. # html_use_opensearch = '' # This is the file name suffix for HTML files (e.g. ".xhtml"). # html_file_suffix = None # Output file base name for HTML help builder. htmlhelp_basename = 'MasakarimonitorsReleaseNotesdoc' # -- Options for LaTeX output --------------------------------------------- latex_elements = { # The paper size ('letterpaper' or 'a4paper'). # 'papersize': 'letterpaper', # The font size ('10pt', '11pt' or '12pt'). # 'pointsize': '10pt', # Additional stuff for the LaTeX preamble. # 'preamble': '', } # Grouping the document tree into LaTeX files. List of tuples # (source start file, target name, title, # author, documentclass [howto, manual, or own class]). latex_documents = [ ('index', 'MasakarimonitorsReleaseNotes.tex', u'Masakarimonitors Release' u'Notes Documentation', u'Masakarimonitors Developers', 'manual'), ] # The name of an image file (relative to this directory) to place at the top of # the title page. # latex_logo = None # For "manual" documents, if this is true, then toplevel headings are parts, # not chapters. # latex_use_parts = False # If true, show page references after internal links. # latex_show_pagerefs = False # If true, show URL addresses after external links. # latex_show_urls = False # Documents to append as an appendix to all manuals. # latex_appendices = [] # If false, no module index is generated. # latex_domain_indices = True # -- Options for manual page output --------------------------------------- # One entry per manual page. List of tuples # (source start file, name, description, authors, manual section). man_pages = [ ('index', 'Masakarimonitorsreleasenotes', u'Masakarimonitors Release Notes' u'Documentation', [u'Masakarimonitors Developers'], 1) ] # If true, show URL addresses after external links. # man_show_urls = False # -- Options for Texinfo output ------------------------------------------- # Grouping the document tree into Texinfo files. List of tuples # (source start file, target name, title, author, # dir menu entry, description, category) texinfo_documents = [ ('index', 'MasakarimonitorsReleaseNotes', u'Masakarimonitors Release Notes' u'Documentation', u'Masakarimonitors Developers', 'MasakarimonitorsReleaseNotes', 'One line description of project.', 'Miscellaneous'), ] # Documents to append as an appendix to all manuals. # texinfo_appendices = [] # If false, no module index is generated. # texinfo_domain_indices = True # How to display URL addresses: 'footnote', 'no', or 'inline'. # texinfo_show_urls = 'footnote' # If true, do not generate a @detailmenu in the "Top" node's menu. # texinfo_no_detailmenu = False # -- Options for Internationalization output ------------------------------ locale_dirs = ['locale/'] masakari-monitors-9.0.0/releasenotes/source/stein.rst0000664000175000017500000000022113656750014023051 0ustar zuulzuul00000000000000=================================== Stein Series Release Notes =================================== .. release-notes:: :branch: stable/stein masakari-monitors-9.0.0/releasenotes/source/queens.rst0000664000175000017500000000021513656750014023232 0ustar zuulzuul00000000000000============================ Queens Series Release Notes ============================ .. release-notes:: :branch: origin/stable/queens masakari-monitors-9.0.0/releasenotes/source/unreleased.rst0000664000175000017500000000016013656750014024060 0ustar zuulzuul00000000000000============================== Current Series Release Notes ============================== .. release-notes:: masakari-monitors-9.0.0/releasenotes/source/rocky.rst0000664000175000017500000000022113656750014023056 0ustar zuulzuul00000000000000=================================== Rocky Series Release Notes =================================== .. release-notes:: :branch: stable/rocky masakari-monitors-9.0.0/releasenotes/source/index.rst0000664000175000017500000000035013656750014023041 0ustar zuulzuul00000000000000Welcome to Masakarimonitor Release Notes documentation! ======================================================== Contents ======== .. toctree:: :maxdepth: 1 unreleased train stein rocky queens pike ocata masakari-monitors-9.0.0/releasenotes/source/ocata.rst0000664000175000017500000000021113656750014023015 0ustar zuulzuul00000000000000=========================== Ocata Series Release Notes =========================== .. release-notes:: :branch: origin/stable/ocata masakari-monitors-9.0.0/releasenotes/source/_templates/0000775000175000017500000000000013656750123023340 5ustar zuulzuul00000000000000masakari-monitors-9.0.0/releasenotes/source/_templates/.placeholder0000664000175000017500000000000013656750014025610 0ustar zuulzuul00000000000000masakari-monitors-9.0.0/releasenotes/source/pike.rst0000664000175000017500000000020513656750014022661 0ustar zuulzuul00000000000000========================== Pike Series Release Notes ========================== .. release-notes:: :branch: origin/stable/pike masakari-monitors-9.0.0/PKG-INFO0000664000175000017500000000567313656750123016322 0ustar zuulzuul00000000000000Metadata-Version: 1.1 Name: masakari-monitors Version: 9.0.0 Summary: Monitors for Masakari Home-page: http://www.openstack.org/ Author: OpenStack Author-email: openstack-dev@lists.openstack.org License: UNKNOWN Description: =============================== masakari-monitors =============================== Monitors for Masakari ===================== Monitors for Masakari provides Virtual Machine High Availability (VMHA) service for OpenStack clouds by automatically detecting the failure events such as VM process down, provisioning process down, and nova-compute host failure. If it detect the events, it sends notifications to the masakari-api. Original version of Masakari: https://github.com/ntt-sic/masakari Tokyo Summit Session: https://www.youtube.com/watch?v=BmjNKceW_9A Monitors for Masakari is distributed under the terms of the Apache License, Version 2.0. The full terms and conditions of this license are detailed in the LICENSE file. * Free software: Apache license * Documentation: https://docs.openstack.org/masakari-monitors * Source: https://git.openstack.org/cgit/openstack/masakari-monitors * Bugs: https://bugs.launchpad.net/masakari-monitors Configure masakari-monitors --------------------------- #. Clone masakari using:: $ git clone https://github.com/openstack/masakari-monitors.git #. Create masakarimonitors directory in /etc/. #. Run setup.py from masakari-monitors:: $ sudo python setup.py install #. Copy masakarimonitors.conf and process_list.yaml files from masakari-monitors/etc/ to /etc/masakarimonitors folder and make necessary changes to the masakarimonitors.conf and process_list.yaml files. To generate the sample masakarimonitors.conf file, run the following command from the top level of the masakari-monitors directory:: $ tox -egenconfig #. To run masakari-processmonitor, masakari-hostmonitor and masakari-instancemonitor simply use following binary:: $ masakari-processmonitor $ masakari-hostmonitor $ masakari-instancemonitor Features -------- * TODO Platform: UNKNOWN Classifier: Environment :: OpenStack Classifier: Intended Audience :: Information Technology Classifier: Intended Audience :: System Administrators Classifier: License :: OSI Approved :: Apache Software License Classifier: Operating System :: POSIX :: Linux Classifier: Programming Language :: Python Classifier: Programming Language :: Python :: 3 Classifier: Programming Language :: Python :: 3.6 Classifier: Programming Language :: Python :: 3.7 masakari-monitors-9.0.0/bindep.txt0000664000175000017500000000045313656750014017215 0ustar zuulzuul00000000000000# This is a cross-platform list tracking distribution packages needed for install and tests; # see https://docs.openstack.org/infra/bindep/ for additional information. # libvirt-dev is needed by libvirt library. libvirt-dev [platform:dpkg] python-libvirt [platform:dpkg] pkg-config [platform:dpkg] masakari-monitors-9.0.0/test-requirements.txt0000664000175000017500000000110613656750014021450 0ustar zuulzuul00000000000000# The order of packages is significant, because pip processes them in the order # of appearance. Changing the order has an impact on the overall integration # process, which may cause wedges in the gate later. hacking>=3.0,<3.1.0 # Apache-2.0 coverage!=4.4,>=4.0 # Apache-2.0 python-subunit>=1.0.0 # Apache-2.0/BSD sphinx!=1.6.6,!=1.6.7,>=1.6.2 # BSD oslosphinx>=4.7.0 # Apache-2.0 oslotest>=3.2.0 # Apache-2.0 os-testr>=1.0.0 # Apache-2.0 testrepository>=0.0.18 # Apache-2.0/BSD testscenarios>=0.4 # Apache-2.0/BSD testtools>=2.2.0 # MIT # releasenotes reno>=2.5.0 # Apache-2.0 masakari-monitors-9.0.0/lower-constraints.txt0000664000175000017500000000362113656750014021451 0ustar zuulzuul00000000000000automaton==1.9.0 alabaster==0.7.10 appdirs==1.4.3 asn1crypto==0.24.0 Babel==2.5.3 certifi==2018.1.18 cffi==1.11.5 chardet==3.0.4 cliff==2.11.0 cmd2==0.8.1 coverage==4.0 cryptography==2.1.4 debtcollector==1.19.0 decorator==4.2.1 deprecation==2.0 docutils==0.14 dogpile.cache==0.6.5 dulwich==0.19.0 enum-compat==0.0.2 eventlet==0.20.0 extras==1.0.0 fasteners==0.14.1 fixtures==3.0.0 future==0.16.0 greenlet==0.4.13 idna==2.6 imagesize==1.0.0 iso8601==0.1.12 Jinja2==2.10 jmespath==0.9.3 jsonpatch==1.21 jsonpointer==2.0 jsonschema==2.6.0 keystoneauth1==3.4.0 libvirt-python==3.5.0 linecache2==1.0.0 MarkupSafe==1.0 mccabe==0.2.1 mock==2.0.0 monotonic==1.4 mox3==0.25.0 msgpack==0.5.6 munch==2.2.0 netaddr==0.7.19 netifaces==0.10.6 lxml==3.4.1 openstacksdk==0.13.0 os-client-config==1.29.0 os-service-types==1.2.0 os-testr==1.0.0 osc-lib==1.10.0 oslo.cache==1.26.0 oslo.concurrency==3.26.0 oslo.config==5.2.0 oslo.context==2.20.0 oslo.i18n==3.15.3 oslo.log==3.36.0 oslo.middleware==3.31.0 oslo.privsep==1.23.0 oslo.serialization==2.25.0 oslo.service==1.24.0 oslo.utils==3.33.0 oslosphinx==4.7.0 oslotest==3.2.0 packaging==17.1 Paste==2.0.3 PasteDeploy==1.5.2 pbr==2.0.0 prettytable==0.7.2 pycparser==2.18 Pygments==2.2.0 pyinotify==0.9.6 pyOpenSSL==17.5.0 pyparsing==2.2.0 pyperclip==1.6.0 python-cinderclient==3.5.0 python-dateutil==2.7.0 python-glanceclient==2.9.1 python-keystoneclient==3.15.0 python-masakariclient==5.0.0 python-mimeparse==1.6.0 python-novaclient==10.1.0 python-subunit==1.0.0 pytz==2018.3 PyYAML==3.12 reno==2.5.0 repoze.lru==0.7 requests==2.18.4 requestsexceptions==1.4.0 rfc3986==1.1.0 Routes==2.4.1 simplejson==3.13.2 six==1.10.0 snowballstemmer==1.2.1 Sphinx==1.6.2 sphinxcontrib-websupport==1.0.1 statsd==3.2.2 stestr==2.0.0 stevedore==1.28.0 testrepository==0.0.18 testscenarios==0.4 testtools==2.2.0 traceback2==1.4.0 unittest2==1.1.0 urllib3==1.22 voluptuous==0.11.1 warlock==1.3.0 WebOb==1.7.4 wrapt==1.10.11 masakari-monitors-9.0.0/AUTHORS0000664000175000017500000000225313656750122016263 0ustar zuulzuul00000000000000Andreas Jaeger Corey Bryant Dinesh Bhor Doug Hellmann Ghanshyam Mann Keiji Niwa Kengo Takahara Kengo Takahara Liam Young Louie KWAN Nguyen Hai OpenStack Release Bot Sampath Priyankara Sean McGinnis Shilpa Devharakar Takahiro Izumi Takashi NATSUME avnish dineshbhor huang.zhiping jayashri bidwe liyingjun nitesh.vanarase openstack poojajadhav shilpa.devharakar tpatil wangqiangbj yangkun.lc masakari-monitors-9.0.0/babel.cfg0000664000175000017500000000002113656750014016730 0ustar zuulzuul00000000000000[python: **.py] masakari-monitors-9.0.0/CONTRIBUTING.rst0000664000175000017500000000122513656750014017652 0ustar zuulzuul00000000000000If you would like to contribute to the development of OpenStack, you must follow the steps in this page: http://docs.openstack.org/infra/manual/developers.html If you already have a good understanding of how the system works and your OpenStack accounts are set up, you can skip to the development workflow section of this documentation to learn how changes to OpenStack should be submitted for review via the Gerrit tool: http://docs.openstack.org/infra/manual/developers.html#development-workflow Pull requests submitted through GitHub will be ignored. Bugs should be filed on Launchpad, not GitHub: https://bugs.launchpad.net/masakari-monitors masakari-monitors-9.0.0/README.rst0000664000175000017500000000344513656750014016706 0ustar zuulzuul00000000000000=============================== masakari-monitors =============================== Monitors for Masakari ===================== Monitors for Masakari provides Virtual Machine High Availability (VMHA) service for OpenStack clouds by automatically detecting the failure events such as VM process down, provisioning process down, and nova-compute host failure. If it detect the events, it sends notifications to the masakari-api. Original version of Masakari: https://github.com/ntt-sic/masakari Tokyo Summit Session: https://www.youtube.com/watch?v=BmjNKceW_9A Monitors for Masakari is distributed under the terms of the Apache License, Version 2.0. The full terms and conditions of this license are detailed in the LICENSE file. * Free software: Apache license * Documentation: https://docs.openstack.org/masakari-monitors * Source: https://git.openstack.org/cgit/openstack/masakari-monitors * Bugs: https://bugs.launchpad.net/masakari-monitors Configure masakari-monitors --------------------------- #. Clone masakari using:: $ git clone https://github.com/openstack/masakari-monitors.git #. Create masakarimonitors directory in /etc/. #. Run setup.py from masakari-monitors:: $ sudo python setup.py install #. Copy masakarimonitors.conf and process_list.yaml files from masakari-monitors/etc/ to /etc/masakarimonitors folder and make necessary changes to the masakarimonitors.conf and process_list.yaml files. To generate the sample masakarimonitors.conf file, run the following command from the top level of the masakari-monitors directory:: $ tox -egenconfig #. To run masakari-processmonitor, masakari-hostmonitor and masakari-instancemonitor simply use following binary:: $ masakari-processmonitor $ masakari-hostmonitor $ masakari-instancemonitor Features -------- * TODO masakari-monitors-9.0.0/.zuul.yaml0000664000175000017500000000031213656750014017146 0ustar zuulzuul00000000000000- project: templates: - check-requirements - openstack-lower-constraints-jobs - openstack-python3-ussuri-jobs - publish-openstack-docs-pti - release-notes-jobs-python3 masakari-monitors-9.0.0/setup.py0000664000175000017500000000200613656750014016721 0ustar zuulzuul00000000000000# Copyright (c) 2013 Hewlett-Packard Development Company, L.P. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or # implied. # See the License for the specific language governing permissions and # limitations under the License. # THIS FILE IS MANAGED BY THE GLOBAL REQUIREMENTS REPO - DO NOT EDIT import setuptools # In python < 2.7.4, a lazy loading of package `pbr` will break # setuptools if some other modules registered functions in `atexit`. # solution from: http://bugs.python.org/issue15881#msg170215 try: import multiprocessing # noqa except ImportError: pass setuptools.setup( setup_requires=['pbr>=2.0.0'], pbr=True) masakari-monitors-9.0.0/.stestr.conf0000664000175000017500000000007613656750014017465 0ustar zuulzuul00000000000000[DEFAULT] test_path=./masakarimonitors/tests/unit top_dir=./ masakari-monitors-9.0.0/LICENSE0000664000175000017500000002363713656750014016231 0ustar zuulzuul00000000000000 Apache License Version 2.0, January 2004 http://www.apache.org/licenses/ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 1. Definitions. "License" shall mean the terms and conditions for use, reproduction, and distribution as defined by Sections 1 through 9 of this document. "Licensor" shall mean the copyright owner or entity authorized by the copyright owner that is granting the License. "Legal Entity" shall mean the union of the acting entity and all other entities that control, are controlled by, or are under common control with that entity. For the purposes of this definition, "control" means (i) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity. "You" (or "Your") shall mean an individual or Legal Entity exercising permissions granted by this License. "Source" form shall mean the preferred form for making modifications, including but not limited to software source code, documentation source, and configuration files. "Object" form shall mean any form resulting from mechanical transformation or translation of a Source form, including but not limited to compiled object code, generated documentation, and conversions to other media types. "Work" shall mean the work of authorship, whether in Source or Object form, made available under the License, as indicated by a copyright notice that is included in or attached to the work (an example is provided in the Appendix below). "Derivative Works" shall mean any work, whether in Source or Object form, that is based on (or derived from) the Work and for which the editorial revisions, annotations, elaborations, or other modifications represent, as a whole, an original work of authorship. For the purposes of this License, Derivative Works shall not include works that remain separable from, or merely link (or bind by name) to the interfaces of, the Work and Derivative Works thereof. "Contribution" shall mean any work of authorship, including the original version of the Work and any modifications or additions to that Work or Derivative Works thereof, that is intentionally submitted to Licensor for inclusion in the Work by the copyright owner or by an individual or Legal Entity authorized to submit on behalf of the copyright owner. For the purposes of this definition, "submitted" means any form of electronic, verbal, or written communication sent to the Licensor or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, the Licensor for the purpose of discussing and improving the Work, but excluding communication that is conspicuously marked or otherwise designated in writing by the copyright owner as "Not a Contribution." "Contributor" shall mean Licensor and any individual or Legal Entity on behalf of whom a Contribution has been received by Licensor and subsequently incorporated within the Work. 2. Grant of Copyright License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare Derivative Works of, publicly display, publicly perform, sublicense, and distribute the Work and such Derivative Works in Source or Object form. 3. Grant of Patent License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims licensable by such Contributor that are necessarily infringed by their Contribution(s) alone or by combination of their Contribution(s) with the Work to which such Contribution(s) was submitted. If You institute patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Work or a Contribution incorporated within the Work constitutes direct or contributory patent infringement, then any patent licenses granted to You under this License for that Work shall terminate as of the date such litigation is filed. 4. Redistribution. You may reproduce and distribute copies of the Work or Derivative Works thereof in any medium, with or without modifications, and in Source or Object form, provided that You meet the following conditions: (a) You must give any other recipients of the Work or Derivative Works a copy of this License; and (b) You must cause any modified files to carry prominent notices stating that You changed the files; and (c) You must retain, in the Source form of any Derivative Works that You distribute, all copyright, patent, trademark, and attribution notices from the Source form of the Work, excluding those notices that do not pertain to any part of the Derivative Works; and (d) If the Work includes a "NOTICE" text file as part of its distribution, then any Derivative Works that You distribute must include a readable copy of the attribution notices contained within such NOTICE file, excluding those notices that do not pertain to any part of the Derivative Works, in at least one of the following places: within a NOTICE text file distributed as part of the Derivative Works; within the Source form or documentation, if provided along with the Derivative Works; or, within a display generated by the Derivative Works, if and wherever such third-party notices normally appear. The contents of the NOTICE file are for informational purposes only and do not modify the License. You may add Your own attribution notices within Derivative Works that You distribute, alongside or as an addendum to the NOTICE text from the Work, provided that such additional attribution notices cannot be construed as modifying the License. You may add Your own copyright statement to Your modifications and may provide additional or different license terms and conditions for use, reproduction, or distribution of Your modifications, or for any such Derivative Works as a whole, provided Your use, reproduction, and distribution of the Work otherwise complies with the conditions stated in this License. 5. Submission of Contributions. Unless You explicitly state otherwise, any Contribution intentionally submitted for inclusion in the Work by You to the Licensor shall be under the terms and conditions of this License, without any additional terms or conditions. Notwithstanding the above, nothing herein shall supersede or modify the terms of any separate license agreement you may have executed with Licensor regarding such Contributions. 6. Trademarks. This License does not grant permission to use the trade names, trademarks, service marks, or product names of the Licensor, except as required for reasonable and customary use in describing the origin of the Work and reproducing the content of the NOTICE file. 7. Disclaimer of Warranty. Unless required by applicable law or agreed to in writing, Licensor provides the Work (and each Contributor provides its Contributions) on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing the Work and assume any risks associated with Your exercise of permissions under this License. 8. Limitation of Liability. In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall any Contributor be liable to You for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising as a result of this License or out of the use or inability to use the Work (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if such Contributor has been advised of the possibility of such damages. 9. Accepting Warranty or Additional Liability. While redistributing the Work or Derivative Works thereof, You may choose to offer, and charge a fee for, acceptance of support, warranty, indemnity, or other liability obligations and/or rights consistent with this License. However, in accepting such obligations, You may act only on Your own behalf and on Your sole responsibility, not on behalf of any other Contributor, and only if You agree to indemnify, defend, and hold each Contributor harmless for any liability incurred by, or claims asserted against, such Contributor by reason of your accepting any such warranty or additional liability. masakari-monitors-9.0.0/.coveragerc0000664000175000017500000000011513656750014017327 0ustar zuulzuul00000000000000[run] branch = True source = masakarimonitors [report] ignore_errors = True