pax_global_header00006660000000000000000000000064122571121550014513gustar00rootroot0000000000000052 comment=f10890386bb37239f7a342060b2562ea31d7ee6d reclass-1.2.2/000077500000000000000000000000001225711215500131515ustar00rootroot00000000000000reclass-1.2.2/.gitignore000066400000000000000000000001161225711215500151370ustar00rootroot00000000000000*.py[co] .*.sw? /reclass-config.yml /reclass.egg-info /build /dist /.coverage reclass-1.2.2/.pylintrc000066400000000000000000000000371225711215500150160ustar00rootroot00000000000000[MASTER] [REPORTS] reports=no reclass-1.2.2/ChangeLog.rst000077700000000000000000000000001225711215500222532doc/source/changelog.rstustar00rootroot00000000000000reclass-1.2.2/LICENSE000066400000000000000000000222061225711215500141600ustar00rootroot00000000000000reclass is © 2007–2013 by martin f. krafft Released under the terms of the Artistic Licence 2.0. "The Artistic Licence 2.0" Copyright (c) 2000-2006, The Perl Foundation. http://www.perlfoundation.org/legal/licenses/artistic-2_0.html Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed. Preamble ~~~~~~~~ This license establishes the terms under which a given free software Package may be copied, modified, distributed, and/or redistributed. The intent is that the Copyright Holder maintains some artistic control over the development of that Package while still keeping the Package available as open source and free software. You are always permitted to make arrangements wholly outside of this license directly with the Copyright Holder of a given Package. If the terms of this license do not permit the full use that you propose to make of the Package, you should contact the Copyright Holder and seek a different licensing arrangement. Definitions ~~~~~~~~~~~ "Copyright Holder" means the individual(s) or organization(s) named in the copyright notice for the entire Package. "Contributor" means any party that has contributed code or other material to the Package, in accordance with the Copyright Holder's procedures. "You" and "your" means any person who would like to copy, distribute, or modify the Package. "Package" means the collection of files distributed by the Copyright Holder, and derivatives of that collection and/or of those files. A given Package may consist of either the Standard Version, or a Modified Version. "Distribute" means providing a copy of the Package or making it accessible to anyone else, or in the case of a company or organization, to others outside of your company or organization. "Distributor Fee" means any fee that you charge for Distributing this Package or providing support for this Package to another party. It does not mean licensing fees. "Standard Version" refers to the Package if it has not been modified, or has been modified only in ways explicitly requested by the Copyright Holder. "Modified Version" means the Package, if it has been changed, and such changes were not explicitly requested by the Copyright Holder. "Original License" means this Artistic License as Distributed with the Standard Version of the Package, in its current version or as it may be modified by The Perl Foundation in the future. "Source" form means the source code, documentation source, and configuration files for the Package. "Compiled" form means the compiled bytecode, object code, binary, or any other form resulting from mechanical transformation or translation of the Source form. Permission for Use and Modification Without Distribution ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ (1) You are permitted to use the Standard Version and create and use Modified Versions for any purpose without restriction, provided that you do not Distribute the Modified Version. Permissions for Redistribution of the Standard Version ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ (2) You may Distribute verbatim copies of the Source form of the Standard Version of this Package in any medium without restriction, either gratis or for a Distributor Fee, provided that you duplicate all of the original copyright notices and associated disclaimers. At your discretion, such verbatim copies may or may not include a Compiled form of the Package. (3) You may apply any bug fixes, portability changes, and other modifications made available from the Copyright Holder. The resulting Package will still be considered the Standard Version, and as such will be subject to the Original License. Distribution of Modified Versions of the Package as Source ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ (4) You may Distribute your Modified Version as Source (either gratis or for a Distributor Fee, and with or without a Compiled form of the Modified Version) provided that you clearly document how it differs from the Standard Version, including, but not limited to, documenting any non-standard features, executables, or modules, and provided that you do at least ONE of the following: (a) make the Modified Version available to the Copyright Holder of the Standard Version, under the Original License, so that the Copyright Holder may include your modifications in the Standard Version. (b) ensure that installation of your Modified Version does not prevent the user installing or running the Standard Version. In addition, the Modified Version must bear a name that is different from the name of the Standard Version. (c) allow anyone who receives a copy of the Modified Version to make the Source form of the Modified Version available to others under (i) the Original License or (ii) a license that permits the licensee to freely copy, modify and redistribute the Modified Version using the same licensing terms that apply to the copy that the licensee received, and requires that the Source form of the Modified Version, and of any works derived from it, be made freely available in that license fees are prohibited but Distributor Fees are allowed. Distribution of Compiled Forms of the Standard Version or Modified Versions without the Source ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ (5) You may Distribute Compiled forms of the Standard Version without the Source, provided that you include complete instructions on how to get the Source of the Standard Version. Such instructions must be valid at the time of your distribution. If these instructions, at any time while you are carrying out such distribution, become invalid, you must provide new instructions on demand or cease further distribution. If you provide valid instructions or cease distribution within thirty days after you become aware that the instructions are invalid, then you do not forfeit any of your rights under this license. (6) You may Distribute a Modified Version in Compiled form without the Source, provided that you comply with Section 4 with respect to the Source of the Modified Version. Aggregating or Linking the Package ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ (7) You may aggregate the Package (either the Standard Version or Modified Version) with other packages and Distribute the resulting aggregation provided that you do not charge a licensing fee for the Package. Distributor Fees are permitted, and licensing fees for other components in the aggregation are permitted. The terms of this license apply to the use and Distribution of the Standard or Modified Versions as included in the aggregation. (8) You are permitted to link Modified and Standard Versions with other works, to embed the Package in a larger work of your own, or to build stand-alone binary or bytecode versions of applications that include the Package, and Distribute the result without restriction, provided the result does not expose a direct interface to the Package. Items That are Not Considered Part of a Modified Version ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ (9) Works (including, but not limited to, modules and scripts) that merely extend or make use of the Package, do not, by themselves, cause the Package to be a Modified Version. In addition, such works are not considered parts of the Package itself, and are not subject to the terms of this license. General Provisions ~~~~~~~~~~~~~~~~~~ (10) Any use, modification, and distribution of the Standard or Modified Versions is governed by this Artistic License. By using, modifying or distributing the Package, you accept this license. Do not use, modify, or distribute the Package, if you do not accept this license. (11) If your Modified Version has been derived from a Modified Version made by someone other than you, you are nevertheless required to ensure that your Modified Version complies with the requirements of this license. (12) This license does not grant you the right to use any trademark, service mark, tradename, or logo of the Copyright Holder. (13) This license includes the non-exclusive, worldwide, free-of-charge patent license to make, have made, use, offer to sell, sell, import and otherwise transfer the Package with respect to any patent claims licensable by the Copyright Holder that are necessarily infringed by the Package. If you institute patent litigation (including a cross-claim or counterclaim) against any party alleging that the Package constitutes direct or contributory patent infringement, then this Artistic License to you shall terminate on the date that such litigation is filed. (14) Disclaimer of Warranty: THE PACKAGE IS PROVIDED BY THE COPYRIGHT HOLDER AND CONTRIBUTORS "AS IS' AND WITHOUT ANY EXPRESS OR IMPLIED WARRANTIES. THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR NON-INFRINGEMENT ARE DISCLAIMED TO THE EXTENT PERMITTED BY YOUR LOCAL LAW. UNLESS REQUIRED BY LAW, NO COPYRIGHT HOLDER OR CONTRIBUTOR WILL BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, OR CONSEQUENTIAL DAMAGES ARISING IN ANY WAY OUT OF THE USE OF THE PACKAGE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. reclass-1.2.2/Makefile000066400000000000000000000025261225711215500146160ustar00rootroot00000000000000# # This file is part of reclass (http://github.com/madduck/reclass) # # Copyright © 2007–13 martin f. krafft # Released under the terms of the Artistic Licence 2.0 # PYFILES = $(shell find -name .git -o -name dist -o -name build -prune -o -name '*.py' -print) tests: python ./run_tests.py .PHONY: tests lint: @echo pylint --rcfile=.pylintrc $(ARGS) … @pylint --rcfile=.pylintrc $(ARGS) $(PYFILES) .PHONY: lint lint-errors: ARGS=--errors-only lint-errors: lint .PHONY: lint-errors lint-report: ARGS=--report=y lint-report: lint .PHONY: lint-report coverage: .coverage python-coverage -r -m .PHONY: coverage .coverage: $(PYFILES) python-coverage -x setup.py nosetests docs: $(MAKE) -C doc man html GH_BRANCH=gh-pages HTMLDIR=doc/build/html docspub: ifeq ($(shell git branch --list $(GH_BRANCH)-base),) @echo "Please fetch the $(GH_BRANCH)-base branch from Github to be able to publish documentation:" >&2 @echo " git branch gh-pages-base origin/gh-pages-base" >&2 @false else $(MAKE) docs git checkout $(GH_BRANCH) || git checkout -b $(GH_BRANCH) $(GH_BRANCH)-base git reset --hard $(GH_BRANCH)-base git add $(HTMLDIR) git mv $(HTMLDIR)/* . git commit -m'Webpage update' git push -f $(shell git config --get branch.$(GH_BRANCH)-base.remote) $(GH_BRANCH) git checkout '@{-1}' endif docsclean: $(MAKE) -C doc clean reclass-1.2.2/README.rst000066400000000000000000000001671225711215500146440ustar00rootroot00000000000000reclass README ============== The documentation for **reclass** is available from http://reclass.pantsfullofunix.net. reclass-1.2.2/doc/000077500000000000000000000000001225711215500137165ustar00rootroot00000000000000reclass-1.2.2/doc/.gitignore000066400000000000000000000000071225711215500157030ustar00rootroot00000000000000/build reclass-1.2.2/doc/Makefile000066400000000000000000000127141225711215500153630ustar00rootroot00000000000000# Makefile for Sphinx documentation # # You can set these variables from the command line. SPHINXOPTS = SPHINXBUILD = sphinx-build -N PAPER = BUILDDIR = build # Internal variables. PAPEROPT_a4 = -D latex_paper_size=a4 PAPEROPT_letter = -D latex_paper_size=letter ALLSPHINXOPTS = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) source # the i18n builder cannot share the environment and doctrees with the others I18NSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) source .PHONY: help clean html dirhtml singlehtml pickle json htmlhelp qthelp devhelp epub latex latexpdf text man changes linkcheck doctest gettext help: @echo "Please use \`make ' where is one of" @echo " html to make standalone HTML files" @echo " dirhtml to make HTML files named index.html in directories" @echo " singlehtml to make a single large HTML file" @echo " pickle to make pickle files" @echo " json to make JSON files" @echo " htmlhelp to make HTML files and a HTML help project" @echo " qthelp to make HTML files and a qthelp project" @echo " devhelp to make HTML files and a Devhelp project" @echo " epub to make an epub" @echo " latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter" @echo " latexpdf to make LaTeX files and run them through pdflatex" @echo " text to make text files" @echo " man to make manual pages" @echo " texinfo to make Texinfo files" @echo " info to make Texinfo files and run them through makeinfo" @echo " gettext to make PO message catalogs" @echo " changes to make an overview of all changed/added/deprecated items" @echo " linkcheck to check all external links for integrity" @echo " doctest to run all doctests embedded in the documentation (if enabled)" clean: -rm -rf $(BUILDDIR)/* html: $(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html @echo @echo "Build finished. The HTML pages are in $(BUILDDIR)/html." dirhtml: $(SPHINXBUILD) -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/dirhtml @echo @echo "Build finished. The HTML pages are in $(BUILDDIR)/dirhtml." singlehtml: $(SPHINXBUILD) -b singlehtml $(ALLSPHINXOPTS) $(BUILDDIR)/singlehtml @echo @echo "Build finished. The HTML page is in $(BUILDDIR)/singlehtml." pickle: $(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) $(BUILDDIR)/pickle @echo @echo "Build finished; now you can process the pickle files." json: $(SPHINXBUILD) -b json $(ALLSPHINXOPTS) $(BUILDDIR)/json @echo @echo "Build finished; now you can process the JSON files." htmlhelp: $(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) $(BUILDDIR)/htmlhelp @echo @echo "Build finished; now you can run HTML Help Workshop with the" \ ".hhp project file in $(BUILDDIR)/htmlhelp." qthelp: $(SPHINXBUILD) -b qthelp $(ALLSPHINXOPTS) $(BUILDDIR)/qthelp @echo @echo "Build finished; now you can run "qcollectiongenerator" with the" \ ".qhcp project file in $(BUILDDIR)/qthelp, like this:" @echo "# qcollectiongenerator $(BUILDDIR)/qthelp/reclass.qhcp" @echo "To view the help file:" @echo "# assistant -collectionFile $(BUILDDIR)/qthelp/reclass.qhc" devhelp: $(SPHINXBUILD) -b devhelp $(ALLSPHINXOPTS) $(BUILDDIR)/devhelp @echo @echo "Build finished." @echo "To view the help file:" @echo "# mkdir -p $$HOME/.local/share/devhelp/reclass" @echo "# ln -s $(BUILDDIR)/devhelp $$HOME/.local/share/devhelp/reclass" @echo "# devhelp" epub: $(SPHINXBUILD) -b epub $(ALLSPHINXOPTS) $(BUILDDIR)/epub @echo @echo "Build finished. The epub file is in $(BUILDDIR)/epub." latex: $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex @echo @echo "Build finished; the LaTeX files are in $(BUILDDIR)/latex." @echo "Run \`make' in that directory to run these through (pdf)latex" \ "(use \`make latexpdf' here to do that automatically)." latexpdf: $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex @echo "Running LaTeX files through pdflatex..." $(MAKE) -C $(BUILDDIR)/latex all-pdf @echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex." text: $(SPHINXBUILD) -b text $(ALLSPHINXOPTS) $(BUILDDIR)/text @echo @echo "Build finished. The text files are in $(BUILDDIR)/text." man: $(SPHINXBUILD) -b man $(ALLSPHINXOPTS) $(BUILDDIR)/man @echo @echo "Build finished. The manual pages are in $(BUILDDIR)/man." texinfo: $(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo @echo @echo "Build finished. The Texinfo files are in $(BUILDDIR)/texinfo." @echo "Run \`make' in that directory to run these through makeinfo" \ "(use \`make info' here to do that automatically)." info: $(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo @echo "Running Texinfo files through makeinfo..." make -C $(BUILDDIR)/texinfo info @echo "makeinfo finished; the Info files are in $(BUILDDIR)/texinfo." gettext: $(SPHINXBUILD) -b gettext $(I18NSPHINXOPTS) $(BUILDDIR)/locale @echo @echo "Build finished. The message catalogs are in $(BUILDDIR)/locale." changes: $(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) $(BUILDDIR)/changes @echo @echo "The overview file is in $(BUILDDIR)/changes." linkcheck: $(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) $(BUILDDIR)/linkcheck @echo @echo "Link check complete; look for any errors in the above output " \ "or in $(BUILDDIR)/linkcheck/output.txt." doctest: $(SPHINXBUILD) -b doctest $(ALLSPHINXOPTS) $(BUILDDIR)/doctest @echo "Testing of doctests in the sources finished, look at the " \ "results in $(BUILDDIR)/doctest/output.txt." reclass-1.2.2/doc/source/000077500000000000000000000000001225711215500152165ustar00rootroot00000000000000reclass-1.2.2/doc/source/ansible.rst000066400000000000000000000200441225711215500173650ustar00rootroot00000000000000========================== Using reclass with Ansible ========================== .. warning:: I was kicked out of the Ansible community, presumably for `asking the wrong questions`_, and therefore I have no interest in developing this adapter anymore. If you use it and have changes, I will take your patch. .. _asking the wrong questions: https://github.com/madduck/reclass/issues/6 Quick start with Ansible ------------------------ The following steps should get you up and running quickly with |reclass| and `Ansible`_. Generally, we will be working in ``/etc/ansible``. However, if you are using a source-code checkout of Ansible, you might also want to work inside the ``./hacking`` directory instead. Or you can also just look into ``./examples/ansible`` of your |reclass| checkout, where the following steps have already been prepared. /…/reclass refers to the location of your |reclass| checkout. .. todo:: With |reclass| now in Debian, as well as installable from source, the following should be checked for path consistency… #. Complete the installation steps described in the :doc:`installation section `. #. Symlink ``/usr/share/reclass/reclass-ansible`` (or wherever your distro put that file), or ``/…/reclass/reclass/adapters/ansible.py`` (if running from source) to ``/etc/ansible/hosts`` (or ``./hacking/hosts``). #. Copy the two directories ``nodes`` and ``classes`` from the example subdirectory in the |reclass| checkout to ``/etc/ansible`` If you prefer to put those directories elsewhere, you can create ``/etc/ansible/reclass-config.yml`` with contents such as:: storage_type: yaml_fs inventory_base_uri: /srv/reclass Note that ``yaml_fs`` is currently the only supported ``storage_type``, and it's the default if you don't set it. #. Check out your inventory by invoking :: $ ./hosts --list which should return 5 groups in JSON format, and each group has exactly one member ``localhost``. 4. See the node information for ``localhost``:: $ ./hosts --host localhost This should print a set of keys and values, including a greeting, a colour, and a sub-class called ``__reclas__``. 5. Execute some ansible commands, e.g.:: $ ansible -i hosts \* --list-hosts $ ansible -i hosts \* -m ping $ ansible -i hosts \* -m debug -a 'msg="${greeting}"' $ ansible -i hosts \* -m setup $ ansible-playbook -i hosts test.yml 6. You can also invoke |reclass| directly, which gives a slightly different view onto the same data, i.e. before it has been adapted for Ansible:: $ /…/reclass/reclass.py --pretty-print --inventory $ /…/reclass/reclass.py --pretty-print --nodeinfo localhost Or, if |reclass| is properly installed, just use the |reclass| command. Integration with Ansible ------------------------ The integration between |reclass| and Ansible is performed through an adapter, and needs not be of our concern too much. However, Ansible has no concept of "nodes", "applications", "parameters", and "classes". Therefore it is necessary to explain how those correspond to Ansible. Crudely, the following mapping exists: ================= =============== |reclass| concept Ansible concept ================= =============== nodes hosts classes groups applications playbooks parameters host_vars ================= =============== |reclass| does not provide any ``group_vars`` because of its node-centric perspective. While class definitions include parameters, those are inherited by the node definitions and hence become node_vars. |reclass| also does not provide playbooks, nor does it deal with any of the related Ansible concepts, i.e. ``vars_files``, vars, tasks, handlers, roles, etc.. Let it be said at this point that you'll probably want to stop using ``host_vars``, ``group_vars`` and ``vars_files`` altogether, and if only because you should no longer need them, but also because the variable precedence rules of Ansible are full of surprises, at least to me. |reclass|' Ansible adapter massage the |reclass| output into Ansible-usable data, namely: - Every class in the ancestry of a node becomes a group to Ansible. This is mainly useful to be able to target nodes during interactive use of Ansible, e.g.:: $ ansible debiannode@wheezy -m command -a 'apt-get upgrade' → upgrade all Debian nodes running wheezy $ ansible ssh.server -m command -a 'invoke-rc.d ssh restart' → restart all SSH server processes $ ansible mailserver -m command -a 'tail -n1000 /var/log/mail.err' → obtain the last 1,000 lines of all mailserver error log files The attentive reader might stumble over the use of singular words, whereas it might make more sense to address all ``mailserver*s*`` with this tool. This is convention and up to you. I prefer to think of my node as a (singular) mailserver when I add ``mailserver`` to its parent classes. - Every entry in the list of a host's applications might well correspond to an Ansible playbook. Therefore, |reclass| creates a (Ansible-)group for every application, and adds ``_hosts`` to the name. This postfix can be configured with a CLI option (``--applications-postfix``) or in the configuration file (``applications_postfix``). For instance, the ssh.server class adds the ssh.server application to a node's application list. Now the admin might create an Ansible playbook like so:: - name: SSH server management hosts: ssh.server_hosts ← SEE HERE tasks: - name: install SSH package action: … … There's a bit of redundancy in this, but unfortunately Ansible playbooks hardcode the nodes to which a playbook applies. It's now trivial to apply this playbook across your infrastructure:: $ ansible-playbook ssh.server.yml My suggested way to use Ansible site-wide is then to create a ``site.yml`` playbook that includes all the other playbooks (which shall hopefully be based on Ansible roles), and then to invoke Ansible like this: ansible-playbook site.yml or, if you prefer only to reconfigure a subset of nodes, e.g. all webservers:: $ ansible-playbook site.yml --limit webserver Again, if the singular word ``webserver`` puts you off, change the convention as you wish. And if anyone comes up with a way to directly connect groups in the inventory with roles, thereby making it unnecessary to write playbook files (containing redundant information), please tell me! - Parameters corresponding to a node become ``host_vars`` for that host. Variable interpolation ---------------------- Ansible allows you to include `Jinja2`_-style variables in parameter values:: parameters: motd: greeting: Welcome to {{ ansible_fqdn }}! closing: This system is part of {{ realm }} dict_reference: {{ motd }} However, in resolving this, Ansible casts everything to a string, so in this example, ``dict_reference`` would be the string-representation of the dictionary under the ``motd`` key [#string_casts]_. To get at facts (such as ``ansible_fqdn``), you still have to use this approach, but for pure parameter references, I strongly suggest to use |reclass| interpolation instead, as it supports deep references, does not clobber type information, and is more efficient anyway:: parameters: motd: greeting: Welcome to {{ ansible_fqdn }}! closing: This system is part of ${realm} dict_reference: ${motd} Now you just need to specify realm somewhere. The reference can reside in a parent class, while the variable is defined e.g. in the node definition. And as expected, ``dict_reference`` now points to a dictionary, not a string-representation thereof. .. [#string_casts] I pointed this out to Michael Dehaan, Ansible's chief developer, but he denied this behaviour. When I tried to provide further insights, I found myself banned from the mailing list, apparently because I dared to point out flaws. If you care, you may look at https://github.com/madduck/reclass/issues/6 for more information. .. include:: extrefs.inc .. include:: substs.inc reclass-1.2.2/doc/source/changelog.rst000066400000000000000000000027001225711215500176760ustar00rootroot00000000000000========= ChangeLog ========= ========= ========== ======================================================== Version Date Changes ========= ========== ======================================================== 1.2.2 2013-12-27 * Recurse classes obtained from class mappings (closes: #16) * Fix class mapping regexp rendering in docs (closes: #15) 1.2.1 2013-12-26 * Fix Salt adapter wrt. class mappings (closes: #14) 1.2 2013-12-10 * Introduce class mappings (see :doc:`operations`) (closes: #5) * Fix parameter interpolation across merged lists (closes: #13). * Caching of classes for performance reasons, especially during the inventory runs * yaml_fs: nodes may be defined in subdirectories (closes: #10). * Classes and nodes URI must not overlap anymore * Class names must not contain spaces 1.1 2013-08-28 Salt adapter: fix interface to include minion_id, filter output accordingly; fixes master_tops 1.0.2 2013-08-27 Fix incorrect versioning in setuptools 1.0.1 2013-08-27 Documentation updates, new homepage 1.0 2013-08-26 Initial release ========= ========== ======================================================== reclass-1.2.2/doc/source/concepts.rst000066400000000000000000000142401225711215500175670ustar00rootroot00000000000000================ reclass concepts ================ |reclass| assumes a node-centric perspective into your inventory. This is obvious when you query |reclass| for node-specific information, but it might not be clear when you ask |reclass| to provide you with a list of groups. In that case, |reclass| loops over all nodes it can find in its database, reads all information it can find about the nodes, and finally reorders the result to provide a list of groups with the nodes they contain. Since the term "groups" is somewhat ambiguous, it helps to start off with a short glossary of |reclass|-specific terminology: ============ ============================================================== Concept Description ============ ============================================================== node A node, usually a computer in your infrastructure class A category, tag, feature, or role that applies to a node Classes may be nested, i.e. there can be a class hierarchy application A specific set of behaviour to apply parameter Node-specific variables, with inheritance throughout the class hierarchy. ============ ============================================================== A class consists of zero or more parent classes, zero or more applications, and any number of parameters. A class name must not contain spaces. A node is almost equivalent to a class, except that it usually does not (but can) specify applications. When |reclass| parses a node (or class) definition and encounters a parent class, it recurses to this parent class first before reading any data of the node (or class). When |reclass| returns from the recursive, depth first walk, it then merges all information of the current node (or class) into the information it obtained during the recursion. Furthermore, a node (or class) may define a list of classes it derives from, in which case classes defined further down the list will be able to override classes further up the list. Information in this context is essentially one of a list of applications or a list of parameters. The interaction between the depth-first walk and the delayed merging of data means that the node (and any class) may override any of the data defined by any of the parent classes (ancestors). This is in line with the assumption that more specific definitions ("this specific host") should have a higher precedence than more general definitions ("all webservers", which includes all webservers in Munich, which includes "this specific host", for example). Here's a quick example, showing how parameters accumulate and can get replaced. All "unixnodes" (i.e. nodes who have the ``unixnode`` class in their ancestry) have ``/etc/motd`` centrally-managed (through the ``motd`` application), and the `unixnode` class definition provides a generic message-of-the-day to be put into this file. All descendants of the class ``debiannode``, a descendant of ``unixnode``, should include the Debian codename in this message, so the message-of-the-day is overwritten in the ``debiannodes`` class. The node ``quantum.example.org`` (a `debiannode`) will have a scheduled downtime this weekend, so until Monday, an appropriate message-of-the-day is added to the node definition. When the ``motd`` application runs, it receives the appropriate message-of-the-day (from ``quantum.example.org`` when run on that node) and writes it into ``/etc/motd``. At this point it should be noted that parameters whose values are lists or key-value pairs don't get overwritten by children classes or node definitions, but the information gets merged (recursively) instead. Similarly to parameters, applications also accumulate during the recursive walk through the class ancestry. It is possible for a node or child class to *remove* an application added by a parent class, by prefixing the application with `~`. Finally, |reclass| happily lets you use multiple inheritance, and ensures that the resolution of parameters is still well-defined. Here's another example building upon the one about ``/etc/motd`` above: ``quantum.example.org`` (which is back up and therefore its node definition no longer contains a message-of-the-day) is at a site in Munich. Therefore, it is a child of the class ``hosted@munich``. This class is independent of the ``unixnode`` hierarchy, ``quantum.example.org`` derives from both. In this example infrastructure, ``hosted@munich`` is more specific than ``debiannode`` because there are plenty of Debian nodes at other sites (and some non-Debian nodes in Munich). Therefore, ``quantum.example.org`` derives from ``hosted@munich`` _after_ ``debiannodes``. When an electricity outage is expected over the weekend in Munich, the admin can change the message-of-the-day in the ``hosted@munich`` class, and it will apply to all hosts in Munich. However, not all hosts in Munich have ``/etc/motd``, because some of them are of class ``windowsnode``. Since the ``windowsnode`` ancestry does not specify the ``motd`` application, those hosts have access to the message-of-the-day in the node variables, but the message won't get used… … unless, of course, ``windowsnode`` specified a Windows-specific application to bring such notices to the attention of the user. It's also trivial to ensure a certain order of class evaluation. Here's another example: The ``ssh.server`` class defines the ``permit_root_login`` parameter to ``no``. The ``backuppc.client`` class defines the parameter to ``without-password``, because the BackupPC server might need to log in to the host as root. Now, what happens if the admin accidentally provides the following two classes? - ``backuppc.client`` - ``ssh.server`` Theoretically, this would mean ``permit_root_login`` gets set to ``no``. However, since all ``backuppc.client`` nodes need ``ssh.server`` (at least in most setups), the class ``backuppc.client`` itself derives from ``ssh.server``, ensuring that it gets parsed before ``backuppc.client``. When |reclass| returns to the node and encounters the ``ssh.server`` class defined there, it simply skips it, as it's already been processed. Now read about :doc:`operations`! .. include:: substs.inc reclass-1.2.2/doc/source/conf.py000066400000000000000000000172561225711215500165300ustar00rootroot00000000000000# -*- coding: utf-8 -*- # # reclass documentation build configuration file, created by # sphinx-quickstart on Mon Aug 26 12:56:14 2013. # # This file is execfile()d with the current directory set to its containing dir. # # Note that not all possible configuration values are present in this # autogenerated file. # # All configuration values have a default; values that are commented out # serve to show the default. import sys, os # If extensions (or modules to document with autodoc) are in another directory, # add these directories to sys.path here. If the directory is relative to the # documentation root, use os.path.abspath to make it absolute, like shown here. #sys.path.insert(0, os.path.abspath('.')) # -- General configuration ----------------------------------------------------- # If your documentation needs a minimal Sphinx version, state it here. #needs_sphinx = '1.0' # Add any Sphinx extension module names here, as strings. They can be extensions # coming with Sphinx (named 'sphinx.ext.*') or your custom ones. extensions = ['sphinx.ext.autodoc', 'sphinx.ext.todo'] # Add any paths that contain templates here, relative to this directory. templates_path = ['_templates'] # The suffix of source filenames. source_suffix = '.rst' # The encoding of source files. #source_encoding = 'utf-8-sig' # The master toctree document. master_doc = 'index' # General information about the project. project = u'reclass' copyright = u'2013, martin f. krafft' # The version info for the project you're documenting, acts as replacement for # |version| and |release|, also used in various other places throughout the # built documents. import reclass.version # The short X.Y version. version = '.'.join(reclass.version.VERSION.split('.')[:2]) # The full version, including alpha/beta/rc tags. release = reclass.version.VERSION # The language for content autogenerated by Sphinx. Refer to documentation # for a list of supported languages. #language = None # There are two options for replacing |today|: either, you set today to some # non-false value, then it is used: #today = '' # Else, today_fmt is used as the format for a strftime call. #today_fmt = '%B %d, %Y' # List of patterns, relative to source directory, that match files and # directories to ignore when looking for source files. exclude_patterns = [] # The reST default role (used for this markup: `text`) to use for all documents. #default_role = None # If true, '()' will be appended to :func: etc. cross-reference text. #add_function_parentheses = True # If true, the current module name will be prepended to all description # unit titles (such as .. function::). #add_module_names = True # If true, sectionauthor and moduleauthor directives will be shown in the # output. They are ignored by default. #show_authors = False # The name of the Pygments (syntax highlighting) style to use. pygments_style = 'sphinx' # A list of ignored prefixes for module index sorting. #modindex_common_prefix = [] # -- Options for HTML output --------------------------------------------------- # The theme to use for HTML and HTML Help pages. See the documentation for # a list of builtin themes. html_theme = 'default' # Theme options are theme-specific and customize the look and feel of a theme # further. For a list of options available for each theme, see the # documentation. #html_theme_options = {} # Add any paths that contain custom themes here, relative to this directory. #html_theme_path = [] # The name for this set of Sphinx documents. If None, it defaults to # " v documentation". #html_title = None # A shorter title for the navigation bar. Default is the same as html_title. html_short_title = 'reclass' # The name of an image file (relative to this directory) to place at the top # of the sidebar. #html_logo = None # The name of an image file (within the static path) to use as favicon of the # docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32 # pixels large. #html_favicon = None # Add any paths that contain custom static files (such as style sheets) here, # relative to this directory. They are copied after the builtin static files, # so a file named "default.css" will overwrite the builtin "default.css". #html_static_path = ['_static'] # If not '', a 'Last updated on:' timestamp is inserted at every page bottom, # using the given strftime format. #html_last_updated_fmt = '%b %d, %Y' # If true, SmartyPants will be used to convert quotes and dashes to # typographically correct entities. #html_use_smartypants = True # Custom sidebar templates, maps document names to template names. #html_sidebars = {} # Additional templates that should be rendered to pages, maps page names to # template names. #html_additional_pages = {} # If false, no module index is generated. #html_domain_indices = True # If false, no index is generated. #html_use_index = True # If true, the index is split into individual pages for each letter. #html_split_index = False # If true, links to the reST sources are added to the pages. html_show_sourcelink = False # If true, "Created using Sphinx" is shown in the HTML footer. Default is True. #html_show_sphinx = True # If true, "(C) Copyright ..." is shown in the HTML footer. Default is True. #html_show_copyright = True # If true, an OpenSearch description file will be output, and all pages will # contain a tag referring to it. The value of this option must be the # base URL from which the finished HTML is served. #html_use_opensearch = '' # This is the file name suffix for HTML files (e.g. ".xhtml"). #html_file_suffix = None # Output file base name for HTML help builder. htmlhelp_basename = 'reclassdoc' # -- Options for LaTeX output -------------------------------------------------- latex_elements = { # The paper size ('letterpaper' or 'a4paper'). 'papersize': 'a4paper', # The font size ('10pt', '11pt' or '12pt'). #'pointsize': '10pt', # Additional stuff for the LaTeX preamble. #'preamble': '', } # Grouping the document tree into LaTeX files. List of tuples # (source start file, target name, title, author, documentclass [howto/manual]). latex_documents = [ ('index', 'reclass.tex', u'reclass Documentation', u'martin f. krafft', 'manual'), ] # The name of an image file (relative to this directory) to place at the top of # the title page. #latex_logo = None # For "manual" documents, if this is true, then toplevel headings are parts, # not chapters. #latex_use_parts = False # If true, show page references after internal links. #latex_show_pagerefs = False # If true, show URL addresses after external links. #latex_show_urls = False # Documents to append as an appendix to all manuals. #latex_appendices = [] # If false, no module index is generated. #latex_domain_indices = True # -- Options for manual page output -------------------------------------------- # One entry per manual page. List of tuples # (source start file, name, description, authors, manual section). man_pages = [ ('manpage', 'reclass', u'command-line interface', [u'martin f. krafft'], 1) ] # If true, show URL addresses after external links. #man_show_urls = False # -- Options for Texinfo output ------------------------------------------------ # Grouping the document tree into Texinfo files. List of tuples # (source start file, target name, title, author, # dir menu entry, description, category) texinfo_documents = [ ('index', 'reclass', u'reclass Documentation', u'martin f. krafft', 'reclass', 'One line description of project.', 'Miscellaneous'), ] # Documents to append as an appendix to all manuals. #texinfo_appendices = [] # If false, no module index is generated. #texinfo_domain_indices = True # How to display URL addresses: 'footnote', 'no', or 'inline'. #texinfo_show_urls = 'footnote' reclass-1.2.2/doc/source/configfile.rst000066400000000000000000000025661225711215500200660ustar00rootroot00000000000000========================== reclass configuration file ========================== |reclass| can read some of its configuration from a file. The file is a YAML-file and simply defines key-value pairs. The configuration file can be used to set defaults for all the options that are otherwise configurable via the command-line interface, so please use the ``--help`` output of |reclass| (or the :doc:`manual page `) for reference. The command-line option ``--nodes-uri`` corresponds to the key ``nodes_uri`` in the configuration file. For example:: storage_type: yaml_fs pretty_print: True output: json inventory_base_uri: /etc/reclass nodes_uri: ../nodes |reclass| first looks in the current directory for the file called ``reclass-config.yml`` (see ``reclass/defaults.py``) and if no such file is found, it looks in ``$HOME``, then in ``/etc/reclass``, and then "next to" the ``reclass`` script itself, i.e. if the script is symlinked to ``/srv/provisioning/reclass``, then the the script will try to access ``/srv/provisioning/reclass-config.yml``. Note that ``yaml_fs`` is currently the only supported ``storage_type``, and it's the default if you don't set it. Adapters may implement their own lookup logic, of course, so make sure to read their documentation (for :doc:`Salt `, for :doc:`Ansible `, and for :doc:`Puppet `). .. include:: substs.inc reclass-1.2.2/doc/source/extrefs.inc000066400000000000000000000005031225711215500173670ustar00rootroot00000000000000.. _Puppet: http://puppetlabs.com/puppet/puppet-open-source .. _Salt: http://saltstack.com/community .. _Ansible: http://www.ansibleworks.com .. _Hiera: http://projects.puppetlabs.com/projects/hiera .. _Artistic Licence 2.0: http://www.perlfoundation.org/legal/licenses/artistic-2_0.html .. _Jinja2: http://jinja.pocoo.org reclass-1.2.2/doc/source/hacking.rst000066400000000000000000000040441225711215500173560ustar00rootroot00000000000000================== Hacking on reclass ================== Installation ------------ If you just want to run |reclass| from source, e.g. because you are going to be making and testing changes, install it in "development mode":: python setup.py develop Now the ``reclass`` script, as well as the adapters, will be available in ``/usr/local/bin``, and you can also invoke them directly from the source tree. To uninstall:: python setup.py develop --uninstall Discussing reclass ------------------ If you want to talk about |reclass|, use the `mailing list`_ or to find me on IRC, in ``#reclass`` on ``irc.oftc.net``. .. _mailing list: http://lists.pantsfullofunix.net/listinfo/reclass Contributing to reclass ----------------------- |reclass| is currently maintained `on Github `_. Conttributions to |reclass| are very welcome. Since I prefer to keep a somewhat clean history, I will not just merge pull request. You can submit pull requests, of course, and I'll rebase them onto ``HEAD`` before merging. Or send your patches using ``git-format-patch`` and ``git-send-e-mail`` to `the mailing list `_. I have added rudimentary unit tests, and it would be nice if you could submit your changes with appropriate changes to the tests. To run tests, invoke :: $ make tests in the top-level checkout directory. The tests are rather inconsistent, some using mock objects, and only the datatypes-related code is covered. If you are a testing expert, I could certainly use some help here to improve the consistency of the existing tests, as well as their coverage. Also, there is a Makefile giving access to PyLint and ``coverage.py`` (running tests). If you run that, you can see there is a lot of work to be done cleaning up the code. If this is the sort of stuff you want to do — by all means — be my guest! ;) There are a number of items on the :doc:`to-do list `, so if you are bored… If you have larger ideas, I'll be looking forward to discuss them with you. .. include:: substs.inc reclass-1.2.2/doc/source/index.rst000066400000000000000000000043121225711215500170570ustar00rootroot00000000000000================================================ reclass — Recursive external node classification ================================================ .. include:: intro.inc Releases and source code ------------------------ The latest released |reclass| version is |release|. Please have a look at the :doc:`change log ` for information about recent changes. For now, |reclass| is hosted `on Github`_, and you may clone it with the following command:: git clone https://github.com/madduck/reclass.git Please see the :doc:`install instructions ` for information about distribution packages and tarballs. .. _on Github: https://github.com/madduck/reclass Community --------- There is a `mailing list`_, where you can bring up anything related to |reclass|. .. _mailing list: http://lists.pantsfullofunix.net/listinfo/reclass For real-time communication, please join the ``#reclass`` IRC channel on ``irc.oftc.net``. If you're using `Salt`_, you can also ask your |reclass|-and-Salt-related questions on the mailing list, ideally specifying "reclass" in the subject of your message. Licence ------- |reclass| is © 2007–2013 by martin f. krafft and released under the terms of the `Artistic Licence 2.0`_. Contents -------- These documents aim to get you started with |reclass|: .. toctree:: :maxdepth: 2 install concepts operations usage refs manpage configfile salt ansible puppet hacking todo changelog About the name -------------- "reclass" stands for **r**\ ecursive **e**\ xternal node **class**\ ifier, which is somewhat of a misnomer. I chose the name very early on, based on the recursive nature of the data merging. However, to the user, a better paradigm would be "hierarchical", as s/he does not and should not care too much about the implementation internals. By the time that I realised this, unfortunately, `Hiera`_ (Puppet-specific) had already occupied this prefix. Oh well. Once you start using |reclass|, you'll think recursively as well as hierarchically at the same time. It's really quite simple. .. Indices and tables ================== * :ref:`genindex` * :ref:`modindex` * :ref:`search` .. include:: extrefs.inc .. include:: substs.inc reclass-1.2.2/doc/source/install.rst000066400000000000000000000060061225711215500174200ustar00rootroot00000000000000============ Installation ============ For Debian users (including Ubuntu) ----------------------------------- |reclass| has been `packaged for Debian`_. To use it, just install it with APT:: $ apt-get install reclass [reclass-doc] .. _packaged for Debian: http://packages.debian.org/search?keywords=reclass For ArchLinux users ------------------- |reclass| is `available for ArchLinux`_, thanks to Niels Abspoel. Dowload the tarball_ from ``aur`` or ``yaourt``:: $ yaourt -S reclass or:: $ tar xvzf reclass-git.tar.gz $ cd reclass-git; makepkg $ sudo pacman -U reclass-git-.tar.gz .. _available for ArchLinux: https://aur.archlinux.org/packages/reclass-git/ .. _tarball: https://aur.archlinux.org/packages/re/reclass-git/reclass-git.tar.gz Other distributions ------------------- Developers of other distributions are cordially invited to package |reclass| themselves and `write to the mailing list `_ to have details included here. Or send a patch! From source ----------- |reclass| is currently maintained `on Github `_, so to obtain the source, run:: $ git clone https://github.com/madduck/reclass.git or:: $ git clone ssh://git@github.com:madduck/reclass.git If you want a tarball, please `obtain it from the Debian archive`_. .. _obtain it from the Debian archive: http://http.debian.net/debian/pool/main/r/reclass/ Before you can use |reclass|, you need to install it into a place where Python can find it. The following step should install the package to ``/usr/local``:: $ python setup.py install If you want to install to a different location, use --prefix like so:: $ python setup.py install --prefix=/opt/local .. todo:: These will install the ``reclass-salt`` and ``reclass-ansible`` adapters to ``$prefix/bin``, but they should go to ``$prefix/share/reclass``. How can setup.py be told to do so? It would be better for consistency if this was done "upstream", rather than fixed by the distros. Just make sure that the destination is in the Python module search path, which you can check like this:: $ python -c 'import sys; print sys.path' More options can be found in the output of :: $ python setup.py install --help $ python setup.py --help $ python setup.py --help-commands $ python setup.py --help [cmd] If you just want to run |reclass| from source, e.g. because you are going to be making and testing changes, install it in "development mode":: $ python setup.py develop To uninstall (the rm call is necessary due to `a bug in setuptools`_):: $ python setup.py develop --uninstall $ rm /usr/local/bin/reclass* `Uninstallation currently isn't possible`_ for packages installed to /usr/local as per the above method, unfortunately. The following should do:: $ rm -r /usr/local/lib/python*/dist-packages/reclass* /usr/local/bin/reclass* .. _a bug in setuptools: http://bugs.debian.org/714960 .. _Uninstallation currently isn't possible: http://bugs.python.org/issue4673 .. include:: substs.inc reclass-1.2.2/doc/source/intro.inc000066400000000000000000000026071225711215500170510ustar00rootroot00000000000000|reclass| is an "external node classifier" (ENC) as can be used with automation tools, such as `Puppet`_, `Salt`_, and `Ansible`_. It is also a stand-alone tool for merging data sources recursively. The purpose of an ENC is to allow a system administrator to maintain an inventory of nodes to be managed, completely separately from the configuration of the automation tool. Usually, the external node classifier completely replaces the tool-specific inventory (such as ``site.pp`` for Puppet, ``ext_pillar``/``master_tops`` for Salt, or ``/etc/ansible/hosts``). With respect to the configuration management tool, the ENC then fulfills two jobs: - it provides information about groups of nodes and group memberships - it gives access to node-specific information, such as variables |reclass| allows you to define your nodes through class inheritance, while always able to override details further up the tree (i.e. in more specific nodes). Think of classes as feature sets, as commonalities between nodes, or as tags. Add to that the ability to nest classes (multiple inheritance is allowed, well-defined, and encouraged), and you can assemble your infrastructure from smaller bits, eliminating duplication and exposing all important parameters to a single location, logically organised. And if that isn't enough, |reclass| lets you reference other parameters in the very hierarchy you are currently assembling. reclass-1.2.2/doc/source/manpage.rst000066400000000000000000000027011225711215500173600ustar00rootroot00000000000000=============== reclass manpage =============== Synopsis -------- | |reclass| --help | |reclass| *[options]* --inventory | |reclass| *[options]* --nodeinfo=NODENAME Description ----------- .. include:: intro.inc |reclass| will be used indirectly through adapters most of the time. However, there exists a command-line interface that allows querying the database. This manual page describes this interface. Options ------- Please see the output of ``reclass --help`` for the default values of these options: Database options '''''''''''''''' -s, --storage-type The type of storage backend to use -b, --inventory-base-uri The base URI to prepend to nodes and classes -u, --nodes-uri The URI to the nodes storage -c, --classes-uri The URI to the classes storage Output options '''''''''''''' -o, --output The output format to use (yaml or json) -y, --pretty-print Try to make the output prettier Modes ''''' -i, --inventory Output the entire inventory -n, --nodeinfo Output information for a specific node Information ''''''''''' -h, --help Help output --version Display version number See also -------- Please visit http://reclass.pantsfullofunix.net/ for more information about |reclass|. The documentation is also available from the ``./doc`` subtree in the source checkout, or from ``/usr/share/doc/reclass-doc``. .. include:: substs.inc .. include:: extrefs.inc reclass-1.2.2/doc/source/operations.rst000066400000000000000000000140101225711215500201270ustar00rootroot00000000000000================== reclass operations ================== YAML FS storage --------------- While |reclass| has been built to support different storage backends through plugins, currently only the ``yaml_fs`` storage backend exists. This is a very simple, yet powerful, YAML-based backend, using flat files on the filesystem (as suggested by the ``_fs`` postfix). ``yaml_fs`` works with two directories, one for node definitions, and another for class definitions. The two directories must not be the same, nor can one be a parent of the other. Files in those directories are YAML-files, specifying key-value pairs. The following three keys are read by |reclass|: ============ ================================================================ Key Description ============ ================================================================ classes a list of parent classes appliations a list of applications to append to the applications defined by ancestors. If an application name starts with ``~``, it would remove this application from the list, if it had already been added — but it does not prevent a future addition. E.g. ``~firewalled`` parameters key-value pairs to set defaults in class definitions, override existing data, or provide node-specific information in node specifications. \ By convention, parameters corresponding to an application should be provided as subkey-value pairs, keyed by the name of the application, e.g.:: applications: - ssh.server parameters: ssh.server: permit_root_login: no ============ ================================================================ Nodes may be defined in subdirectories. However, node names (filename) must be unique across all subdirectories, and |reclass| will exit with an error if a node is defined multiple times. Subdirectories therefore really only exist for the administrator's local data structuring. They may be used in mappings (see below) to tag additional classes onto nodes. Data merging ------------ |reclass| has two modes of operation: node information retrieval and inventory listing. The second is really just a loop of the first across all defined nodes, and needs not be further described. When retrieving information about a node, |reclass| first obtains the node definition from the storage backend. Then, it iterates the list of classes defined for the node and recursively asks the storage backend for each class definition (unless already cached). Next, |reclass| recursively descends each class, looking at the classes it defines, and so on, until a leaf node is reached, i.e. a class that references no other classes. Now, the merging starts. At every step, the list of applications and the set of parameters at each level is merged into what has been accumulated so far. Merging of parameters is done "deeply", meaning that lists and dictionaries are extended (recursively), rather than replaced. However, a scalar value *does* overwrite a dictionary or list value. While the scalar could be appended to an existing list, there is sane default assumption in the context of a dictionary, so this behaviour seems the most logical. After all classes (and the classes they reference) have been visited, |reclass| finally merges the applications list and parameters defined for the node into what has been accumulated during the processing of the classes, and returns the final result. Wildcard/Regexp mappings ------------------------ Using the :doc:`configuration file `, it is also possible to provide a list mappings between node names and classes. For instance:: class_mappings: - \* default - /^www\d+/ webserver - \*.ch hosted@switzerland another_class_to_show_that_it_can_take_lists This will assign the ``default`` class to all nodes (make sure to escape a leading asterisk (\*) to keep YAML happy), ``webserver`` to all nodes named ``www1`` or ``www999``, and ``hosted-in-switzerland`` to all nodes whose names end with ``.ch`` (again, note the escaped leading asterisk). Multiple classes can be assigned to each mapping by providing a space-separated list (class names cannot contain spaces anyway). .. warning:: The class mappings do not really belong in the configuration file, as they are data, not configuration inmformation. Therefore, they are likely going to move elsewhere, but I have not quite figured out to where. Most likely, there will be an additional file, specified in the configuration file, which then lists the mappings. Note that mappings are not designed to replace node definitions. Mappings can be used to pre-populate the classes of existing nodes, but you still need to define all nodes (and if only to allow them to be enumerated for the inventory). The mapped classes can also contain backreferences when regular expressions are used, although they need to be escaped, e.g.:: class_mappings: - /\.(\S+)$/ tld-\\1 Furthermore, since the outer slashes ('/') are used to "quote" the regular expression, *any* slashes within the regular expression must be escaped. For instance, the following class mapping assigns a ``subdir-X`` class to all nodes that are defined in a subdirectory (using yaml_fs):: class_mappings: - /^([^\/]+)\// subdir-\\1 Parameter interpolation ------------------------ Parameters may reference each other, including deep references, e.g.:: parameters: location: Munich, Germany motd: header: This node sits in ${location} for_demonstration: ${motd:header} dict_reference: ${motd} After merging and interpolation, which happens automatically inside the storage modules, the ``for_demonstration`` parameter will have a value of "This node sits in Munich, Germany". Types are preserved if the value contains nothing but a reference. Hence, the value of ``dict_reference`` will actually be a dictionary. You should now be ready to :doc:`use reclass `! .. include:: substs.inc reclass-1.2.2/doc/source/puppet.rst000066400000000000000000000010061225711215500172620ustar00rootroot00000000000000========================= Using reclass with Puppet ========================= .. todo:: The adapter between |reclass| and `Puppet`_ has not actually been written, since I rage-quit using Puppet before the rewrite of |reclass|. It should be trivial to do, and if you need it or are interested in working on it, and you require assistance, please get in touch with me `on the mailing list `_. Else just send the patch! .. include:: extrefs.inc .. include:: substs.inc reclass-1.2.2/doc/source/refs.rst000066400000000000000000000015551225711215500167150ustar00rootroot00000000000000=================== External references =================== * I gave `a talk about reclass`__ at `DebConf13`_, which has been recorded: * `Slides`__ * Video recording: `high quality (ogv)`__ | `high quality (webm)`__ | `low(er) quality (ogv)`__ __ http://penta.debconf.org/dc13_schedule/events/1048.en.html __ http://annex.debconf.org/debconf-share/debconf13/slides/reclass.pdf __ http://meetings-archive.debian.net/pub/debian-meetings/2013/debconf13/high/1048_Recursive_node_classification_for_system_automation.ogv __ http://meetings-archive.debian.net/pub/debian-meetings/2013/debconf13/webm-high/1048_Recursive_node_classification_for_system_automation.webm __ http://meetings-archive.debian.net/pub/debian-meetings/2013/debconf13/low/1048_Recursive_node_classification_for_system_automation.ogv .. _DebConf13: http://debconf13.debconf.org .. include:: substs.inc reclass-1.2.2/doc/source/salt.rst000066400000000000000000000156701225711215500167240ustar00rootroot00000000000000======================= Using reclass with Salt ======================= .. warning:: You need Salt 0.17 to use `reclass`, as older versions do not include the `reclass` adapter. You could use the ``cmd_yaml`` adapters, but at least for ``ext_pillar``, they are currently not useable, as they `do not export the minion ID to the command they run`_. .. _do not export the minion ID to the command they run: https://github.com/saltstack/salt/issues/2276 Quick start ----------- The following steps should get you up and running quickly with |reclass| and `Salt`_. You will need to decide for yourself where to put your |reclass| inventory. This can be your first ``base`` ``file_root`` (the default), or it could be ``/etc/reclass``, or ``/srv/salt``. The following shall assume the latter. Or you can also just look into ``./examples/salt`` of your |reclass| checkout (``/usr/share/doc/examples/salt`` on Debian-systems), where the following steps have already been prepared. /…/reclass refers to the location of your |reclass| checkout. .. todo:: With |reclass| now in Debian, as well as installable from source, the following should be checked for path consistency… #. Complete the installation steps described in the :doc:`installation section `. Alternatively, you can also tell Salt via the master config file where to look for |reclass|, but then you won't be able to interact with |reclass| through the command line. #. Copy the two directories ``nodes`` and ``classes`` from the example subdirectory in the |reclass| checkout to e.g. ``/srv/salt``. It's handy to symlink |reclass|' Salt adapter itself to that directory:: $ ln -s /usr/share/reclass/reclass-salt /srv/salt/states/reclass As you can now just inspect the data right there from the command line:: $ ./reclass --top If you don't want to do this, you can also let |reclass| know where to look for the inventory with the following contents in ``$HOME/reclass-config.yml``:: storage_type: yaml_fs base_inventory_uri: /srv/reclass Or you can reuse the first entry of ``file_roots`` under ``base`` in the Salt master config. Note that ``yaml_fs`` is currently the only supported ``storage_type``, and it's the default if you don't set it. #. Check out your inventory by invoking :: $ reclass-salt --top which should return all the information about all defined nodes, which is only ``localhost`` in the example. This is essentially the same information that you would keep in your ``top.sls`` file. If you symlinked the script to your inventory base directory, use :: $ ./reclass --top #. See the pillar information for ``localhost``:: $ reclass-salt --pillar localhost #. Now add |reclass| to ``/etc/salt/master``, like so:: reclass: &reclass inventory_base_uri: /srv/salt reclass_source_path: ~/code/reclass master_tops: […] reclass: *reclass ext_pillar: - reclass: *reclass .. warning:: When using ``ext_pillar`` and/or ``master_tops``, you should make sure that your ``file_roots`` paths do not contain a ``top.sls`` file. Even though they ought to be able to coexist, there are a few sharp edges around at the moment, so beware! If you did not install |reclass| (but you are running it from source), you can either specify the source path like above, or you can add it to ``PYTHONPATH`` before invoking the Salt master, to ensure that Python can find it:: PYTHONPATH=/…/reclass /etc/init.d/salt-master restart #. Provided that you have set up ``localhost`` as a Salt minion, the following commands should now return the same data as above, but processed through salt:: $ salt localhost pillar.items # shows just the parameters $ salt localhost state.show_top # shows only the states (applications) Alternatively, if you don't have the Salt minion running yet:: $ salt-call pillar.items # shows just the parameters $ salt-call state.show_top # shows only the states (applications) #. You can also invoke |reclass| directly, which gives a slightly different view onto the same data, i.e. before it has been adapted for Salt:: $ reclass --inventory $ reclass --nodeinfo localhost Configuration file and master configuration ------------------------------------------- Even though the Salt adapter of |reclass| looks for and reads the :doc:`configuration file `, a better means to pass information to the adapter is via Salt's master configuration file, as shown above. Not all configuration options can be passed this way (e.g. ``output`` is hardcoded to YAML, which Salt uses), but it *is* possible to specify :doc:`class mappings ` next to all the storage-specific options. .. warning:: The Salt CLI adapter does *not* read Salt's master configuration, so if you are calling ``reclass-salt`` from the command-line (the CLI exists for debugging purposes, mainly), be aware that it will be run in a different environment than when Salt queries reclass directly. Integration with Salt --------------------- |reclass| hooks into Salt at two different points: ``master_tops`` and ``ext_pillar``. For both, Salt provides plugins. These plugins need to know where to find |reclass|, so if |reclass| is not properly installed (but you are running it from source), make sure to export ``PYTHONPATH`` accordingly before you start your Salt master, or specify the path in the master configuration file, as show above. Salt has no concept of "nodes", "applications", "parameters", and "classes". Therefore it is necessary to explain how those correspond to Salt. Crudely, the following mapping exists: ================= ================ |reclass| concept Salt terminology ================= ================ nodes hosts classes (none) [#nodegroups]_ applications states parameters pillar ================= ================ .. [#nodegroups] See `Salt issue #5787`_ for steps into the direction of letting |reclass| provide nodegroup information. .. _Salt issue #5787: https://github.com/saltstack/salt/issues/5787 Whatever applications you define for a node will become states applicable to a host. If those applications are added via ancestor classes, then that's fine, but currently, Salt does not do anything with the classes ancestry. Similarly, all parameters that are collected and merged eventually end up in the pillar data of a specific node. However, the pillar data of a node include all the information about classes and applications, so you could theoretically use them to target your Salt calls at groups of nodes defined in the |reclass| inventory, e.g. :: salt -I __reclass__:classes:salt_minion test.ping Unfortunately, this does not work yet, please stay tuned, and let me know if you figure out a way. `Salt issue #5787`_ is also of relevance. .. include:: substs.inc .. include:: extrefs.inc reclass-1.2.2/doc/source/substs.inc000066400000000000000000000000431225711215500172310ustar00rootroot00000000000000.. |reclass| replace:: **reclass** reclass-1.2.2/doc/source/todo.rst000066400000000000000000000054621225711215500167240ustar00rootroot00000000000000================== reclass to-do list ================== Common set of classes --------------------- A lot of the classes I have set up during the various stages of development of |reclass| are generic. It would probably be sensible to make them available as part of |reclass|, to give people a common baseline to work from, and to ensure a certain level of consistency between users. Testing framework ----------------- There is rudimentary testing in place, but it's inconsistent. I got side-tracked into discussions about the philosphy of mocking objects. This could all be fixed and unified. Also, storage, outputters, CLI and adapters have absolutely no tests yet… Configurable file extension --------------------------- Right now, ``.yml`` is hard-coded. This could be exported to the configuration file, or even given as a list, so that ``.yml`` and ``.yaml`` can both be used. Verbosity, debugging -------------------- Verbose output and debug logging would be a very useful addition to help people understand what's going on, where data are being changed/merged, and to help solve problems. Data from CMS for interpolation ------------------------------- Depending on the CMS in question, it would be nice if |reclass| had access to the host-specific data (facts, grains, etc.) and could use those in parameter interpolation. I can imagine this working for Salt, where the ``grains`` dictionary (and results from previous external node classifiers) is made available to the external node classifiers, but I am not convinced this will be possible in Ansible and Puppet. Ideally, |reclass| could unify the interface so that even templates can be shared between the various CMS. Node environments ----------------- At least Salt and Puppet support the notion of "environments", but the Salt adapter just puts everything into the "base" environment at the moment. Part of the reason that multiple environments aren't (yet) supported is because I don't see the use-case (anymore) with |reclass|. If you still see a use-case, then please help me understand it and let's figure out a good way to introduce this concept into |reclass|. Membership information ---------------------- It would be nice if |reclass| could provide e.g. the Nagios master node with a list of clients that define it as their master. That would short-circuit Puppet's ``storeconfigs`` and Salt's ``mine``. Configuration file lookup improvements -------------------------------------- Right now, the adapters and the CLI look for the :doc:`configuration file ` in a fixed set of locations. On of those derives from ``OPT_INVENTORY_BASE_URI``, the default inventory base URI (``/etc/reclass``). This should probably be updated in case the user changes the URI. Furthermore, ``$CWD`` and ``~`` might not make a lot of sense in all use-cases. .. include:: substs.inc reclass-1.2.2/doc/source/usage.rst000066400000000000000000000035371225711215500170640ustar00rootroot00000000000000============= Using reclass ============= .. todo:: With |reclass| now in Debian, as well as installable from source, the following should be checked for path consistency… For information on how to use |reclass| directly, call ``reclass --help`` and study the output, or have a look at its :doc:`manual page `. The three options, ``--inventory-base-uri``, ``--nodes-uri``, and ``--classes-uri`` together specify the location of the inventory. If the base URI is specified, then it is prepended to the other two URIs, unless they are absolute URIs. If these two URIs are not specified, they default to ``nodes`` and ``classes``. Therefore, if your inventory is in ``/etc/reclass/nodes`` and ``/etc/reclass/classes``, all you need to specify is the base URI as ``/etc/reclass`` — which is actually the default (specified in ``reclass/defaults.py``). If you've installed |reclass| from source as per the :doc:`installation instructions `, try to run it from the source directory like this:: $ reclass -b examples/ --inventory $ reclass -b examples/ --node localhost This will make it use the data from ``examples/nodes`` and ``examples/classes``, and you can surely make your own way from here. On Debian-systems, use the following:: $ reclass -b /usr/share/doc/reclass/examples/ --inventory $ reclass -b /usr/share/doc/reclass/examples/ --node localhost More commonly, however, use of |reclass| will happen indirectly, and through so-called adapters. The job of an adapter is to translate between different invocation paradigms, provide a sane set of default options, and massage the data from |reclass| into the format expected by the automation tool in use. Please have a look at the respective README files for these adapters, i.e. for :doc:`Salt `, for :doc:`Ansible `, and for :doc:`Puppet `. .. include:: substs.inc reclass-1.2.2/examples/000077500000000000000000000000001225711215500147675ustar00rootroot00000000000000reclass-1.2.2/examples/ansible/000077500000000000000000000000001225711215500164045ustar00rootroot00000000000000reclass-1.2.2/examples/ansible/hosts000077500000000000000000000001401225711215500174650ustar00rootroot00000000000000#!/bin/sh cd ../../ PYTHONPATH="`pwd`:$PYTHONPATH" exec python reclass/adapters/ansible.py "$@" reclass-1.2.2/examples/ansible/reclass-config.yml000066400000000000000000000000271225711215500220250ustar00rootroot00000000000000inventory_base_uri: .. reclass-1.2.2/examples/ansible/test.yml000066400000000000000000000002031225711215500201010ustar00rootroot00000000000000- name: Test playbook against all test hosts hosts: test_hosts tasks: - name: Greet the world debug: msg='$greeting' reclass-1.2.2/examples/classes/000077500000000000000000000000001225711215500164245ustar00rootroot00000000000000reclass-1.2.2/examples/classes/basenode.yml000066400000000000000000000000441225711215500207250ustar00rootroot00000000000000parameters: inventory_db: reclass reclass-1.2.2/examples/classes/mysite.yml000066400000000000000000000001071225711215500204570ustar00rootroot00000000000000applications: - test parameters: realm: mysite greeting: Servus! reclass-1.2.2/examples/classes/unixnode.yml000066400000000000000000000001231225711215500207740ustar00rootroot00000000000000classes: - basenode applications: - motd parameters: greeting: Hello, world! reclass-1.2.2/examples/nodes/000077500000000000000000000000001225711215500160775ustar00rootroot00000000000000reclass-1.2.2/examples/nodes/localhost.yml000066400000000000000000000000761225711215500206150ustar00rootroot00000000000000classes: - unixnode - mysite parameters: colour: yellow reclass-1.2.2/examples/salt/000077500000000000000000000000001225711215500157325ustar00rootroot00000000000000reclass-1.2.2/examples/salt/reclass000077500000000000000000000001351225711215500173130ustar00rootroot00000000000000#!/bin/sh cd ../../ PYTHONPATH="`pwd`:$PYTHONPATH" exec python reclass/adapters/salt.py "$@" reclass-1.2.2/examples/salt/reclass-config.yml000066400000000000000000000000271225711215500213530ustar00rootroot00000000000000inventory_base_uri: .. reclass-1.2.2/reclass.py000077500000000000000000000004251225711215500151630ustar00rootroot00000000000000#!/usr/bin/env python # -*- coding: utf-8 -*- # # This file is part of reclass (http://github.com/madduck/reclass) # # Copyright © 2007–13 martin f. krafft # Released under the terms of the Artistic Licence 2.0 # import reclass.cli reclass.cli.main() reclass-1.2.2/reclass/000077500000000000000000000000001225711215500146055ustar00rootroot00000000000000reclass-1.2.2/reclass/__init__.py000066400000000000000000000027441225711215500167250ustar00rootroot00000000000000# # -*- coding: utf-8 -*- # # This file is part of reclass (http://github.com/madduck/reclass) # # Copyright © 2007–13 martin f. krafft # Released under the terms of the Artistic Licence 2.0 # from output import OutputLoader from storage import StorageBackendLoader from config import path_mangler def get_storage(storage_type, nodes_uri, classes_uri, class_mappings): storage_class = StorageBackendLoader(storage_type).load() return storage_class(nodes_uri, classes_uri, class_mappings) def get_nodeinfo(storage_type, inventory_base_uri, nodes_uri, classes_uri, nodename, class_mappings): nodes_uri, classes_uri = path_mangler(inventory_base_uri, nodes_uri, classes_uri) storage = get_storage(storage_type, nodes_uri, classes_uri, class_mappings) # TODO: template interpolation return storage.nodeinfo(nodename) def get_inventory(storage_type, inventory_base_uri, nodes_uri, classes_uri, class_mappings): nodes_uri, classes_uri = path_mangler(inventory_base_uri, nodes_uri, classes_uri) storage = get_storage(storage_type, nodes_uri, classes_uri, class_mappings) return storage.inventory() def output(data, fmt, pretty_print=False): output_class = OutputLoader(fmt).load() outputter = output_class() return outputter.dump(data, pretty_print=pretty_print) reclass-1.2.2/reclass/adapters/000077500000000000000000000000001225711215500164105ustar00rootroot00000000000000reclass-1.2.2/reclass/adapters/__init__.py000077500000000000000000000003321225711215500205220ustar00rootroot00000000000000# # -*- coding: utf-8 -*- # # This file is part of reclass (http://github.com/madduck/reclass) # # Copyright © 2007–13 martin f. krafft # Released under the terms of the Artistic Licence 2.0 # reclass-1.2.2/reclass/adapters/ansible.py000077500000000000000000000076061225711215500204130ustar00rootroot00000000000000#!/usr/bin/env python # -*- coding: utf-8 -*- # # This file is part of reclass (http://github.com/madduck/reclass) # # IMPORTANT NOTICE: I was kicked out of the Ansible community, and therefore # I have no interest in developing this adapter anymore. If you use it and # have changes, I will take your patch. # # Copyright © 2007–13 martin f. krafft # Released under the terms of the Artistic Licence 2.0 # import os, sys, posix, optparse from reclass import get_nodeinfo, get_inventory, output from reclass.errors import ReclassException from reclass.config import find_and_read_configfile, get_options from reclass.version import * from reclass.constants import MODE_NODEINFO def cli(): try: # this adapter has to be symlinked to ansible_dir, so we can use this # information to initialise the inventory_base_uri to ansible_dir: ansible_dir = os.path.abspath(os.path.dirname(sys.argv[0])) defaults = {'inventory_base_uri': ansible_dir, 'pretty_print' : True, 'output' : 'json', 'applications_postfix': '_hosts' } defaults.update(find_and_read_configfile()) def add_ansible_options_group(parser, defaults): group = optparse.OptionGroup(parser, 'Ansible options', 'Ansible-specific options') group.add_option('--applications-postfix', dest='applications_postfix', default=defaults.get('applications_postfix'), help='postfix to append to applications to '\ 'turn them into groups') parser.add_option_group(group) options = get_options(RECLASS_NAME, VERSION, DESCRIPTION, inventory_shortopt='-l', inventory_longopt='--list', inventory_help='output the inventory', nodeinfo_shortopt='-t', nodeinfo_longopt='--host', nodeinfo_dest='hostname', nodeinfo_help='output host_vars for the given host', add_options_cb=add_ansible_options_group, defaults=defaults) class_mappings = defaults.get('class_mappings') if options.mode == MODE_NODEINFO: data = get_nodeinfo(options.storage_type, options.inventory_base_uri, options.nodes_uri, options.classes_uri, options.hostname, class_mappings) # Massage and shift the data like Ansible wants it data['parameters']['__reclass__'] = data['__reclass__'] for i in ('classes', 'applications'): data['parameters']['__reclass__'][i] = data[i] data = data['parameters'] else: data = get_inventory(options.storage_type, options.inventory_base_uri, options.nodes_uri, options.classes_uri, class_mappings) # Ansible inventory is only the list of groups. Groups are the set # of classes plus the set of applications with the postfix added: groups = data['classes'] apps = data['applications'] if options.applications_postfix: postfix = options.applications_postfix groups.update([(k + postfix, v) for k,v in apps.iteritems()]) else: groups.update(apps) data = groups print output(data, options.output, options.pretty_print) except ReclassException, e: e.exit_with_message(sys.stderr) sys.exit(posix.EX_OK) if __name__ == '__main__': cli() reclass-1.2.2/reclass/adapters/salt.py000077500000000000000000000076761225711215500177500ustar00rootroot00000000000000#!/usr/bin/env python # -*- coding: utf-8 -*- # # This file is part of reclass (http://github.com/madduck/reclass) # # Copyright © 2007–13 martin f. krafft # Released under the terms of the Artistic Licence 2.0 # import os, sys, posix from reclass import get_nodeinfo, get_inventory, output from reclass.errors import ReclassException from reclass.config import find_and_read_configfile, get_options from reclass.constants import MODE_NODEINFO from reclass.defaults import * from reclass.version import * def ext_pillar(minion_id, pillar, storage_type=OPT_STORAGE_TYPE, inventory_base_uri=OPT_INVENTORY_BASE_URI, nodes_uri=OPT_NODES_URI, classes_uri=OPT_CLASSES_URI, class_mappings=None): data = get_nodeinfo(storage_type, inventory_base_uri, nodes_uri, classes_uri, minion_id, class_mappings) params = data.get('parameters', {}) params['__reclass__'] = {} params['__reclass__']['applications'] = data['applications'] params['__reclass__']['classes'] = data['classes'] return params def top(minion_id, storage_type=OPT_STORAGE_TYPE, inventory_base_uri=OPT_INVENTORY_BASE_URI, nodes_uri=OPT_NODES_URI, classes_uri=OPT_CLASSES_URI, class_mappings=None): env = 'base' # TODO: node environments # if the minion_id is not None, then return just the applications for the # specific minion, otherwise return the entire top data (which we need for # CLI invocations of the adapter): if minion_id is not None: data = get_nodeinfo(storage_type, inventory_base_uri, nodes_uri, classes_uri, minion_id, class_mappings) applications = data.get('applications', []) return {env: applications} else: data = get_inventory(storage_type, inventory_base_uri, nodes_uri, classes_uri, class_mappings) nodes = {} for node_id, node_data in data['nodes'].iteritems(): nodes[node_id] = node_data['applications'] return {env: nodes} def cli(): try: inventory_dir = os.path.abspath(os.path.dirname(sys.argv[0])) defaults = {'pretty_print' : True, 'output' : 'yaml', 'inventory_base_uri': inventory_dir } defaults.update(find_and_read_configfile()) options = get_options(RECLASS_NAME, VERSION, DESCRIPTION, inventory_shortopt='-t', inventory_longopt='--top', inventory_help='output the state tops (inventory)', nodeinfo_shortopt='-p', nodeinfo_longopt='--pillar', nodeinfo_dest='nodename', nodeinfo_help='output pillar data for a specific node', defaults=defaults) class_mappings = defaults.get('class_mappings') if options.mode == MODE_NODEINFO: data = ext_pillar(options.nodename, {}, storage_type=options.storage_type, inventory_base_uri=options.inventory_base_uri, nodes_uri=options.nodes_uri, classes_uri=options.classes_uri, class_mappings=class_mappings) else: data = top(minion_id=None, storage_type=options.storage_type, inventory_base_uri=options.inventory_base_uri, nodes_uri=options.nodes_uri, classes_uri=options.classes_uri, class_mappings=class_mappings) print output(data, options.output, options.pretty_print) except ReclassException, e: e.exit_with_message(sys.stderr) sys.exit(posix.EX_OK) if __name__ == '__main__': cli() reclass-1.2.2/reclass/cli.py000066400000000000000000000031321225711215500157250ustar00rootroot00000000000000# # -*- coding: utf-8 -*- # # This file is part of reclass (http://github.com/madduck/reclass) # # Copyright © 2007–13 martin f. krafft # Released under the terms of the Artistic Licence 2.0 # import sys, os, posix from reclass import get_nodeinfo, get_inventory, output from reclass.config import find_and_read_configfile, get_options from reclass.errors import ReclassException from reclass.defaults import * from reclass.constants import MODE_NODEINFO from reclass.version import * def main(): try: defaults = {'pretty_print' : OPT_PRETTY_PRINT, 'output' : OPT_OUTPUT } defaults.update(find_and_read_configfile()) options = get_options(RECLASS_NAME, VERSION, DESCRIPTION, defaults=defaults) class_mappings = defaults.get('class_mappings') if options.mode == MODE_NODEINFO: data = get_nodeinfo(options.storage_type, options.inventory_base_uri, options.nodes_uri, options.classes_uri, options.nodename, class_mappings) else: data = get_inventory(options.storage_type, options.inventory_base_uri, options.nodes_uri, options.classes_uri, class_mappings) print output(data, options.output, options.pretty_print) except ReclassException, e: e.exit_with_message(sys.stderr) sys.exit(posix.EX_OK) if __name__ == '__main__': main() reclass-1.2.2/reclass/config.py000066400000000000000000000205141225711215500164260ustar00rootroot00000000000000# # -*- coding: utf-8 -*- # # This file is part of reclass (http://github.com/madduck/reclass) # # Copyright © 2007–13 martin f. krafft # Released under the terms of the Artistic Licence 2.0 # import yaml, os, optparse, posix, sys import errors from defaults import * from constants import MODE_NODEINFO, MODE_INVENTORY def make_db_options_group(parser, defaults={}): ret = optparse.OptionGroup(parser, 'Database options', 'Configure from where {0} collects data'.format(parser.prog)) ret.add_option('-s', '--storage-type', dest='storage_type', default=defaults.get('storage_type', OPT_STORAGE_TYPE), help='the type of storage backend to use [%default]') ret.add_option('-b', '--inventory-base-uri', dest='inventory_base_uri', default=defaults.get('inventory_base_uri', OPT_INVENTORY_BASE_URI), help='the base URI to prepend to nodes and classes [%default]'), ret.add_option('-u', '--nodes-uri', dest='nodes_uri', default=defaults.get('nodes_uri', OPT_NODES_URI), help='the URI to the nodes storage [%default]'), ret.add_option('-c', '--classes-uri', dest='classes_uri', default=defaults.get('classes_uri', OPT_CLASSES_URI), help='the URI to the classes storage [%default]') return ret def make_output_options_group(parser, defaults={}): ret = optparse.OptionGroup(parser, 'Output options', 'Configure the way {0} prints data'.format(parser.prog)) ret.add_option('-o', '--output', dest='output', default=defaults.get('output', OPT_OUTPUT), help='output format (yaml or json) [%default]') ret.add_option('-y', '--pretty-print', dest='pretty_print', action="store_true", default=defaults.get('pretty_print', OPT_PRETTY_PRINT), help='try to make the output prettier [%default]') return ret def make_modes_options_group(parser, inventory_shortopt, inventory_longopt, inventory_help, nodeinfo_shortopt, nodeinfo_longopt, nodeinfo_dest, nodeinfo_help): def _mode_checker_cb(option, opt_str, value, parser): if hasattr(parser.values, 'mode'): raise optparse.OptionValueError('Cannot specify multiple modes') if option == parser.get_option(nodeinfo_longopt): setattr(parser.values, 'mode', MODE_NODEINFO) setattr(parser.values, nodeinfo_dest, value) else: setattr(parser.values, 'mode', MODE_INVENTORY) setattr(parser.values, nodeinfo_dest, None) ret = optparse.OptionGroup(parser, 'Modes', 'Specify one of these to determine what to do.') ret.add_option(inventory_shortopt, inventory_longopt, action='callback', callback=_mode_checker_cb, help=inventory_help) ret.add_option(nodeinfo_shortopt, nodeinfo_longopt, default=None, dest=nodeinfo_dest, type='string', action='callback', callback=_mode_checker_cb, help=nodeinfo_help) return ret def make_parser_and_checker(name, version, description, inventory_shortopt='-i', inventory_longopt='--inventory', inventory_help='output the entire inventory', nodeinfo_shortopt='-n', nodeinfo_longopt='--nodeinfo', nodeinfo_dest='nodename', nodeinfo_help='output information for a specific node', add_options_cb=None, defaults={}): parser = optparse.OptionParser(version=version) parser.prog = name parser.version = version parser.description = description.capitalize() parser.usage = '%prog [options] ( {0} | {1} {2} )'.format(inventory_longopt, nodeinfo_longopt, nodeinfo_dest.upper()) parser.epilog = 'Exactly one mode has to be specified.' db_group = make_db_options_group(parser, defaults) parser.add_option_group(db_group) output_group = make_output_options_group(parser, defaults) parser.add_option_group(output_group) if callable(add_options_cb): add_options_cb(parser, defaults) modes_group = make_modes_options_group(parser, inventory_shortopt, inventory_longopt, inventory_help, nodeinfo_shortopt, nodeinfo_longopt, nodeinfo_dest, nodeinfo_help) parser.add_option_group(modes_group) def option_checker(options, args): if len(args) > 0: parser.error('No arguments allowed') elif not hasattr(options, 'mode') \ or options.mode not in (MODE_NODEINFO, MODE_INVENTORY): parser.error('You need to specify exactly one mode '\ '({0} or {1})'.format(inventory_longopt, nodeinfo_longopt)) elif options.mode == MODE_NODEINFO \ and not getattr(options, nodeinfo_dest, None): parser.error('Mode {0} needs {1}'.format(nodeinfo_longopt, nodeinfo_dest.upper())) elif options.inventory_base_uri is None and options.nodes_uri is None: parser.error('Must specify --inventory-base-uri or --nodes-uri') elif options.inventory_base_uri is None and options.classes_uri is None: parser.error('Must specify --inventory-base-uri or --classes-uri') return parser, option_checker def get_options(name, version, description, inventory_shortopt='-i', inventory_longopt='--inventory', inventory_help='output the entire inventory', nodeinfo_shortopt='-n', nodeinfo_longopt='--nodeinfo', nodeinfo_dest='nodename', nodeinfo_help='output information for a specific node', add_options_cb=None, defaults={}): parser, checker = make_parser_and_checker(name, version, description, inventory_shortopt, inventory_longopt, inventory_help, nodeinfo_shortopt, nodeinfo_longopt, nodeinfo_dest, nodeinfo_help, add_options_cb, defaults=defaults) options, args = parser.parse_args() checker(options, args) return options def vvv(msg): #print >>sys.stderr, msg pass def find_and_read_configfile(filename=CONFIG_FILE_NAME, dirs=CONFIG_FILE_SEARCH_PATH): for d in dirs: f = os.path.join(d, filename) if os.access(f, os.R_OK): vvv('Using config file: {0}'.format(f)) return yaml.safe_load(file(f)) elif os.path.isfile(f): raise PermissionsError('cannot read %s' % f) return {} def path_mangler(inventory_base_uri, nodes_uri, classes_uri): if inventory_base_uri is None: # if inventory_base is not given, default to current directory inventory_base_uri = os.getcwd() nodes_uri = nodes_uri or 'nodes' classes_uri = classes_uri or 'classes' def _path_mangler_inner(path): ret = os.path.join(inventory_base_uri, path) ret = os.path.expanduser(ret) return os.path.abspath(ret) n, c = map(_path_mangler_inner, (nodes_uri, classes_uri)) if n == c: raise errors.DuplicateUriError(n, c) common = os.path.commonprefix((n, c)) if common == n or common == c: raise errors.UriOverlapError(n, c) return n, c reclass-1.2.2/reclass/constants.py000066400000000000000000000004301225711215500171700ustar00rootroot00000000000000# # -*- coding: utf-8 -*- # # This file is part of reclass (http://github.com/madduck/reclass) # # Copyright © 2007–13 martin f. krafft # Released under the terms of the Artistic Licence 2.0 # class MODE_NODEINFO: pass class MODE_INVENTORY: pass reclass-1.2.2/reclass/datatypes/000077500000000000000000000000001225711215500166035ustar00rootroot00000000000000reclass-1.2.2/reclass/datatypes/__init__.py000066400000000000000000000005301225711215500207120ustar00rootroot00000000000000# # -*- coding: utf-8 -*- # # This file is part of reclass (http://github.com/madduck/reclass) # # Copyright © 2007–13 martin f. krafft # Released under the terms of the Artistic Licence 2.0 # from applications import Applications from classes import Classes from entity import Entity from parameters import Parameters reclass-1.2.2/reclass/datatypes/applications.py000066400000000000000000000045161225711215500216510ustar00rootroot00000000000000# # -*- coding: utf-8 -*- # # This file is part of reclass (http://github.com/madduck/reclass) # # Copyright © 2007–13 martin f. krafft # Released under the terms of the Artistic Licence 2.0 # from classes import Classes class Applications(Classes): ''' Extends Classes with the possibility to let specially formatted items remove earlier occurences of the item. For instance, if the "negater" is '~', then "adding" an element "~foo" to a list causes a previous element "foo" to be removed. If no such element exists, nothing happens, but a reference of the negation is kept, in case the instance is later used to extend another instance, in which case the negations should apply to the instance to be extended. ''' DEFAULT_NEGATION_PREFIX = '~' def __init__(self, iterable=None, negation_prefix=DEFAULT_NEGATION_PREFIX): self._negation_prefix = negation_prefix self._offset = len(negation_prefix) self._negations = [] super(Applications, self).__init__(iterable) def _get_negation_prefix(self): return self._negation_prefix negation_prefix = property(_get_negation_prefix) def append_if_new(self, item): self._assert_is_string(item) if item.startswith(self._negation_prefix): item = item[self._offset:] self._negations.append(item) try: self._items.remove(item) except ValueError: pass else: super(Applications, self)._append_if_new(item) def merge_unique(self, iterable): if isinstance(iterable, self.__class__): # we might be extending ourselves to include negated applications, # in which case we need to remove our own content accordingly: for negation in iterable._negations: try: self._items.remove(negation) except ValueError: pass iterable = iterable.as_list() for i in iterable: self.append_if_new(i) def __repr__(self): contents = self._items + \ ['%s%s' % (self._negation_prefix, i) for i in self._negations] return "%s(%r, %r)" % (self.__class__.__name__, contents, self._negation_prefix) reclass-1.2.2/reclass/datatypes/classes.py000066400000000000000000000042661225711215500206220ustar00rootroot00000000000000# # -*- coding: utf-8 -*- # # This file is part of reclass (http://github.com/madduck/reclass) # # Copyright © 2007–13 martin f. krafft # Released under the terms of the Artistic Licence 2.0 # import types import os from reclass.errors import InvalidClassnameError INVALID_CHARACTERS_FOR_CLASSNAMES = ' ' + os.sep class Classes(object): ''' A very limited ordered set of strings with O(n) uniqueness constraints. It is neither a proper list or a proper set, on purpose, to keep things simple. ''' def __init__(self, iterable=None): self._items = [] if iterable is not None: self.merge_unique(iterable) def __len__(self): return len(self._items) def __eq__(self, rhs): if isinstance(rhs, list): return self._items == rhs else: try: return self._items == rhs._items except AttributeError as e: return False def __ne__(self, rhs): return not self.__eq__(rhs) def as_list(self): return self._items[:] def merge_unique(self, iterable): if isinstance(iterable, self.__class__): iterable = iterable.as_list() # Cannot just call list.extend here, as iterable's items might not # be unique by themselves, or in the context of self. for i in iterable: self.append_if_new(i) def _assert_is_string(self, item): if not isinstance(item, types.StringTypes): raise TypeError('%s instances can only contain strings, '\ 'not %s' % (self.__class__.__name__, type(item))) def _assert_valid_characters(self, item): for c in INVALID_CHARACTERS_FOR_CLASSNAMES: if c in item: raise InvalidClassnameError(c, item) def _append_if_new(self, item): if item not in self._items: self._items.append(item) def append_if_new(self, item): self._assert_is_string(item) self._assert_valid_characters(item) self._append_if_new(item) def __repr__(self): return '%s(%r)' % (self.__class__.__name__, self._items) reclass-1.2.2/reclass/datatypes/entity.py000066400000000000000000000065501225711215500204770ustar00rootroot00000000000000# # -*- coding: utf-8 -*- # # This file is part of reclass (http://github.com/madduck/reclass) # # Copyright © 2007–13 martin f. krafft # Released under the terms of the Artistic Licence 2.0 # from classes import Classes from applications import Applications from parameters import Parameters class Entity(object): ''' A collection of Classes, Parameters, and Applications, mainly as a wrapper for merging. The name and uri of an Entity will be updated to the name and uri of the Entity that is being merged. ''' def __init__(self, classes=None, applications=None, parameters=None, uri=None, name=None): if classes is None: classes = Classes() self._set_classes(classes) if applications is None: applications = Applications() self._set_applications(applications) if parameters is None: parameters = Parameters() self._set_parameters(parameters) self._uri = uri or '' self._name = name or '' name = property(lambda s: s._name) uri = property(lambda s: s._uri) classes = property(lambda s: s._classes) applications = property(lambda s: s._applications) parameters = property(lambda s: s._parameters) def _set_classes(self, classes): if not isinstance(classes, Classes): raise TypeError('Entity.classes cannot be set to '\ 'instance of type %s' % type(classes)) self._classes = classes def _set_applications(self, applications): if not isinstance(applications, Applications): raise TypeError('Entity.applications cannot be set to '\ 'instance of type %s' % type(applications)) self._applications = applications def _set_parameters(self, parameters): if not isinstance(parameters, Parameters): raise TypeError('Entity.parameters cannot be set to '\ 'instance of type %s' % type(parameters)) self._parameters = parameters def merge(self, other): self._classes.merge_unique(other._classes) self._applications.merge_unique(other._applications) self._parameters.merge(other._parameters) self._name = other.name self._uri = other.uri def interpolate(self): self._parameters.interpolate() def __eq__(self, other): return isinstance(other, type(self)) \ and self._applications == other._applications \ and self._classes == other._classes \ and self._parameters == other._parameters \ and self._name == other._name \ and self._uri == other._uri def __ne__(self, other): return not self.__eq__(other) def __repr__(self): return "%s(%r, %r, %r, uri=%r, name=%r)" % (self.__class__.__name__, self.classes, self.applications, self.parameters, self.uri, self.name) def as_dict(self): return {'classes': self._classes.as_list(), 'applications': self._applications.as_list(), 'parameters': self._parameters.as_dict() } reclass-1.2.2/reclass/datatypes/parameters.py000066400000000000000000000202061225711215500213200ustar00rootroot00000000000000# # -*- coding: utf-8 -*- # # This file is part of reclass (http://github.com/madduck/reclass) # # Copyright © 2007–13 martin f. krafft # Released under the terms of the Artistic Licence 2.0 # import types from reclass.defaults import PARAMETER_INTERPOLATION_DELIMITER from reclass.utils.dictpath import DictPath from reclass.utils.refvalue import RefValue from reclass.errors import InfiniteRecursionError, UndefinedVariableError class Parameters(object): ''' A class to hold nested dictionaries with the following specialities: 1. "merging" a dictionary (the "new" dictionary) into the current Parameters causes a recursive walk of the new dict, during which - scalars (incl. tuples) are replaced with the value from the new dictionary; - lists are extended, not replaced; - dictionaries are updated (using dict.update), not replaced; 2. "interpolating" a dictionary means that values within the dictionary can reference other values in the same dictionary. Those references are collected during merging and then resolved during interpolation, which avoids having to walk the dictionary twice. If a referenced value contains references itself, those are resolved first, in topological order. Therefore, deep references work. Cyclical references cause an error. To support these specialities, this class only exposes very limited functionality and does not try to be a really mapping object. ''' DEFAULT_PATH_DELIMITER = PARAMETER_INTERPOLATION_DELIMITER def __init__(self, mapping=None, delimiter=None): if delimiter is None: delimiter = Parameters.DEFAULT_PATH_DELIMITER self._delimiter = delimiter self._base = {} self._occurrences = {} if mapping is not None: # we initialise by merging, otherwise the list of references might # not be updated self.merge(mapping) delimiter = property(lambda self: self._delimiter) def __len__(self): return len(self._base) def __repr__(self): return '%s(%r, %r)' % (self.__class__.__name__, self._base, self.delimiter) def __eq__(self, other): return isinstance(other, type(self)) \ and self._base == other._base \ and self._delimiter == other._delimiter def __ne__(self, other): return not self.__eq__(other) def as_dict(self): return self._base.copy() def _update_scalar(self, cur, new, path): if self.delimiter is None or not isinstance(new, (types.StringTypes, RefValue)): # either there is no delimiter defined (and hence no references # are being used), or the new value is not a string (and hence # cannot be turned into a RefValue), and not a RefValue. We can # shortcut and just return the new scalar return new elif isinstance(new, RefValue): # the new value is (already) a RefValue, so we need not touch it # at all ret = new else: # the new value is a string, let's see if it contains references, # by way of wrapping it in a RefValue and querying the result ret = RefValue(new, self.delimiter) if not ret.has_references(): # do not replace with RefValue instance if there are no # references, i.e. discard the RefValue in ret, just return # the new value return new # So we now have a RefValue. Let's, keep a reference to the instance # we just created, in a dict indexed by the dictionary path, instead # of just a list. The keys are required to resolve dependencies during # interpolation self._occurrences[path] = ret return ret def _extend_list(self, cur, new, path): if isinstance(cur, list): ret = cur offset = len(cur) else: ret = [cur] offset = 1 for i in xrange(len(new)): ret.append(self._merge_recurse(None, new[i], path.new_subpath(offset + i))) return ret def _merge_dict(self, cur, new, path): if isinstance(cur, dict): ret = cur else: # nothing sensible to do raise TypeError('Cannot merge dict into {0} ' 'objects'.format(type(cur))) if self.delimiter is None: # a delimiter of None indicates that there is no value # processing to be done, and since there is no current # value, we do not need to walk the new dictionary: ret.update(new) return ret for key, newvalue in new.iteritems(): ret[key] = self._merge_recurse(ret.get(key), newvalue, path.new_subpath(key)) return ret def _merge_recurse(self, cur, new, path=None): if path is None: path = DictPath(self.delimiter) if isinstance(new, dict): if cur is None: cur = {} return self._merge_dict(cur, new, path) elif isinstance(new, list): if cur is None: cur = [] return self._extend_list(cur, new, path) else: return self._update_scalar(cur, new, path) def merge(self, other): if isinstance(other, dict): self._base = self._merge_recurse(self._base, other, None) elif isinstance(other, self.__class__): self._base = self._merge_recurse(self._base, other._base, None) else: raise TypeError('Cannot merge %s objects into %s' % (type(other), self.__class__.__name__)) def has_unresolved_refs(self): return len(self._occurrences) > 0 def interpolate(self): while self.has_unresolved_refs(): # we could use a view here, but this is simple enough: # _interpolate_inner removes references from the refs hash after # processing them, so we cannot just iterate the dict path, refvalue = self._occurrences.iteritems().next() self._interpolate_inner(path, refvalue) def _interpolate_inner(self, path, refvalue): self._occurrences[path] = True # mark as seen for ref in refvalue.get_references(): path_from_ref = DictPath(self.delimiter, ref) try: refvalue_inner = self._occurrences[path_from_ref] # If there is no reference, then this will throw a KeyError, # look further down where this is caught and execution passed # to the next iteration of the loop # # If we get here, then the ref references another parameter, # requiring us to recurse, dereferencing first those refs that # are most used and are thus at the leaves of the dependency # tree. if refvalue_inner is True: # every call to _interpolate_inner replaces the value of # the saved occurrences of a reference with True. # Therefore, if we encounter True instead of a refvalue, # it means that we have already processed it and are now # faced with a cyclical reference. raise InfiniteRecursionError(path, ref) self._interpolate_inner(path_from_ref, refvalue_inner) except KeyError as e: # not actually an error, but we are done resolving all # dependencies of the current ref, so move on continue try: new = refvalue.render(self._base) path.set_value(self._base, new) # finally, remove the reference from the occurrences cache del self._occurrences[path] except UndefinedVariableError as e: raise UndefinedVariableError(e.var, path) reclass-1.2.2/reclass/datatypes/tests/000077500000000000000000000000001225711215500177455ustar00rootroot00000000000000reclass-1.2.2/reclass/datatypes/tests/__init__.py000066400000000000000000000000001225711215500220440ustar00rootroot00000000000000reclass-1.2.2/reclass/datatypes/tests/test_applications.py000066400000000000000000000044201225711215500240440ustar00rootroot00000000000000# # -*- coding: utf-8 -*- # # This file is part of reclass (http://github.com/madduck/reclass) # # Copyright © 2007–13 martin f. krafft # Released under the terms of the Artistic Licence 2.0 # from reclass.datatypes import Applications, Classes import unittest try: import unittest.mock as mock except ImportError: import mock TESTLIST1 = ['one', 'two', 'three'] TESTLIST2 = ['red', 'green', '~two', '~three'] GOALLIST = ['one', 'red', 'green'] #TODO: mock out the underlying list class TestApplications(unittest.TestCase): def test_inheritance(self): a = Applications() self.assertIsInstance(a, Classes) def test_constructor_negate(self): a = Applications(TESTLIST1 + TESTLIST2) self.assertSequenceEqual(a, GOALLIST) def test_merge_unique_negate_list(self): a = Applications(TESTLIST1) a.merge_unique(TESTLIST2) self.assertSequenceEqual(a, GOALLIST) def test_merge_unique_negate_instance(self): a = Applications(TESTLIST1) a.merge_unique(Applications(TESTLIST2)) self.assertSequenceEqual(a, GOALLIST) def test_append_if_new_negate(self): a = Applications(TESTLIST1) a.append_if_new(TESTLIST2[2]) self.assertSequenceEqual(a, TESTLIST1[::2]) def test_repr_empty(self): negater = '%%' a = Applications(negation_prefix=negater) self.assertEqual('%r' % a, "%s(%r, '%s')" % (a.__class__.__name__, [], negater)) def test_repr_contents(self): negater = '%%' a = Applications(TESTLIST1, negation_prefix=negater) self.assertEqual('%r' % a, "%s(%r, '%s')" % (a.__class__.__name__, TESTLIST1, negater)) def test_repr_negations(self): negater = '~' a = Applications(TESTLIST2, negation_prefix=negater) self.assertEqual('%r' % a, "%s(%r, '%s')" % (a.__class__.__name__, TESTLIST2, negater)) def test_repr_negations_interspersed(self): l = ['a', '~b', 'a', '~d'] a = Applications(l) is_negation = lambda x: x.startswith(a.negation_prefix) GOAL = filter(lambda x: not is_negation(x), set(l)) + filter(is_negation, l) self.assertEqual('%r' % a, "%s(%r, '~')" % (a.__class__.__name__, GOAL)) if __name__ == '__main__': unittest.main() reclass-1.2.2/reclass/datatypes/tests/test_classes.py000066400000000000000000000073321225711215500230200ustar00rootroot00000000000000# # -*- coding: utf-8 -*- # # This file is part of reclass (http://github.com/madduck/reclass) # # Copyright © 2007–13 martin f. krafft # Released under the terms of the Artistic Licence 2.0 # from reclass.datatypes import Classes from reclass.datatypes.classes import INVALID_CHARACTERS_FOR_CLASSNAMES import unittest try: import unittest.mock as mock except ImportError: import mock from reclass.errors import InvalidClassnameError TESTLIST1 = ['one', 'two', 'three'] TESTLIST2 = ['red', 'green', 'blue'] #TODO: mock out the underlying list class TestClasses(unittest.TestCase): def test_len_empty(self): with mock.patch.object(Classes, 'merge_unique') as m: c = Classes() self.assertEqual(len(c), 0) self.assertFalse(m.called) def test_constructor(self): with mock.patch.object(Classes, 'merge_unique') as m: c = Classes(TESTLIST1) m.assert_called_once_with(TESTLIST1) def test_equality_list_empty(self): self.assertEqual(Classes(), []) def test_equality_list(self): self.assertEqual(Classes(TESTLIST1), TESTLIST1) def test_equality_instance_empty(self): self.assertEqual(Classes(), Classes()) def test_equality_instance(self): self.assertEqual(Classes(TESTLIST1), Classes(TESTLIST1)) def test_inequality(self): self.assertNotEqual(Classes(TESTLIST1), Classes(TESTLIST2)) def test_construct_duplicates(self): c = Classes(TESTLIST1 + TESTLIST1) self.assertSequenceEqual(c, TESTLIST1) def test_append_if_new(self): c = Classes() c.append_if_new(TESTLIST1[0]) self.assertEqual(len(c), 1) self.assertSequenceEqual(c, TESTLIST1[:1]) def test_append_if_new_duplicate(self): c = Classes(TESTLIST1) c.append_if_new(TESTLIST1[0]) self.assertEqual(len(c), len(TESTLIST1)) self.assertSequenceEqual(c, TESTLIST1) def test_append_if_new_nonstring(self): c = Classes() with self.assertRaises(TypeError): c.append_if_new(0) def test_append_invalid_characters(self): c = Classes() invalid_name = ' '.join(('foo', 'bar')) with self.assertRaises(InvalidClassnameError): c.append_if_new(invalid_name) def test_merge_unique(self): c = Classes(TESTLIST1) c.merge_unique(TESTLIST2) self.assertSequenceEqual(c, TESTLIST1 + TESTLIST2) def test_merge_unique_duplicate1_list(self): c = Classes(TESTLIST1) c.merge_unique(TESTLIST1) self.assertSequenceEqual(c, TESTLIST1) def test_merge_unique_duplicate1_instance(self): c = Classes(TESTLIST1) c.merge_unique(Classes(TESTLIST1)) self.assertSequenceEqual(c, TESTLIST1) def test_merge_unique_duplicate2_list(self): c = Classes(TESTLIST1) c.merge_unique(TESTLIST2 + TESTLIST2) self.assertSequenceEqual(c, TESTLIST1 + TESTLIST2) def test_merge_unique_duplicate2_instance(self): c = Classes(TESTLIST1) c.merge_unique(Classes(TESTLIST2 + TESTLIST2)) self.assertSequenceEqual(c, TESTLIST1 + TESTLIST2) def test_merge_unique_nonstring(self): c = Classes() with self.assertRaises(TypeError): c.merge_unique([0,1,2]) def test_repr_empty(self): c = Classes() self.assertEqual('%r' % c, '%s(%r)' % (c.__class__.__name__, [])) def test_repr_contents(self): c = Classes(TESTLIST1) self.assertEqual('%r' % c, '%s(%r)' % (c.__class__.__name__, TESTLIST1)) def test_as_list(self): c = Classes(TESTLIST1) self.assertListEqual(c.as_list(), TESTLIST1) if __name__ == '__main__': unittest.main() reclass-1.2.2/reclass/datatypes/tests/test_entity.py000066400000000000000000000124351225711215500226770ustar00rootroot00000000000000# # -*- coding: utf-8 -*- # # This file is part of reclass (http://github.com/madduck/reclass) # # Copyright © 2007–13 martin f. krafft # Released under the terms of the Artistic Licence 2.0 # from reclass.datatypes import Entity, Classes, Parameters, Applications import unittest try: import unittest.mock as mock except ImportError: import mock @mock.patch.multiple('reclass.datatypes', autospec=True, Classes=mock.DEFAULT, Applications=mock.DEFAULT, Parameters=mock.DEFAULT) class TestEntity(unittest.TestCase): def _make_instances(self, Classes, Applications, Parameters): return Classes(), Applications(), Parameters() def test_constructor_default(self, **mocks): # Actually test the real objects by calling the default constructor, # all other tests shall pass instances to the constructor e = Entity() self.assertEqual(e.name, '') self.assertEqual(e.uri, '') self.assertIsInstance(e.classes, Classes) self.assertIsInstance(e.applications, Applications) self.assertIsInstance(e.parameters, Parameters) def test_constructor_empty(self, **types): instances = self._make_instances(**types) e = Entity(*instances) self.assertEqual(e.name, '') self.assertEqual(e.uri, '') cl, al, pl = [getattr(i, '__len__') for i in instances] self.assertEqual(len(e.classes), cl.return_value) cl.assert_called_once_with() self.assertEqual(len(e.applications), al.return_value) al.assert_called_once_with() self.assertEqual(len(e.parameters), pl.return_value) pl.assert_called_once_with() def test_constructor_empty_named(self, **types): name = 'empty' e = Entity(*self._make_instances(**types), name=name) self.assertEqual(e.name, name) def test_constructor_empty_uri(self, **types): uri = 'test://uri' e = Entity(*self._make_instances(**types), uri=uri) self.assertEqual(e.uri, uri) def test_equal_empty(self, **types): instances = self._make_instances(**types) self.assertEqual(Entity(*instances), Entity(*instances)) for i in instances: i.__eq__.assert_called_once_with(i) def test_equal_empty_named(self, **types): instances = self._make_instances(**types) self.assertEqual(Entity(*instances), Entity(*instances)) name = 'empty' self.assertEqual(Entity(*instances, name=name), Entity(*instances, name=name)) def test_unequal_empty_named(self, **types): instances = self._make_instances(**types) uri = 'test://uri' self.assertNotEqual(Entity(*instances, uri=uri), Entity(*instances, name=uri[::-1])) for i in instances: i.__eq__.assert_called_once_with(i) def test_unequal_empty_named(self, **types): instances = self._make_instances(**types) name = 'empty' self.assertNotEqual(Entity(*instances, name=name), Entity(*instances, name=name[::-1])) for i in instances: i.__eq__.assert_called_once_with(i) def test_unequal_types(self, **types): instances = self._make_instances(**types) self.assertNotEqual(Entity(*instances, name='empty'), None) for i in instances: self.assertEqual(i.__eq__.call_count, 0) def _test_constructor_wrong_types(self, which_replace, **types): instances = self._make_instances(**types) instances[which_replace] = 'Invalid type' e = Entity(*instances) def test_constructor_wrong_type_classes(self, **types): self.assertRaises(TypeError, self._test_constructor_wrong_types, 0) def test_constructor_wrong_type_applications(self, **types): self.assertRaises(TypeError, self._test_constructor_wrong_types, 1) def test_constructor_wrong_type_parameters(self, **types): self.assertRaises(TypeError, self._test_constructor_wrong_types, 2) def test_merge(self, **types): instances = self._make_instances(**types) e = Entity(*instances) e.merge(e) for i, fn in zip(instances, ('merge_unique', 'merge_unique', 'merge')): getattr(i, fn).assert_called_once_with(i) def test_merge_newname(self, **types): instances = self._make_instances(**types) newname = 'newname' e1 = Entity(*instances, name='oldname') e2 = Entity(*instances, name=newname) e1.merge(e2) self.assertEqual(e1.name, newname) def test_merge_newuri(self, **types): instances = self._make_instances(**types) newuri = 'test://uri2' e1 = Entity(*instances, uri='test://uri1') e2 = Entity(*instances, uri=newuri) e1.merge(e2) self.assertEqual(e1.uri, newuri) def test_as_dict(self, **types): instances = self._make_instances(**types) entity = Entity(*instances, name='test') comp = {} comp['classes'] = instances[0].as_list() comp['applications'] = instances[1].as_list() comp['parameters'] = instances[2].as_dict() d = entity.as_dict() self.assertDictEqual(d, comp) if __name__ == '__main__': unittest.main() reclass-1.2.2/reclass/datatypes/tests/test_parameters.py000066400000000000000000000206231225711215500235240ustar00rootroot00000000000000# # -*- coding: utf-8 -*- # # This file is part of reclass (http://github.com/madduck/reclass) # # Copyright © 2007–13 martin f. krafft # Released under the terms of the Artistic Licence 2.0 # from reclass.datatypes import Parameters from reclass.defaults import PARAMETER_INTERPOLATION_SENTINELS from reclass.errors import InfiniteRecursionError import unittest try: import unittest.mock as mock except ImportError: import mock SIMPLE = {'one': 1, 'two': 2, 'three': 3} class TestParameters(unittest.TestCase): def _construct_mocked_params(self, iterable=None, delimiter=None): p = Parameters(iterable, delimiter) self._base = base = p._base p._base = mock.MagicMock(spec_set=dict, wraps=base) p._base.__repr__ = mock.MagicMock(autospec=dict.__repr__, return_value=repr(base)) return p, p._base def test_len_empty(self): p, b = self._construct_mocked_params() l = 0 b.__len__.return_value = l self.assertEqual(len(p), l) b.__len__.assert_called_with() def test_constructor(self): p, b = self._construct_mocked_params(SIMPLE) l = len(SIMPLE) b.__len__.return_value = l self.assertEqual(len(p), l) b.__len__.assert_called_with() def test_repr_empty(self): p, b = self._construct_mocked_params() b.__repr__.return_value = repr({}) self.assertEqual('%r' % p, '%s(%r, %r)' % (p.__class__.__name__, {}, Parameters.DEFAULT_PATH_DELIMITER)) b.__repr__.assert_called_once_with() def test_repr(self): p, b = self._construct_mocked_params(SIMPLE) b.__repr__.return_value = repr(SIMPLE) self.assertEqual('%r' % p, '%s(%r, %r)' % (p.__class__.__name__, SIMPLE, Parameters.DEFAULT_PATH_DELIMITER)) b.__repr__.assert_called_once_with() def test_repr_delimiter(self): delim = '%' p, b = self._construct_mocked_params(SIMPLE, delim) b.__repr__.return_value = repr(SIMPLE) self.assertEqual('%r' % p, '%s(%r, %r)' % (p.__class__.__name__, SIMPLE, delim)) b.__repr__.assert_called_once_with() def test_equal_empty(self): p1, b1 = self._construct_mocked_params() p2, b2 = self._construct_mocked_params() b1.__eq__.return_value = True self.assertEqual(p1, p2) b1.__eq__.assert_called_once_with(b2) def test_equal_default_delimiter(self): p1, b1 = self._construct_mocked_params(SIMPLE) p2, b2 = self._construct_mocked_params(SIMPLE, Parameters.DEFAULT_PATH_DELIMITER) b1.__eq__.return_value = True self.assertEqual(p1, p2) b1.__eq__.assert_called_once_with(b2) def test_equal_contents(self): p1, b1 = self._construct_mocked_params(SIMPLE) p2, b2 = self._construct_mocked_params(SIMPLE) b1.__eq__.return_value = True self.assertEqual(p1, p2) b1.__eq__.assert_called_once_with(b2) def test_unequal_content(self): p1, b1 = self._construct_mocked_params() p2, b2 = self._construct_mocked_params(SIMPLE) b1.__eq__.return_value = False self.assertNotEqual(p1, p2) b1.__eq__.assert_called_once_with(b2) def test_unequal_delimiter(self): p1, b1 = self._construct_mocked_params(delimiter=':') p2, b2 = self._construct_mocked_params(delimiter='%') b1.__eq__.return_value = False self.assertNotEqual(p1, p2) b1.__eq__.assert_called_once_with(b2) def test_unequal_types(self): p1, b1 = self._construct_mocked_params() self.assertNotEqual(p1, None) self.assertEqual(b1.__eq__.call_count, 0) def test_construct_wrong_type(self): with self.assertRaises(TypeError): self._construct_mocked_params('wrong type') def test_merge_wrong_type(self): p, b = self._construct_mocked_params() with self.assertRaises(TypeError): p.merge('wrong type') def test_get_dict(self): p, b = self._construct_mocked_params(SIMPLE) self.assertDictEqual(p.as_dict(), SIMPLE) def test_merge_scalars(self): p1, b1 = self._construct_mocked_params(SIMPLE) mergee = {'five':5,'four':4,'None':None,'tuple':(1,2,3)} p2, b2 = self._construct_mocked_params(mergee) p1.merge(p2) for key, value in mergee.iteritems(): # check that each key, value in mergee resulted in a get call and # a __setitem__ call against b1 (the merge target) self.assertIn(mock.call(key), b1.get.call_args_list) self.assertIn(mock.call(key, value), b1.__setitem__.call_args_list) class TestParametersNoMock(unittest.TestCase): def test_merge_scalars(self): p = Parameters(SIMPLE) mergee = {'five':5,'four':4,'None':None,'tuple':(1,2,3)} p.merge(mergee) goal = SIMPLE.copy() goal.update(mergee) self.assertDictEqual(p.as_dict(), goal) def test_merge_scalars_overwrite(self): p = Parameters(SIMPLE) mergee = {'two':5,'four':4,'three':None,'one':(1,2,3)} p.merge(mergee) goal = SIMPLE.copy() goal.update(mergee) self.assertDictEqual(p.as_dict(), goal) def test_merge_lists(self): l1 = [1,2,3] l2 = [2,3,4] p1 = Parameters(dict(list=l1[:])) p2 = Parameters(dict(list=l2)) p1.merge(p2) self.assertListEqual(p1.as_dict()['list'], l1+l2) def test_merge_list_into_scalar(self): l = ['foo', 1, 2] p1 = Parameters(dict(key=l[0])) p1.merge(Parameters(dict(key=l[1:]))) self.assertListEqual(p1.as_dict()['key'], l) def test_merge_scalar_over_list(self): l = ['foo', 1, 2] p1 = Parameters(dict(key=l[:2])) p1.merge(Parameters(dict(key=l[2]))) self.assertEqual(p1.as_dict()['key'], l[2]) def test_merge_dicts(self): mergee = {'five':5,'four':4,'None':None,'tuple':(1,2,3)} p = Parameters(dict(dict=SIMPLE)) p.merge(Parameters(dict(dict=mergee))) goal = SIMPLE.copy() goal.update(mergee) self.assertDictEqual(p.as_dict(), dict(dict=goal)) def test_merge_dicts_overwrite(self): mergee = {'two':5,'four':4,'three':None,'one':(1,2,3)} p = Parameters(dict(dict=SIMPLE)) p.merge(Parameters(dict(dict=mergee))) goal = SIMPLE.copy() goal.update(mergee) self.assertDictEqual(p.as_dict(), dict(dict=goal)) def test_merge_dict_into_scalar(self): p = Parameters(dict(base='foo')) with self.assertRaises(TypeError): p.merge(Parameters(dict(base=SIMPLE))) def test_merge_scalar_over_dict(self): p = Parameters(dict(base=SIMPLE)) mergee = {'base':'foo'} p.merge(Parameters(mergee)) self.assertDictEqual(p.as_dict(), mergee) def test_interpolate_single(self): v = 42 d = {'foo': 'bar'.join(PARAMETER_INTERPOLATION_SENTINELS), 'bar': v} p = Parameters(d) p.interpolate() self.assertEqual(p.as_dict()['foo'], v) def test_interpolate_multiple(self): v = '42' d = {'foo': 'bar'.join(PARAMETER_INTERPOLATION_SENTINELS) + 'meep'.join(PARAMETER_INTERPOLATION_SENTINELS), 'bar': v[0], 'meep': v[1]} p = Parameters(d) p.interpolate() self.assertEqual(p.as_dict()['foo'], v) def test_interpolate_multilevel(self): v = 42 d = {'foo': 'bar'.join(PARAMETER_INTERPOLATION_SENTINELS), 'bar': 'meep'.join(PARAMETER_INTERPOLATION_SENTINELS), 'meep': v} p = Parameters(d) p.interpolate() self.assertEqual(p.as_dict()['foo'], v) def test_interpolate_list(self): l = [41,42,43] d = {'foo': 'bar'.join(PARAMETER_INTERPOLATION_SENTINELS), 'bar': l} p = Parameters(d) p.interpolate() self.assertEqual(p.as_dict()['foo'], l) def test_interpolate_infrecursion(self): v = 42 d = {'foo': 'bar'.join(PARAMETER_INTERPOLATION_SENTINELS), 'bar': 'foo'.join(PARAMETER_INTERPOLATION_SENTINELS)} p = Parameters(d) with self.assertRaises(InfiniteRecursionError): p.interpolate() if __name__ == '__main__': unittest.main() reclass-1.2.2/reclass/defaults.py000066400000000000000000000015311225711215500167660ustar00rootroot00000000000000# # -*- coding: utf-8 -*- # # This file is part of reclass (http://github.com/madduck/reclass) # # Copyright © 2007–13 martin f. krafft # Released under the terms of the Artistic Licence 2.0 # import os, sys from version import RECLASS_NAME # defaults for the command-line options OPT_STORAGE_TYPE = 'yaml_fs' OPT_INVENTORY_BASE_URI = os.path.join('/etc', RECLASS_NAME) OPT_NODES_URI = 'nodes' OPT_CLASSES_URI = 'classes' OPT_PRETTY_PRINT = True OPT_OUTPUT = 'yaml' CONFIG_FILE_SEARCH_PATH = [os.getcwd(), os.path.expanduser('~'), OPT_INVENTORY_BASE_URI, os.path.dirname(sys.argv[0]) ] CONFIG_FILE_NAME = RECLASS_NAME + '-config.yml' PARAMETER_INTERPOLATION_SENTINELS = ('${', '}') PARAMETER_INTERPOLATION_DELIMITER = ':' reclass-1.2.2/reclass/errors.py000066400000000000000000000116221225711215500164750ustar00rootroot00000000000000# # -*- coding: utf-8 -*- # # This file is part of reclass (http://github.com/madduck/reclass) # # Copyright © 2007–13 martin f. krafft # Released under the terms of the Artistic Licence 2.0 # import posix, sys import traceback from reclass.defaults import PARAMETER_INTERPOLATION_SENTINELS class ReclassException(Exception): def __init__(self, msg, rc=posix.EX_SOFTWARE, *args): super(ReclassException, self).__init__(msg, *args) self._rc = rc self._traceback = traceback.format_exc() def __str__(self): return self.message rc = property(lambda self: self._rc) def exit_with_message(self, out=sys.stderr): print >>out, self.message print >>out, self._traceback sys.exit(self.rc) class PermissionError(ReclassException): def __init__(self, msg, rc=posix.EX_NOPERM): super(PermissionError, self).__init__(msg, rc) class InvocationError(ReclassException): def __init__(self, msg, rc=posix.EX_USAGE): super(InvocationError, self).__init__(msg, rc) class ConfigError(ReclassException): def __init__(self, msg, rc=posix.EX_CONFIG): super(ConfigError, self).__init__(msg, rc) class DuplicateUriError(ConfigError): def __init__(self, nodes_uri, classes_uri): msg = "The inventory URIs must not be the same for nodes and classes: " msg += nodes_uri super(DuplicateUriError, self).__init__(msg) class UriOverlapError(ConfigError): def __init__(self, nodes_uri, classes_uri): msg = "The URIs for the nodes and classes inventories must not " \ "overlap, but {0} and {1} do." msg = msg.format(nodes_uri, classes_uri) super(UriOverlapError, self).__init__(msg) class NotFoundError(ReclassException): def __init__(self, msg, rc=posix.EX_IOERR): super(NotFoundError, self).__init__(msg, rc) class NodeNotFound(NotFoundError): def __init__(self, storage, nodename, uri): self._storage = storage self._name = nodename self._uri = uri msg = "Node '{0}' not found under {1}://{2}".format(self._name, self._storage, self._uri) super(NodeNotFound, self).__init__(msg) class ClassNotFound(NotFoundError): def __init__(self, storage, classname, uri, nodename): self._storage = storage self._name = classname self._uri = uri self._nodename = nodename msg = "Class '{0}' (in ancestry of node '{1}') not found under {2}://{3}" \ .format(self._name, self._nodename, self._storage, self._uri) super(ClassNotFound, self).__init__(msg) class InterpolationError(ReclassException): def __init__(self, msg, rc=posix.EX_DATAERR): super(InterpolationError, self).__init__(msg, rc) class UndefinedVariableError(InterpolationError): def __init__(self, var, context=None): self._var = var msg = "Cannot resolve " + var.join(PARAMETER_INTERPOLATION_SENTINELS) if context: msg += ' in the context of %s' % context super(UndefinedVariableError, self).__init__(msg) var = property(lambda x: x._var) class IncompleteInterpolationError(InterpolationError): def __init__(self, string, end_sentinel): msg = "Missing '%s' to end reference: %s" % \ (end_sentinel, string.join(PARAMETER_INTERPOLATION_SENTINELS)) super(IncompleteInterpolationError, self).__init__(msg) class InfiniteRecursionError(InterpolationError): def __init__(self, path, ref): msg = "Infinite recursion while resolving %s at %s" \ % (ref.join(PARAMETER_INTERPOLATION_SENTINELS), path) super(InfiniteRecursionError, self).__init__(msg) class MappingError(ReclassException): def __init__(self, msg, rc=posix.EX_DATAERR): super(MappingError, self).__init__(msg, rc) class MappingFormatError(MappingError): def __init__(self, msg): super(MappingFormatError, self).__init__(msg) class NameError(ReclassException): def __init__(self, msg, rc=posix.EX_DATAERR): super(NameError, self).__init__(msg, rc) class InvalidClassnameError(NameError): def __init__(self, invalid_character, classname): self._invalid_character = invalid_character self._classname = classname msg = "Invalid character '{0}' in class name '{1}'." msg = msg.format(invalid_character, classname) super(InvalidClassnameError, self).__init__(msg) class DuplicateNodeNameError(NameError): def __init__(self, storage, name, uri1, uri2): msg = "{0}: Definition of node '{1}' in '{2}' collides with " \ "definition in '{3}'. Nodes can only be defined once per inventory." msg = msg.format(storage, name, uri2, uri1) super(DuplicateNodeNameError, self).__init__(msg) reclass-1.2.2/reclass/output/000077500000000000000000000000001225711215500161455ustar00rootroot00000000000000reclass-1.2.2/reclass/output/__init__.py000066400000000000000000000016731225711215500202650ustar00rootroot00000000000000# # -*- coding: utf-8 -*- # # This file is part of reclass (http://github.com/madduck/reclass) # # Copyright © 2007–13 martin f. krafft # Released under the terms of the Artistic Licence 2.0 # class OutputterBase(object): def __init__(self): pass def dump(self, data, pretty_print=False): raise NotImplementedError, "dump() method not yet implemented" class OutputLoader(object): def __init__(self, outputter): self._name = 'reclass.output.' + outputter + '_outputter' try: self._module = __import__(self._name, globals(), locals(), self._name) except ImportError: raise NotImplementedError def load(self, attr='Outputter'): klass = getattr(self._module, attr, None) if klass is None: raise AttributeError, \ 'Outputter class {0} does not export "{1}"'.format(self._name, klass) return klass reclass-1.2.2/reclass/output/json_outputter.py000066400000000000000000000010231225711215500216170ustar00rootroot00000000000000# # -*- coding: utf-8 -*- # # This file is part of reclass (http://github.com/madduck/reclass) # # Copyright © 2007–13 martin f. krafft # Released under the terms of the Artistic Licence 2.0 # from reclass.output import OutputterBase import json class Outputter(OutputterBase): def dump(self, data, pretty_print=False): separators = (',', ': ') if pretty_print else (',', ':') indent = 2 if pretty_print else None return json.dumps(data, indent=indent, separators=separators) reclass-1.2.2/reclass/output/yaml_outputter.py000066400000000000000000000006431225711215500216170ustar00rootroot00000000000000# # -*- coding: utf-8 -*- # # This file is part of reclass (http://github.com/madduck/reclass) # # Copyright © 2007–13 martin f. krafft # Released under the terms of the Artistic Licence 2.0 # from reclass.output import OutputterBase import yaml class Outputter(OutputterBase): def dump(self, data, pretty_print=False): return yaml.dump(data, default_flow_style=not pretty_print) reclass-1.2.2/reclass/storage/000077500000000000000000000000001225711215500162515ustar00rootroot00000000000000reclass-1.2.2/reclass/storage/__init__.py000066400000000000000000000144571225711215500203750ustar00rootroot00000000000000# # -*- coding: utf-8 -*- # # This file is part of reclass (http://github.com/madduck/reclass) # # Copyright © 2007–13 martin f. krafft # Released under the terms of the Artistic Licence 2.0 # import time import types import re import sys import fnmatch import shlex from reclass.datatypes import Entity, Classes from reclass.errors import MappingFormatError def _get_timestamp(): return time.strftime('%c') def vvv(msg): print >>sys.stderr, msg pass class NodeStorageBase(object): def __init__(self, nodes_uri, classes_uri, class_mappings): self._nodes_uri = nodes_uri self._classes_uri = classes_uri self._classes_cache = {} self._class_mappings = class_mappings nodes_uri = property(lambda self: self._nodes_uri) classes_uri = property(lambda self: self._classes_uri) class_mappings = property(lambda self: self._class_mappings) def _match_regexp(self, key, nodename): return re.search(key, nodename) def _match_glob(self, key, nodename): return fnmatch.fnmatchcase(nodename, key) def _shlex_split(self, instr): lexer = shlex.shlex(instr, posix=True) lexer.whitespace_split = True lexer.commenters = '' regexp = False if instr[0] == '/': lexer.quotes += '/' lexer.escapedquotes += '/' regexp = True try: key = lexer.get_token() except ValueError, e: raise MappingFormatError('Error in mapping "{0}": missing closing ' 'quote (or slash)'.format(instr)) if regexp: key = '/{0}/'.format(key) return key, list(lexer) def _populate_with_class_mappings(self, nodename): if not self.class_mappings: return Entity(name='empty') c = Classes() for mapping in self.class_mappings: matched = False key, klasses = self._shlex_split(mapping) if key[0] == ('/'): matched = self._match_regexp(key[1:-1], nodename) if matched: for klass in klasses: c.append_if_new(matched.expand(klass)) else: if self._match_glob(key, nodename): for klass in klasses: c.append_if_new(klass) return Entity(classes=c, name='class mappings for node {0}'.format(nodename)) def _get_storage_name(self): raise NotImplementedError, "Storage class does not have a name" def _get_node(self, name, merge_base=None): raise NotImplementedError, "Storage class not implement node entity retrieval" def _get_class(self, name): raise NotImplementedError, "Storage class not implement class entity retrieval" def _recurse_entity(self, entity, merge_base=None, seen=None, nodename=None): if seen is None: seen = {} if merge_base is None: merge_base = Entity(name='empty (@{0})'.format(nodename)) for klass in entity.classes.as_list(): if klass not in seen: try: class_entity = self._classes_cache[klass] except KeyError, e: class_entity = self._get_class(klass, nodename) self._classes_cache[klass] = class_entity descent = self._recurse_entity(class_entity, seen=seen, nodename=nodename) # on every iteration, we merge the result of the recursive # descent into what we have so far… merge_base.merge(descent) seen[klass] = True # … and finally, we merge what we have at this level into the # result of the iteration, so that elements at the current level # overwrite stuff defined by parents merge_base.merge(entity) return merge_base def _nodeinfo(self, nodename): node_entity = self._get_node(nodename) base_entity = self._populate_with_class_mappings(node_entity.name) seen = {} merge_base = self._recurse_entity(base_entity, seen=seen, nodename=base_entity.name) ret = self._recurse_entity(node_entity, merge_base, seen=seen, nodename=node_entity.name) ret.interpolate() return ret def _nodeinfo_as_dict(self, nodename, entity): ret = {'__reclass__' : {'node': entity.name, 'name': nodename, 'uri': entity.uri, 'timestamp': _get_timestamp() }, } ret.update(entity.as_dict()) return ret def nodeinfo(self, nodename): return self._nodeinfo_as_dict(nodename, self._nodeinfo(nodename)) def _list_inventory(self): raise NotImplementedError, "Storage class does not implement inventory listing" def inventory(self): entities = self._list_inventory() nodes = {} applications = {} classes = {} for f, nodeinfo in entities.iteritems(): d = nodes[f] = self._nodeinfo_as_dict(f, nodeinfo) for a in d['applications']: if a in applications: applications[a].append(f) else: applications[a] = [f] for c in d['classes']: if c in classes: classes[c].append(f) else: classes[c] = [f] return {'__reclass__' : {'timestamp': _get_timestamp()}, 'nodes': nodes, 'classes': classes, 'applications': applications } class StorageBackendLoader(object): def __init__(self, storage_type): self._name = 'reclass.storage.' + storage_type try: self._module = __import__(self._name, globals(), locals(), self._name) except ImportError: raise NotImplementedError def load(self, attr='ExternalNodeStorage'): klass = getattr(self._module, attr, None) if klass is None: raise AttributeError, \ 'Storage backend class {0} does not export "{1}"'.format(self._name, klass) return klass reclass-1.2.2/reclass/storage/yaml_fs/000077500000000000000000000000001225711215500177035ustar00rootroot00000000000000reclass-1.2.2/reclass/storage/yaml_fs/__init__.py000066400000000000000000000057151225711215500220240ustar00rootroot00000000000000# # -*- coding: utf-8 -*- # # This file is part of reclass (http://github.com/madduck/reclass) # # Copyright © 2007–13 martin f. krafft # Released under the terms of the Artistic Licence 2.0 # import os, sys import fnmatch from reclass.storage import NodeStorageBase from yamlfile import YamlFile from directory import Directory from reclass.datatypes import Entity import reclass.errors FILE_EXTENSION = '.yml' def vvv(msg): #print >>sys.stderr, msg pass class ExternalNodeStorage(NodeStorageBase): def __init__(self, nodes_uri, classes_uri, class_mappings): super(ExternalNodeStorage, self).__init__(nodes_uri, classes_uri, class_mappings) def _handle_node_duplicates(name, uri1, uri2): raise reclass.errors.DuplicateNodeNameError(self._get_storage_name(), name, uri1, uri2) self._nodes = self._enumerate_inventory(nodes_uri, duplicate_handler=_handle_node_duplicates) self._classes = self._enumerate_inventory(classes_uri) def _get_storage_name(self): return 'yaml_fs' def _enumerate_inventory(self, basedir, duplicate_handler=None): ret = {} def register_fn(dirpath, filenames): filenames = fnmatch.filter(filenames, '*{0}'.format(FILE_EXTENSION)) vvv('REGISTER {0} in path {1}'.format(filenames, dirpath)) for f in filenames: name = os.path.splitext(f)[0] uri = os.path.join(dirpath, f) if name in ret and callable(duplicate_handler): duplicate_handler(name, os.path.join(basedir, ret[name]), uri) ret[name] = os.path.relpath(uri, basedir) d = Directory(basedir) d.walk(register_fn) return ret def _get_node(self, name): vvv('GET NODE {0}'.format(name)) try: relpath = self._nodes[name] path = os.path.join(self.nodes_uri, relpath) name = os.path.splitext(relpath)[0] except KeyError, e: raise reclass.errors.NodeNotFound(self._get_storage_name(), name, self.nodes_uri) entity = YamlFile(path).get_entity(name) return entity def _get_class(self, name, nodename=None): vvv('GET CLASS {0}'.format(name)) try: path = os.path.join(self.classes_uri, self._classes[name]) except KeyError, e: raise reclass.errors.ClassNotFound(self._get_storage_name(), name, self.classes_uri, nodename) entity = YamlFile(path).get_entity() return entity def _list_inventory(self): entities = {} for n in self._nodes.iterkeys(): entities[n] = self._nodeinfo(n) return entities reclass-1.2.2/reclass/storage/yaml_fs/directory.py000066400000000000000000000040551225711215500222650ustar00rootroot00000000000000# # -*- coding: utf-8 -*- # # This file is part of reclass (http://github.com/madduck/reclass) # # Copyright © 2007–13 martin f. krafft # Released under the terms of the Artistic Licence 2.0 # import os import sys from reclass.errors import NotFoundError SKIPDIRS = ( 'CVS', 'SCCS' ) FILE_EXTENSION = '.yml' def vvv(msg): #print >>sys.stderr, msg pass class Directory(object): def __init__(self, path, fileclass=None): ''' Initialise a directory object ''' if not os.path.isdir(path): raise NotFoundError('No such directory: %s' % path) if not os.access(path, os.R_OK|os.X_OK): raise NotFoundError('Cannot change to or read directory: %s' % path) self._path = path self._fileclass = fileclass self._files = {} def _register_files(self, dirpath, filenames): for f in filter(lambda f: f.endswith(FILE_EXTENSION), filenames): vvv('REGISTER {0}'.format(f)) f = os.path.join(dirpath, f) ptr = None if not self._fileclass else self._fileclass(f) self._files[f] = ptr files = property(lambda self: self._files) def walk(self, register_fn=None): if not callable(register_fn): register_fn = self._register_files def _error(exc): raise(exc) for dirpath, dirnames, filenames in os.walk(self._path, topdown=True, onerror=_error, followlinks=True): vvv('RECURSE {0}, {1} files, {2} subdirectories'.format( dirpath.replace(os.getcwd(), '.'), len(filenames), len(dirnames))) for d in dirnames: if d.startswith('.') or d in SKIPDIRS: vvv(' SKIP subdirectory {0}'.format(d)) dirnames.remove(d) register_fn(dirpath, filenames) def __repr__(self): return '<{0} {1}>'.format(self.__class__.__name__, self._path) reclass-1.2.2/reclass/storage/yaml_fs/yamlfile.py000066400000000000000000000034231225711215500220610ustar00rootroot00000000000000# # -*- coding: utf-8 -*- # # This file is part of reclass (http://github.com/madduck/reclass) # # Copyright © 2007–13 martin f. krafft # Released under the terms of the Artistic Licence 2.0 # from reclass import datatypes import yaml import os from reclass.errors import NotFoundError class YamlFile(object): def __init__(self, path): ''' Initialise a yamlfile object ''' if not os.path.isfile(path): raise NotFoundError('No such file: %s' % path) if not os.access(path, os.R_OK): raise NotFoundError('Cannot open: %s' % path) self._path = path self._data = dict() self._read() path = property(lambda self: self._path) def _read(self): fp = file(self._path) data = yaml.safe_load(fp) if data is not None: self._data = data fp.close() def get_entity(self, name=None): classes = self._data.get('classes') if classes is None: classes = [] classes = datatypes.Classes(classes) applications = self._data.get('applications') if applications is None: applications = [] applications = datatypes.Applications(applications) parameters = self._data.get('parameters') if parameters is None: parameters = {} parameters = datatypes.Parameters(parameters) if name is None: name = self._path return datatypes.Entity(classes, applications, parameters, name=name, uri='yaml_fs://{0}'.format(self._path)) def __repr__(self): return '<{0} {1}, {2}>'.format(self.__class__.__name__, self._path, self._data.keys()) reclass-1.2.2/reclass/utils/000077500000000000000000000000001225711215500157455ustar00rootroot00000000000000reclass-1.2.2/reclass/utils/__init__.py000066400000000000000000000000001225711215500200440ustar00rootroot00000000000000reclass-1.2.2/reclass/utils/dictpath.py000066400000000000000000000077711225711215500201330ustar00rootroot00000000000000# # -*- coding: utf-8 -*- # # This file is part of reclass (http://github.com/madduck/reclass) # # Copyright © 2007–13 martin f. krafft # Released under the terms of the Artistic Licence 2.0 # import types, re class DictPath(object): ''' Represents a path into a nested dictionary. Given a dictionary like d['foo']['bar'] = 42 it can be desirable to obtain a reference to the value stored in the sub-levels, allowing that value to be accessed and changed. Unfortunately, Python provides no easy way to do this, since ref = d['foo']['bar'] does become a reference to the integer 42, but that reference is overwritten when one assigns to it. Hence, DictPath represents the path into a nested dictionary, and can be "applied to" a dictionary to obtain and set values, using a list of keys, or a string representation using a delimiter (which can be escaped): p = DictPath(':', 'foo:bar') p.get_value(d) p.set_value(d, 43) This is a bit backwards, but the right way around would require support by the dict() type. The primary purpose of this class within reclass is to cater for parameter interpolation, so that a reference such as ${foo:bar} in a parameter value may be resolved in the context of the Parameter collections (a nested dict). If the value is a list, then the "key" is assumed to be and interpreted as an integer index: d = {'list': [{'one':1},{'two':2}]} p = DictPath(':', 'list:1:two') p.get_value(d) → 2 This heuristic is okay within reclass, because dictionary keys (parameter names) will always be strings. Therefore it is okay to interpret each component of the path as a string, unless one finds a list at the current level down the nested dictionary. ''' def __init__(self, delim, contents=None): self._delim = delim if contents is None: self._parts = [] else: if isinstance(contents, types.StringTypes): self._parts = self._split_string(contents) elif isinstance(contents, tuple): self._parts = list(contents) elif isinstance(contents, list): self._parts = contents else: raise TypeError('DictPath() takes string or list, '\ 'not %s' % type(contents)) def __repr__(self): return "DictPath(%r, %r)" % (self._delim, str(self)) def __str__(self): return self._delim.join(str(i) for i in self._parts) def __eq__(self, other): if isinstance(other, types.StringTypes): other = DictPath(self._delim, other) return self._parts == other._parts \ and self._delim == other._delim def __ne__(self, other): return not self.__eq__(other) def __hash__(self): return hash(str(self)) def _get_path(self): return self._parts path = property(_get_path) def _get_key(self): if len(self._parts) == 0: return None return self._parts[-1] def _get_innermost_container(self, base): container = base for i in self.path[:-1]: if isinstance(container, (list, tuple)): container = container[int(i)] else: container = container[i] return container def _split_string(self, string): return re.split(r'(? # Released under the terms of the Artistic Licence 2.0 # import re from reclass.utils.dictpath import DictPath from reclass.defaults import PARAMETER_INTERPOLATION_SENTINELS, \ PARAMETER_INTERPOLATION_DELIMITER from reclass.errors import IncompleteInterpolationError, \ UndefinedVariableError _SENTINELS = [re.escape(s) for s in PARAMETER_INTERPOLATION_SENTINELS] _RE = '{0}\s*(.+?)\s*{1}'.format(*_SENTINELS) class RefValue(object): ''' Isolates references in string values RefValue can be used to isolate and eventually expand references to other parameters in strings. Those references can then be iterated and rendered in the context of a dictionary to resolve those references. RefValue always gets constructed from a string, because templating — essentially this is what's going on — is necessarily always about strings. Therefore, generally, the rendered value of a RefValue instance will also be a string. Nevertheless, as this might not be desirable, RefValue will return the referenced variable without casting it to a string, if the templated string contains nothing but the reference itself. For instance: mydict = {'favcolour': 'yellow', 'answer': 42, 'list': [1,2,3]} RefValue('My favourite colour is ${favolour}').render(mydict) → 'My favourite colour is yellow' # a string RefValue('The answer is ${answer}').render(mydict) → 'The answer is 42' # a string RefValue('${answer}').render(mydict) → 42 # an int RefValue('${list}').render(mydict) → [1,2,3] # an list The markers used to identify references are set in reclass.defaults, as is the default delimiter. ''' INTERPOLATION_RE = re.compile(_RE) def __init__(self, string, delim=PARAMETER_INTERPOLATION_DELIMITER): self._strings = [] self._refs = [] self._delim = delim self._parse(string) def _parse(self, string): parts = RefValue.INTERPOLATION_RE.split(string) self._refs = parts[1:][::2] self._strings = parts[0:][::2] self._check_strings(string) def _check_strings(self, orig): for s in self._strings: pos = s.find(PARAMETER_INTERPOLATION_SENTINELS[0]) if pos >= 0: raise IncompleteInterpolationError(orig, PARAMETER_INTERPOLATION_SENTINELS[1]) def _resolve(self, ref, context): path = DictPath(self._delim, ref) try: return path.get_value(context) except KeyError as e: raise UndefinedVariableError(ref) def has_references(self): return len(self._refs) > 0 def get_references(self): return self._refs def _assemble(self, resolver): if not self.has_references(): return self._strings[0] if self._strings == ['', '']: # preserve the type of the referenced variable return resolver(self._refs[0]) # reassemble the string by taking a string and str(ref) pairwise ret = '' for i in range(0, len(self._refs)): ret += self._strings[i] + str(resolver(self._refs[i])) if len(self._strings) > len(self._refs): # and finally append a trailing string, if any ret += self._strings[-1] return ret def render(self, context): resolver = lambda s: self._resolve(s, context) return self._assemble(resolver) def __repr__(self): do_not_resolve = lambda s: s.join(PARAMETER_INTERPOLATION_SENTINELS) return 'RefValue(%r, %r)' % (self._assemble(do_not_resolve), self._delim) reclass-1.2.2/reclass/utils/tests/000077500000000000000000000000001225711215500171075ustar00rootroot00000000000000reclass-1.2.2/reclass/utils/tests/__init__.py000066400000000000000000000000001225711215500212060ustar00rootroot00000000000000reclass-1.2.2/reclass/utils/tests/test_dictpath.py000066400000000000000000000103751225711215500223260ustar00rootroot00000000000000# # -*- coding: utf-8 -*- # # This file is part of reclass (http://github.com/madduck/reclass) # # Copyright © 2007–13 martin f. krafft # Released under the terms of the Artistic Licence 2.0 # from reclass.utils.dictpath import DictPath import unittest class TestDictPath(unittest.TestCase): def test_constructor0(self): p = DictPath(':') self.assertListEqual(p._parts, []) def test_constructor_list(self): l = ['a', 'b', 'c'] p = DictPath(':', l) self.assertListEqual(p._parts, l) def test_constructor_str(self): delim = ':' s = 'a{0}b{0}c'.format(delim) l = ['a', 'b', 'c'] p = DictPath(delim, s) self.assertListEqual(p._parts, l) def test_constructor_str_escaped(self): delim = ':' s = 'a{0}b\{0}b{0}c'.format(delim) l = ['a', 'b\\{0}b'.format(delim), 'c'] p = DictPath(delim, s) self.assertListEqual(p._parts, l) def test_constructor_invalid_type(self): with self.assertRaises(TypeError): p = DictPath(':', 5) def test_equality(self): delim = ':' s = 'a{0}b{0}c'.format(delim) l = ['a', 'b', 'c'] p1 = DictPath(delim, s) p2 = DictPath(delim, l) self.assertEqual(p1, p2) def test_inequality_content(self): delim = ':' s = 'a{0}b{0}c'.format(delim) l = ['d', 'e', 'f'] p1 = DictPath(delim, s) p2 = DictPath(delim, l) self.assertNotEqual(p1, p2) def test_inequality_delimiter(self): l = ['a', 'b', 'c'] p1 = DictPath(':', l) p2 = DictPath('%', l) self.assertNotEqual(p1, p2) def test_repr(self): delim = '%' s = 'a:b\:b:c' p = DictPath(delim, s) self.assertEqual('%r' % p, 'DictPath(%r, %r)' % (delim, s)) def test_str(self): s = 'a:b\:b:c' p = DictPath(':', s) self.assertEqual(str(p), s) def test_path_accessor(self): l = ['a', 'b', 'c'] p = DictPath(':', l) self.assertListEqual(p.path, l) def test_new_subpath(self): l = ['a', 'b', 'c'] p = DictPath(':', l[:-1]) p = p.new_subpath(l[-1]) self.assertListEqual(p.path, l) def test_get_value(self): v = 42 l = ['a', 'b', 'c'] d = {'a':{'b':{'c':v}}} p = DictPath(':', l) self.assertEqual(p.get_value(d), v) def test_get_value_escaped(self): v = 42 l = ['a', 'b:b', 'c'] d = {'a':{'b:b':{'c':v}}} p = DictPath(':', l) self.assertEqual(p.get_value(d), v) def test_get_value_listindex_list(self): v = 42 l = ['a', 1, 'c'] d = {'a':[None, {'c':v}, None]} p = DictPath(':', l) self.assertEqual(p.get_value(d), v) def test_get_value_listindex_str(self): v = 42 s = 'a:1:c' d = {'a':[None, {'c':v}, None]} p = DictPath(':', s) self.assertEqual(p.get_value(d), v) def test_set_value(self): v = 42 l = ['a', 'b', 'c'] d = {'a':{'b':{'c':v}}} p = DictPath(':', l) p.set_value(d, v+1) self.assertEqual(d['a']['b']['c'], v+1) def test_set_value_escaped(self): v = 42 l = ['a', 'b:b', 'c'] d = {'a':{'b:b':{'c':v}}} p = DictPath(':', l) p.set_value(d, v+1) self.assertEqual(d['a']['b:b']['c'], v+1) def test_set_value_escaped_listindex_list(self): v = 42 l = ['a', 1, 'c'] d = {'a':[None, {'c':v}, None]} p = DictPath(':', l) p.set_value(d, v+1) self.assertEqual(d['a'][1]['c'], v+1) def test_set_value_escaped_listindex_str(self): v = 42 s = 'a:1:c' d = {'a':[None, {'c':v}, None]} p = DictPath(':', s) p.set_value(d, v+1) self.assertEqual(d['a'][1]['c'], v+1) def test_get_nonexistent_value(self): l = ['a', 'd'] p = DictPath(':', l) with self.assertRaises(KeyError): p.get_value(dict()) def test_set_nonexistent_value(self): l = ['a', 'd'] p = DictPath(':', l) with self.assertRaises(KeyError): p.set_value(dict(), 42) if __name__ == '__main__': unittest.main() reclass-1.2.2/reclass/utils/tests/test_refvalue.py000066400000000000000000000105641225711215500223370ustar00rootroot00000000000000# # -*- coding: utf-8 -*- # # This file is part of reclass (http://github.com/madduck/reclass) # # Copyright © 2007–13 martin f. krafft # Released under the terms of the Artistic Licence 2.0 # from reclass.utils.refvalue import RefValue from reclass.defaults import PARAMETER_INTERPOLATION_SENTINELS, \ PARAMETER_INTERPOLATION_DELIMITER from reclass.errors import UndefinedVariableError, \ IncompleteInterpolationError import unittest def _var(s): return '%s%s%s' % (PARAMETER_INTERPOLATION_SENTINELS[0], s, PARAMETER_INTERPOLATION_SENTINELS[1]) CONTEXT = {'favcolour':'yellow', 'motd':{'greeting':'Servus!', 'colour':'${favcolour}' }, 'int':1, 'list':[1,2,3], 'dict':{1:2,3:4}, 'bool':True } def _poor_mans_template(s, var, value): return s.replace(_var(var), value) class TestRefValue(unittest.TestCase): def test_simple_string(self): s = 'my cat likes to hide in boxes' tv = RefValue(s) self.assertFalse(tv.has_references()) self.assertEquals(tv.render(CONTEXT), s) def _test_solo_ref(self, key): s = _var(key) tv = RefValue(s) res = tv.render(CONTEXT) self.assertTrue(tv.has_references()) self.assertEqual(res, CONTEXT[key]) def test_solo_ref_string(self): self._test_solo_ref('favcolour') def test_solo_ref_int(self): self._test_solo_ref('int') def test_solo_ref_list(self): self._test_solo_ref('list') def test_solo_ref_dict(self): self._test_solo_ref('dict') def test_solo_ref_bool(self): self._test_solo_ref('bool') def test_single_subst_bothends(self): s = 'I like ' + _var('favcolour') + ' and I like it' tv = RefValue(s) self.assertTrue(tv.has_references()) self.assertEqual(tv.render(CONTEXT), _poor_mans_template(s, 'favcolour', CONTEXT['favcolour'])) def test_single_subst_start(self): s = _var('favcolour') + ' is my favourite colour' tv = RefValue(s) self.assertTrue(tv.has_references()) self.assertEqual(tv.render(CONTEXT), _poor_mans_template(s, 'favcolour', CONTEXT['favcolour'])) def test_single_subst_end(self): s = 'I like ' + _var('favcolour') tv = RefValue(s) self.assertTrue(tv.has_references()) self.assertEqual(tv.render(CONTEXT), _poor_mans_template(s, 'favcolour', CONTEXT['favcolour'])) def test_deep_subst_solo(self): var = PARAMETER_INTERPOLATION_DELIMITER.join(('motd', 'greeting')) s = _var(var) tv = RefValue(s) self.assertTrue(tv.has_references()) self.assertEqual(tv.render(CONTEXT), _poor_mans_template(s, var, CONTEXT['motd']['greeting'])) def test_multiple_subst(self): greet = PARAMETER_INTERPOLATION_DELIMITER.join(('motd', 'greeting')) s = _var(greet) + ' I like ' + _var('favcolour') + '!' tv = RefValue(s) self.assertTrue(tv.has_references()) want = _poor_mans_template(s, greet, CONTEXT['motd']['greeting']) want = _poor_mans_template(want, 'favcolour', CONTEXT['favcolour']) self.assertEqual(tv.render(CONTEXT), want) def test_multiple_subst_flush(self): greet = PARAMETER_INTERPOLATION_DELIMITER.join(('motd', 'greeting')) s = _var(greet) + ' I like ' + _var('favcolour') tv = RefValue(s) self.assertTrue(tv.has_references()) want = _poor_mans_template(s, greet, CONTEXT['motd']['greeting']) want = _poor_mans_template(want, 'favcolour', CONTEXT['favcolour']) self.assertEqual(tv.render(CONTEXT), want) def test_undefined_variable(self): s = _var('no_such_variable') tv = RefValue(s) with self.assertRaises(UndefinedVariableError): tv.render(CONTEXT) def test_incomplete_variable(self): s = PARAMETER_INTERPOLATION_SENTINELS[0] + 'incomplete' with self.assertRaises(IncompleteInterpolationError): tv = RefValue(s) if __name__ == '__main__': unittest.main() reclass-1.2.2/reclass/version.py000066400000000000000000000010221225711215500166370ustar00rootroot00000000000000# # -*- coding: utf-8 -*- # # This file is part of reclass (http://github.com/madduck/reclass) # # Copyright © 2007–13 martin f. krafft # Released under the terms of the Artistic Licence 2.0 # RECLASS_NAME = 'reclass' DESCRIPTION = 'merge data by recursive descent down an ancestry hierarchy' VERSION = '1.2.2' AUTHOR = 'martin f. krafft' AUTHOR_EMAIL = 'reclass@pobox.madduck.net' COPYRIGHT = 'Copyright © 2007–13 ' + AUTHOR LICENCE = 'Artistic Licence 2.0' URL = 'https://github.com/madduck/reclass' reclass-1.2.2/run_tests.py000077500000000000000000000005411225711215500155540ustar00rootroot00000000000000#!/usr/bin/env python # -*- coding: utf-8 -*- # # This file is part of reclass (http://github.com/madduck/reclass) # # Copyright © 2007–13 martin f. krafft # Released under the terms of the Artistic Licence 2.0 # import unittest tests = unittest.TestLoader().discover('reclass') unittest.TextTestRunner(verbosity=1).run(tests) reclass-1.2.2/setup.py000066400000000000000000000014341225711215500146650ustar00rootroot00000000000000# # -*- coding: utf-8 -*- # # This file is part of reclass (http://github.com/madduck/reclass) # # Copyright © 2007–13 martin f. krafft # Released under the terms of the Artistic Licence 2.0 # from reclass.version import * from setuptools import setup, find_packages ADAPTERS = ['salt', 'ansible'] console_scripts = ['reclass = reclass.cli:main'] console_scripts.extend('reclass-{0} = reclass.adapters.{0}:cli'.format(i) for i in ADAPTERS) setup( name = RECLASS_NAME, description = DESCRIPTION, version = VERSION, author = AUTHOR, author_email = AUTHOR_EMAIL, license = LICENCE, url = URL, packages = find_packages(), entry_points = { 'console_scripts': console_scripts }, install_requires = ['pyyaml'] )