pax_global_header00006660000000000000000000000064126454165250014524gustar00rootroot0000000000000052 comment=bbcd27aad8ff4d96772253d019672e6d2d78715a ripe-atlas-tools-1.2.2/000077500000000000000000000000001264541652500147255ustar00rootroot00000000000000ripe-atlas-tools-1.2.2/.gitignore000066400000000000000000000013271264541652500167200ustar00rootroot00000000000000# Byte-compiled / optimized / DLL files __pycache__/ *.py[cod] *.swp # C extensions *.so # Distribution / packaging .Python env/ build/ develop-eggs/ dist/ downloads/ eggs/ .eggs/ lib/ lib64/ parts/ sdist/ var/ *.egg-info/ .installed.cfg *.egg # PyInstaller # Usually these files are written by a python script from a template # before PyInstaller builds the exe, so as to inject date/other infos into it. *.manifest *.spec # Installer logs pip-log.txt pip-delete-this-directory.txt # Unit test / coverage reports htmlcov/ .tox/ .coverage .coverage.* .cache nosetests.xml coverage.xml *,cover # Translations *.mo *.pot # Django stuff: *.log # Sphinx documentation docs/_build/ # PyBuilder target/ # PyCharm .idea/ ripe-atlas-tools-1.2.2/.travis.yml000066400000000000000000000003631264541652500170400ustar00rootroot00000000000000language: python sudo: false env: - TOXENV=py27 - TOXENV=py34 - TOXENV=py27-pyflakes - TOXENV=py27-pep8 - TOXENV=py27-coverage install: - pip install tox - pip install coveralls script: tox after_success: coveralls --verbose ripe-atlas-tools-1.2.2/CHANGES.rst000066400000000000000000000027661264541652500165420ustar00rootroot00000000000000Release History =============== 1.2.2 (released 2016-01-13) --------------------------- New Features ~~~~~~~~~~~~ - Cleaner and more consistent implementation of the renderer plugable architecture. - Usage of newest Cousteau library. 1.2.1 (released 2015-12-15) --------------------------- Bug Fixes ~~~~~~~~~ - Restored some required template files. 1.2.0 (released 2015-12-15) --------------------------- Output Changes ~~~~~~~~~~~~~~ - `#119`_: Support HTTP results. - `#122`_: Allow packagers to set the user agent. 1.1.1 (released 2015-11-25) --------------------------- Output Changes ~~~~~~~~~~~~~~ - `#103`_: Removed header from the ``report`` command. Bug Fixes ~~~~~~~~~ - `#105`_: Measurement report and stream broken on Python3.4. 1.1.0 (released 2015-11-12) --------------------------- New features ~~~~~~~~~~~~ - Support for the creation of NTP, SSLCert, and HTTP measurements. - Additional argument in report command to filter results by probe ASN. - Additional renderer that shows the different destination ASNs and some additional stats about them. Bug Fixes ~~~~~~~~~ - Various fixes. Changes ~~~~~~~ - Better testing. - Additional documentation. 1.0.0 (released 2015-11-02) --------------------------- - Initial release. .. _#103: https://github.com/RIPE-NCC/ripe-atlas-tools/issues/103 .. _#105: https://github.com/RIPE-NCC/ripe-atlas-tools/issues/105 .. _#119: https://github.com/RIPE-NCC/ripe-atlas-tools/issues/119 .. _#122: https://github.com/RIPE-NCC/ripe-atlas-tools/issues/122 ripe-atlas-tools-1.2.2/CONTRIBUTING.rst000066400000000000000000000047231264541652500173740ustar00rootroot00000000000000How To Contribute ================= We would love to have contributions from everyone and no contribution is too small. Please submit as many fixes for typos and grammar bloopers as you can! To make participation in this project as pleasant as possible for everyone, we adhere to the `Code of Conduct`_ by the Python Software Foundation. The following steps will help you get started: Fork, then clone the repo: .. code:: bash $ git clone git@github.com:your-username/ripe-atlas-tools.git Make sure the tests pass beforehand: .. code:: bash $ tox or .. code:: bash $ nosetests tests/ Make your changes. Include tests for your change. Make the tests pass: .. code:: bash $ tox or .. code:: bash $ nosetests tests/ Push to your fork and `submit a pull request`_. Here are a few guidelines that will increase the chances of a quick merge of your pull request: - *Always* try to add tests and docs for your code. If a feature is tested and documented, it's easier for us to merge it. - Follow `PEP 8`_. - Write `good commit messages`_. - If you change something that is noteworthy, don't forget to add an entry to the `changes`_. .. note:: - If you think you have a great contribution but aren’t sure whether it adheres -- or even can adhere -- to the rules: **please submit a pull request anyway**! In the best case, we can transform it into something usable, in the worst case the pull request gets politely closed. There’s absolutely nothing to fear. - If you have a great idea but you don't know how or don't have the time to implement it, please consider opening an issue and someone will pick it up as soon as possible. Thank you for considering a contribution to this project! If you have any questions or concerns, feel free to reach out the RIPE Atlas team via the `mailing list`_, `GitHub Issue Queue`_, or `messenger pigeon`_ -- if you must. .. _submit a pull request: https://github.com/RIPE-NCC/ripe-atlas-tools/compare/ .. _PEP 8: https://www.python.org/dev/peps/pep-0008/ .. _good commit messages: http://tbaggery.com/2008/04/19/a-note-about-git-commit-messages.html .. _Code of Conduct: https://www.python.org/psf/codeofconduct/ .. _changes: https://github.com/RIPE-NCC/ripe-atlas-tools/blob/master/CHANGES.rst .. _mailing list: https://www.ripe.net/mailman/listinfo/ripe-atlas .. _GitHub Issue Queue: https://github.com/RIPE-NCC/ripe-atlas-tools/issues .. _messenger pigeon: https://tools.ietf.org/html/rfc1149 ripe-atlas-tools-1.2.2/LICENSE000066400000000000000000001044601264541652500157370ustar00rootroot00000000000000GNU GENERAL PUBLIC LICENSE Version 3, 29 June 2007 Copyright (C) 2007 Free Software Foundation, Inc. Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed. Preamble The GNU General Public License is a free, copyleft license for software and other kinds of works. The licenses for most software and other practical works are designed to take away your freedom to share and change the works. By contrast, the GNU General Public License is intended to guarantee your freedom to share and change all versions of a program--to make sure it remains free software for all its users. We, the Free Software Foundation, use the GNU General Public License for most of our software; it applies also to any other work released this way by its authors. You can apply it to your programs, too. When we speak of free software, we are referring to freedom, not price. Our General Public Licenses are designed to make sure that you have the freedom to distribute copies of free software (and charge for them if you wish), that you receive source code or can get it if you want it, that you can change the software or use pieces of it in new free programs, and that you know you can do these things. To protect your rights, we need to prevent others from denying you these rights or asking you to surrender the rights. Therefore, you have certain responsibilities if you distribute copies of the software, or if you modify it: responsibilities to respect the freedom of others. For example, if you distribute copies of such a program, whether gratis or for a fee, you must pass on to the recipients the same freedoms that you received. You must make sure that they, too, receive or can get the source code. And you must show them these terms so they know their rights. Developers that use the GNU GPL protect your rights with two steps: (1) assert copyright on the software, and (2) offer you this License giving you legal permission to copy, distribute and/or modify it. For the developers' and authors' protection, the GPL clearly explains that there is no warranty for this free software. For both users' and authors' sake, the GPL requires that modified versions be marked as changed, so that their problems will not be attributed erroneously to authors of previous versions. Some devices are designed to deny users access to install or run modified versions of the software inside them, although the manufacturer can do so. This is fundamentally incompatible with the aim of protecting users' freedom to change the software. The systematic pattern of such abuse occurs in the area of products for individuals to use, which is precisely where it is most unacceptable. Therefore, we have designed this version of the GPL to prohibit the practice for those products. If such problems arise substantially in other domains, we stand ready to extend this provision to those domains in future versions of the GPL, as needed to protect the freedom of users. Finally, every program is threatened constantly by software patents. States should not allow patents to restrict development and use of software on general-purpose computers, but in those that do, we wish to avoid the special danger that patents applied to a free program could make it effectively proprietary. To prevent this, the GPL assures that patents cannot be used to render the program non-free. The precise terms and conditions for copying, distribution and modification follow. TERMS AND CONDITIONS 0. Definitions. "This License" refers to version 3 of the GNU General Public License. "Copyright" also means copyright-like laws that apply to other kinds of works, such as semiconductor masks. "The Program" refers to any copyrightable work licensed under this License. Each licensee is addressed as "you". "Licensees" and "recipients" may be individuals or organizations. To "modify" a work means to copy from or adapt all or part of the work in a fashion requiring copyright permission, other than the making of an exact copy. The resulting work is called a "modified version" of the earlier work or a work "based on" the earlier work. A "covered work" means either the unmodified Program or a work based on the Program. To "propagate" a work means to do anything with it that, without permission, would make you directly or secondarily liable for infringement under applicable copyright law, except executing it on a computer or modifying a private copy. Propagation includes copying, distribution (with or without modification), making available to the public, and in some countries other activities as well. To "convey" a work means any kind of propagation that enables other parties to make or receive copies. Mere interaction with a user through a computer network, with no transfer of a copy, is not conveying. An interactive user interface displays "Appropriate Legal Notices" to the extent that it includes a convenient and prominently visible feature that (1) displays an appropriate copyright notice, and (2) tells the user that there is no warranty for the work (except to the extent that warranties are provided), that licensees may convey the work under this License, and how to view a copy of this License. If the interface presents a list of user commands or options, such as a menu, a prominent item in the list meets this criterion. 1. Source Code. The "source code" for a work means the preferred form of the work for making modifications to it. "Object code" means any non-source form of a work. A "Standard Interface" means an interface that either is an official standard defined by a recognized standards body, or, in the case of interfaces specified for a particular programming language, one that is widely used among developers working in that language. The "System Libraries" of an executable work include anything, other than the work as a whole, that (a) is included in the normal form of packaging a Major Component, but which is not part of that Major Component, and (b) serves only to enable use of the work with that Major Component, or to implement a Standard Interface for which an implementation is available to the public in source code form. A "Major Component", in this context, means a major essential component (kernel, window system, and so on) of the specific operating system (if any) on which the executable work runs, or a compiler used to produce the work, or an object code interpreter used to run it. The "Corresponding Source" for a work in object code form means all the source code needed to generate, install, and (for an executable work) run the object code and to modify the work, including scripts to control those activities. However, it does not include the work's System Libraries, or general-purpose tools or generally available free programs which are used unmodified in performing those activities but which are not part of the work. For example, Corresponding Source includes interface definition files associated with source files for the work, and the source code for shared libraries and dynamically linked subprograms that the work is specifically designed to require, such as by intimate data communication or control flow between those subprograms and other parts of the work. The Corresponding Source need not include anything that users can regenerate automatically from other parts of the Corresponding Source. The Corresponding Source for a work in source code form is that same work. 2. Basic Permissions. All rights granted under this License are granted for the term of copyright on the Program, and are irrevocable provided the stated conditions are met. This License explicitly affirms your unlimited permission to run the unmodified Program. The output from running a covered work is covered by this License only if the output, given its content, constitutes a covered work. This License acknowledges your rights of fair use or other equivalent, as provided by copyright law. You may make, run and propagate covered works that you do not convey, without conditions so long as your license otherwise remains in force. You may convey covered works to others for the sole purpose of having them make modifications exclusively for you, or provide you with facilities for running those works, provided that you comply with the terms of this License in conveying all material for which you do not control copyright. Those thus making or running the covered works for you must do so exclusively on your behalf, under your direction and control, on terms that prohibit them from making any copies of your copyrighted material outside their relationship with you. Conveying under any other circumstances is permitted solely under the conditions stated below. Sublicensing is not allowed; section 10 makes it unnecessary. 3. Protecting Users' Legal Rights From Anti-Circumvention Law. No covered work shall be deemed part of an effective technological measure under any applicable law fulfilling obligations under article 11 of the WIPO copyright treaty adopted on 20 December 1996, or similar laws prohibiting or restricting circumvention of such measures. When you convey a covered work, you waive any legal power to forbid circumvention of technological measures to the extent such circumvention is effected by exercising rights under this License with respect to the covered work, and you disclaim any intention to limit operation or modification of the work as a means of enforcing, against the work's users, your or third parties' legal rights to forbid circumvention of technological measures. 4. Conveying Verbatim Copies. You may convey verbatim copies of the Program's source code as you receive it, in any medium, provided that you conspicuously and appropriately publish on each copy an appropriate copyright notice; keep intact all notices stating that this License and any non-permissive terms added in accord with section 7 apply to the code; keep intact all notices of the absence of any warranty; and give all recipients a copy of this License along with the Program. You may charge any price or no price for each copy that you convey, and you may offer support or warranty protection for a fee. 5. Conveying Modified Source Versions. You may convey a work based on the Program, or the modifications to produce it from the Program, in the form of source code under the terms of section 4, provided that you also meet all of these conditions: a) The work must carry prominent notices stating that you modified it, and giving a relevant date. b) The work must carry prominent notices stating that it is released under this License and any conditions added under section 7. This requirement modifies the requirement in section 4 to "keep intact all notices". c) You must license the entire work, as a whole, under this License to anyone who comes into possession of a copy. This License will therefore apply, along with any applicable section 7 additional terms, to the whole of the work, and all its parts, regardless of how they are packaged. This License gives no permission to license the work in any other way, but it does not invalidate such permission if you have separately received it. d) If the work has interactive user interfaces, each must display Appropriate Legal Notices; however, if the Program has interactive interfaces that do not display Appropriate Legal Notices, your work need not make them do so. A compilation of a covered work with other separate and independent works, which are not by their nature extensions of the covered work, and which are not combined with it such as to form a larger program, in or on a volume of a storage or distribution medium, is called an "aggregate" if the compilation and its resulting copyright are not used to limit the access or legal rights of the compilation's users beyond what the individual works permit. Inclusion of a covered work in an aggregate does not cause this License to apply to the other parts of the aggregate. 6. Conveying Non-Source Forms. You may convey a covered work in object code form under the terms of sections 4 and 5, provided that you also convey the machine-readable Corresponding Source under the terms of this License, in one of these ways: a) Convey the object code in, or embodied in, a physical product (including a physical distribution medium), accompanied by the Corresponding Source fixed on a durable physical medium customarily used for software interchange. b) Convey the object code in, or embodied in, a physical product (including a physical distribution medium), accompanied by a written offer, valid for at least three years and valid for as long as you offer spare parts or customer support for that product model, to give anyone who possesses the object code either (1) a copy of the Corresponding Source for all the software in the product that is covered by this License, on a durable physical medium customarily used for software interchange, for a price no more than your reasonable cost of physically performing this conveying of source, or (2) access to copy the Corresponding Source from a network server at no charge. c) Convey individual copies of the object code with a copy of the written offer to provide the Corresponding Source. This alternative is allowed only occasionally and noncommercially, and only if you received the object code with such an offer, in accord with subsection 6b. d) Convey the object code by offering access from a designated place (gratis or for a charge), and offer equivalent access to the Corresponding Source in the same way through the same place at no further charge. You need not require recipients to copy the Corresponding Source along with the object code. If the place to copy the object code is a network server, the Corresponding Source may be on a different server (operated by you or a third party) that supports equivalent copying facilities, provided you maintain clear directions next to the object code saying where to find the Corresponding Source. Regardless of what server hosts the Corresponding Source, you remain obligated to ensure that it is available for as long as needed to satisfy these requirements. e) Convey the object code using peer-to-peer transmission, provided you inform other peers where the object code and Corresponding Source of the work are being offered to the general public at no charge under subsection 6d. A separable portion of the object code, whose source code is excluded from the Corresponding Source as a System Library, need not be included in conveying the object code work. A "User Product" is either (1) a "consumer product", which means any tangible personal property which is normally used for personal, family, or household purposes, or (2) anything designed or sold for incorporation into a dwelling. In determining whether a product is a consumer product, doubtful cases shall be resolved in favor of coverage. For a particular product received by a particular user, "normally used" refers to a typical or common use of that class of product, regardless of the status of the particular user or of the way in which the particular user actually uses, or expects or is expected to use, the product. A product is a consumer product regardless of whether the product has substantial commercial, industrial or non-consumer uses, unless such uses represent the only significant mode of use of the product. "Installation Information" for a User Product means any methods, procedures, authorization keys, or other information required to install and execute modified versions of a covered work in that User Product from a modified version of its Corresponding Source. The information must suffice to ensure that the continued functioning of the modified object code is in no case prevented or interfered with solely because modification has been made. If you convey an object code work under this section in, or with, or specifically for use in, a User Product, and the conveying occurs as part of a transaction in which the right of possession and use of the User Product is transferred to the recipient in perpetuity or for a fixed term (regardless of how the transaction is characterized), the Corresponding Source conveyed under this section must be accompanied by the Installation Information. But this requirement does not apply if neither you nor any third party retains the ability to install modified object code on the User Product (for example, the work has been installed in ROM). The requirement to provide Installation Information does not include a requirement to continue to provide support service, warranty, or updates for a work that has been modified or installed by the recipient, or for the User Product in which it has been modified or installed. Access to a network may be denied when the modification itself materially and adversely affects the operation of the network or violates the rules and protocols for communication across the network. Corresponding Source conveyed, and Installation Information provided, in accord with this section must be in a format that is publicly documented (and with an implementation available to the public in source code form), and must require no special password or key for unpacking, reading or copying. 7. Additional Terms. "Additional permissions" are terms that supplement the terms of this License by making exceptions from one or more of its conditions. Additional permissions that are applicable to the entire Program shall be treated as though they were included in this License, to the extent that they are valid under applicable law. If additional permissions apply only to part of the Program, that part may be used separately under those permissions, but the entire Program remains governed by this License without regard to the additional permissions. When you convey a copy of a covered work, you may at your option remove any additional permissions from that copy, or from any part of it. (Additional permissions may be written to require their own removal in certain cases when you modify the work.) You may place additional permissions on material, added by you to a covered work, for which you have or can give appropriate copyright permission. Notwithstanding any other provision of this License, for material you add to a covered work, you may (if authorized by the copyright holders of that material) supplement the terms of this License with terms: a) Disclaiming warranty or limiting liability differently from the terms of sections 15 and 16 of this License; or b) Requiring preservation of specified reasonable legal notices or author attributions in that material or in the Appropriate Legal Notices displayed by works containing it; or c) Prohibiting misrepresentation of the origin of that material, or requiring that modified versions of such material be marked in reasonable ways as different from the original version; or d) Limiting the use for publicity purposes of names of licensors or authors of the material; or e) Declining to grant rights under trademark law for use of some trade names, trademarks, or service marks; or f) Requiring indemnification of licensors and authors of that material by anyone who conveys the material (or modified versions of it) with contractual assumptions of liability to the recipient, for any liability that these contractual assumptions directly impose on those licensors and authors. All other non-permissive additional terms are considered "further restrictions" within the meaning of section 10. If the Program as you received it, or any part of it, contains a notice stating that it is governed by this License along with a term that is a further restriction, you may remove that term. If a license document contains a further restriction but permits relicensing or conveying under this License, you may add to a covered work material governed by the terms of that license document, provided that the further restriction does not survive such relicensing or conveying. If you add terms to a covered work in accord with this section, you must place, in the relevant source files, a statement of the additional terms that apply to those files, or a notice indicating where to find the applicable terms. Additional terms, permissive or non-permissive, may be stated in the form of a separately written license, or stated as exceptions; the above requirements apply either way. 8. Termination. You may not propagate or modify a covered work except as expressly provided under this License. Any attempt otherwise to propagate or modify it is void, and will automatically terminate your rights under this License (including any patent licenses granted under the third paragraph of section 11). However, if you cease all violation of this License, then your license from a particular copyright holder is reinstated (a) provisionally, unless and until the copyright holder explicitly and finally terminates your license, and (b) permanently, if the copyright holder fails to notify you of the violation by some reasonable means prior to 60 days after the cessation. Moreover, your license from a particular copyright holder is reinstated permanently if the copyright holder notifies you of the violation by some reasonable means, this is the first time you have received notice of violation of this License (for any work) from that copyright holder, and you cure the violation prior to 30 days after your receipt of the notice. Termination of your rights under this section does not terminate the licenses of parties who have received copies or rights from you under this License. If your rights have been terminated and not permanently reinstated, you do not qualify to receive new licenses for the same material under section 10. 9. Acceptance Not Required for Having Copies. You are not required to accept this License in order to receive or run a copy of the Program. Ancillary propagation of a covered work occurring solely as a consequence of using peer-to-peer transmission to receive a copy likewise does not require acceptance. However, nothing other than this License grants you permission to propagate or modify any covered work. These actions infringe copyright if you do not accept this License. Therefore, by modifying or propagating a covered work, you indicate your acceptance of this License to do so. 10. Automatic Licensing of Downstream Recipients. Each time you convey a covered work, the recipient automatically receives a license from the original licensors, to run, modify and propagate that work, subject to this License. You are not responsible for enforcing compliance by third parties with this License. An "entity transaction" is a transaction transferring control of an organization, or substantially all assets of one, or subdividing an organization, or merging organizations. If propagation of a covered work results from an entity transaction, each party to that transaction who receives a copy of the work also receives whatever licenses to the work the party's predecessor in interest had or could give under the previous paragraph, plus a right to possession of the Corresponding Source of the work from the predecessor in interest, if the predecessor has it or can get it with reasonable efforts. You may not impose any further restrictions on the exercise of the rights granted or affirmed under this License. For example, you may not impose a license fee, royalty, or other charge for exercise of rights granted under this License, and you may not initiate litigation (including a cross-claim or counterclaim in a lawsuit) alleging that any patent claim is infringed by making, using, selling, offering for sale, or importing the Program or any portion of it. 11. Patents. A "contributor" is a copyright holder who authorizes use under this License of the Program or a work on which the Program is based. The work thus licensed is called the contributor's "contributor version". A contributor's "essential patent claims" are all patent claims owned or controlled by the contributor, whether already acquired or hereafter acquired, that would be infringed by some manner, permitted by this License, of making, using, or selling its contributor version, but do not include claims that would be infringed only as a consequence of further modification of the contributor version. For purposes of this definition, "control" includes the right to grant patent sublicenses in a manner consistent with the requirements of this License. Each contributor grants you a non-exclusive, worldwide, royalty-free patent license under the contributor's essential patent claims, to make, use, sell, offer for sale, import and otherwise run, modify and propagate the contents of its contributor version. In the following three paragraphs, a "patent license" is any express agreement or commitment, however denominated, not to enforce a patent (such as an express permission to practice a patent or covenant not to sue for patent infringement). To "grant" such a patent license to a party means to make such an agreement or commitment not to enforce a patent against the party. If you convey a covered work, knowingly relying on a patent license, and the Corresponding Source of the work is not available for anyone to copy, free of charge and under the terms of this License, through a publicly available network server or other readily accessible means, then you must either (1) cause the Corresponding Source to be so available, or (2) arrange to deprive yourself of the benefit of the patent license for this particular work, or (3) arrange, in a manner consistent with the requirements of this License, to extend the patent license to downstream recipients. "Knowingly relying" means you have actual knowledge that, but for the patent license, your conveying the covered work in a country, or your recipient's use of the covered work in a country, would infringe one or more identifiable patents in that country that you have reason to believe are valid. If, pursuant to or in connection with a single transaction or arrangement, you convey, or propagate by procuring conveyance of, a covered work, and grant a patent license to some of the parties receiving the covered work authorizing them to use, propagate, modify or convey a specific copy of the covered work, then the patent license you grant is automatically extended to all recipients of the covered work and works based on it. A patent license is "discriminatory" if it does not include within the scope of its coverage, prohibits the exercise of, or is conditioned on the non-exercise of one or more of the rights that are specifically granted under this License. You may not convey a covered work if you are a party to an arrangement with a third party that is in the business of distributing software, under which you make payment to the third party based on the extent of your activity of conveying the work, and under which the third party grants, to any of the parties who would receive the covered work from you, a discriminatory patent license (a) in connection with copies of the covered work conveyed by you (or copies made from those copies), or (b) primarily for and in connection with specific products or compilations that contain the covered work, unless you entered into that arrangement, or that patent license was granted, prior to 28 March 2007. Nothing in this License shall be construed as excluding or limiting any implied license or other defenses to infringement that may otherwise be available to you under applicable patent law. 12. No Surrender of Others' Freedom. If conditions are imposed on you (whether by court order, agreement or otherwise) that contradict the conditions of this License, they do not excuse you from the conditions of this License. If you cannot convey a covered work so as to satisfy simultaneously your obligations under this License and any other pertinent obligations, then as a consequence you may not convey it at all. For example, if you agree to terms that obligate you to collect a royalty for further conveying from those to whom you convey the Program, the only way you could satisfy both those terms and this License would be to refrain entirely from conveying the Program. 13. Use with the GNU Affero General Public License. Notwithstanding any other provision of this License, you have permission to link or combine any covered work with a work licensed under version 3 of the GNU Affero General Public License into a single combined work, and to convey the resulting work. The terms of this License will continue to apply to the part which is the covered work, but the special requirements of the GNU Affero General Public License, section 13, concerning interaction through a network will apply to the combination as such. 14. Revised Versions of this License. The Free Software Foundation may publish revised and/or new versions of the GNU General Public License from time to time. Such new versions will be similar in spirit to the present version, but may differ in detail to address new problems or concerns. Each version is given a distinguishing version number. If the Program specifies that a certain numbered version of the GNU General Public License "or any later version" applies to it, you have the option of following the terms and conditions either of that numbered version or of any later version published by the Free Software Foundation. If the Program does not specify a version number of the GNU General Public License, you may choose any version ever published by the Free Software Foundation. If the Program specifies that a proxy can decide which future versions of the GNU General Public License can be used, that proxy's public statement of acceptance of a version permanently authorizes you to choose that version for the Program. Later license versions may give you additional or different permissions. However, no additional obligations are imposed on any author or copyright holder as a result of your choosing to follow a later version. 15. Disclaimer of Warranty. THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION. 16. Limitation of Liability. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. 17. Interpretation of Sections 15 and 16. If the disclaimer of warranty and limitation of liability provided above cannot be given local legal effect according to their terms, reviewing courts shall apply local law that most closely approximates an absolute waiver of all civil liability in connection with the Program, unless a warranty or assumption of liability accompanies a copy of the Program in return for a fee. END OF TERMS AND CONDITIONS How to Apply These Terms to Your New Programs If you develop a new program, and you want it to be of the greatest possible use to the public, the best way to achieve this is to make it free software which everyone can redistribute and change under these terms. To do so, attach the following notices to the program. It is safest to attach them to the start of each source file to most effectively state the exclusion of warranty; and each file should have at least the "copyright" line and a pointer to where the full notice is found. {one line to give the program's name and a brief idea of what it does.} Copyright (C) {year} {name of author} This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program. If not, see . Also add information on how to contact you by electronic and paper mail. If the program does terminal interaction, make it output a short notice like this when it starts in an interactive mode: {project} Copyright (C) {year} {fullname} This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'. This is free software, and you are welcome to redistribute it under certain conditions; type `show c' for details. The hypothetical commands `show w' and `show c' should show the appropriate parts of the General Public License. Of course, your program's commands might be different; for a GUI interface, you would use an "about box". You should also get your employer (if you work as a programmer) or school, if any, to sign a "copyright disclaimer" for the program, if necessary. For more information on this, and how to apply and follow the GNU GPL, see . The GNU General Public License does not permit incorporating your program into proprietary programs. If your program is a subroutine library, you may consider it more useful to permit linking proprietary applications with the library. If this is what you want to do, use the GNU Lesser General Public License instead of this License. But first, please read .ripe-atlas-tools-1.2.2/MANIFEST.in000066400000000000000000000002771264541652500164710ustar00rootroot00000000000000include LICENSE include README.rst include CHANGES.rst include MANIFEST.in recursive-include ripe *.py recursive-include ripe *.yaml recursive-include ripe *.txt recursive-include tests *.py ripe-atlas-tools-1.2.2/PACKAGING.md000066400000000000000000000044171264541652500165410ustar00rootroot00000000000000We're working with the community to get this project packaged for as many platforms as possible. To that end, We've created this file to help organise resources. If you'd like to package the toolkit for your favourite distro, or if you've already started, please add your name/GitHub below with a pull request so we don't have people wasting time doubling up on work rather than collaborating. ## OpenBSD * [Florian Obser](https://github.com/fobser) ### Status During the Bucharest hackathon, Florian hacked out a package it's already [in the ports tree](http://cvsweb.openbsd.org/cgi-bin/cvsweb/ports/net/py-ripe.atlas.tools/). ## FreeBSD * [Max Stucchi](https://github.com/stucchimax) ### Status Max has submitted the new ports and [they have been accepted](https://svnweb.freebsd.org/ports?view=revision&revision=403526). ## Gentoo * [Daniel Quinn](https://github.com/danielquinn) ### Status After a great deal of help from #gentoo-proxy-maint, Daniel's pull requests were accepted into the Gentoo portage tree. You can now install it with `emerge ripe-atlas-tools`. ## Debian * Apollon Oikonomopoulos ### Status Apollon is working with the RIPE Atlas team to ensure that the package conforms to Debian standards and has volunteered to submit everything once it's ready. ## Arch * [Wouter de Vries](https://github.com/woutifier) ### Status Wouter has added this project to [Arch's AUR repository](https://aur.archlinux.org/packages/ripe-atlas-tools). ## Fedora * * [Jan Včelák](https://github.com/vcelda) ### Status In progress: https://github.com/fcelda/fedora-ripe-atlas-tools Jan is currently building the binary packages in COPR (which will take some time as there is a lot of other packages in the queue): https://copr.fedoraproject.org/coprs/jvcelak/ripe-atlas-tools/ ## Other Platforms We've been talking with members of the community about expanding the package support for Magellan, but so far no one has officially volunteered. The RIPE Atlas team is happy to assist anyone interested in porting this toolkit to any platform, but we're especially keen on at least getting into: * Ubuntu * Red Hat * CentOS If you'd like to try your hand, or would simply like to offer some advice, feel free to add your name here or contact us directly via *atlas at ripe dot net*. ripe-atlas-tools-1.2.2/README.rst000066400000000000000000000052461264541652500164230ustar00rootroot00000000000000RIPE Atlas Tools (Magellan) =========================== |Documentation| |Build Status| |PYPI Version| |Python Versions| |Python Implementations| |Python Format| The official command-line client for RIPE Atlas. Full Documentation ------------------ Everything is up on `ReadTheDocs`_ How Does it Work? ----------------- Presently, the setup is pretty crude. You can create a ping or traceroute measurement with limited options from the command line: .. code:: bash $ ripe-atlas measure ping --target example.com $ ripe-atlas measure ping --packets 7 --size 42 --target example.com $ ripe-atlas measure traceroute --target example.com $ ripe-atlas measure traceroute --packets 2 --target example.com $ ripe-atlas measure dns --query-argument example.com $ ripe-atlas measure dns --use-probe-resolver --query-type AAAA --query-argument example.com This will create a one-off measurement and then wait for the results to roll in, formatting them as they do. You can also use it to connect to a stream of formatted data. This command will start streaming out all of the results from one of our oldest measurements: .. code:: bash $ ripe-atlas stream 1001 Or you can generate a simple report: .. code:: bash $ ripe-atlas report 1001 Configuration is done by way of a config file, and modifying it can be done from the command line: .. code:: bash $ ripe-atlas configure --set authorisation.create=MY_API_KEY Can I Contribute? ----------------- Absolutely. Please read our `guide`_ on how to contribute. Colophon -------- This project was code-named by means of a `poll`_. In order to conform to the RIPE Atlas theme, it had to be named for an explorer, and so the winning suggestion was for Magellan, *"in memory of those times when RTT was ~3 years"*. .. |Documentation| image:: https://readthedocs.org/projects/ripe-atlas-tools/badge/?version=latest :target: http://ripe-atlas-tools.readthedocs.org/en/latest/?badge=latest :alt: Documentation Status .. _ReadTheDocs: https://ripe-atlas-tools.readthedocs.org/ .. _guide: https://github.com/RIPE-NCC/ripe-atlas-tools/blob/master/CONTRIBUTING.rst .. _poll: https://github.com/RIPE-NCC/ripe-atlas-tools/issues/13 .. |Build Status| image:: https://travis-ci.org/RIPE-NCC/ripe-atlas-tools.png?branch=master :target: https://travis-ci.org/RIPE-NCC/ripe-atlas-tools .. |PYPI Version| image:: https://img.shields.io/pypi/v/ripe.atlas.tools.svg .. |Python Versions| image:: https://img.shields.io/pypi/pyversions/ripe.atlas.tools.svg .. |Python Implementations| image:: https://img.shields.io/pypi/implementation/ripe.atlas.tools.svg .. |Python Format| image:: https://img.shields.io/pypi/format/ripe.atlas.tools.svg ripe-atlas-tools-1.2.2/docs/000077500000000000000000000000001264541652500156555ustar00rootroot00000000000000ripe-atlas-tools-1.2.2/docs/Makefile000066400000000000000000000152521264541652500173220ustar00rootroot00000000000000# Makefile for Sphinx documentation # # You can set these variables from the command line. SPHINXOPTS = SPHINXBUILD = sphinx-build PAPER = BUILDDIR = _build # User-friendly check for sphinx-build ifeq ($(shell which $(SPHINXBUILD) >/dev/null 2>&1; echo $$?), 1) $(error The '$(SPHINXBUILD)' command was not found. Make sure you have Sphinx installed, then set the SPHINXBUILD environment variable to point to the full path of the '$(SPHINXBUILD)' executable. Alternatively you can add the directory with the executable to your PATH. If you don't have Sphinx installed, grab it from http://sphinx-doc.org/) endif # Internal variables. PAPEROPT_a4 = -D latex_paper_size=a4 PAPEROPT_letter = -D latex_paper_size=letter ALLSPHINXOPTS = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . # the i18n builder cannot share the environment and doctrees with the others I18NSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . .PHONY: help clean html dirhtml singlehtml pickle json htmlhelp qthelp devhelp epub latex latexpdf text man changes linkcheck doctest gettext help: @echo "Please use \`make ' where is one of" @echo " html to make standalone HTML files" @echo " dirhtml to make HTML files named index.html in directories" @echo " singlehtml to make a single large HTML file" @echo " pickle to make pickle files" @echo " json to make JSON files" @echo " htmlhelp to make HTML files and a HTML help project" @echo " qthelp to make HTML files and a qthelp project" @echo " devhelp to make HTML files and a Devhelp project" @echo " epub to make an epub" @echo " latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter" @echo " latexpdf to make LaTeX files and run them through pdflatex" @echo " latexpdfja to make LaTeX files and run them through platex/dvipdfmx" @echo " text to make text files" @echo " man to make manual pages" @echo " texinfo to make Texinfo files" @echo " info to make Texinfo files and run them through makeinfo" @echo " gettext to make PO message catalogs" @echo " changes to make an overview of all changed/added/deprecated items" @echo " xml to make Docutils-native XML files" @echo " pseudoxml to make pseudoxml-XML files for display purposes" @echo " linkcheck to check all external links for integrity" @echo " doctest to run all doctests embedded in the documentation (if enabled)" clean: rm -rf $(BUILDDIR)/* html: $(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html @echo @echo "Build finished. The HTML pages are in $(BUILDDIR)/html." dirhtml: $(SPHINXBUILD) -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/dirhtml @echo @echo "Build finished. The HTML pages are in $(BUILDDIR)/dirhtml." singlehtml: $(SPHINXBUILD) -b singlehtml $(ALLSPHINXOPTS) $(BUILDDIR)/singlehtml @echo @echo "Build finished. The HTML page is in $(BUILDDIR)/singlehtml." pickle: $(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) $(BUILDDIR)/pickle @echo @echo "Build finished; now you can process the pickle files." json: $(SPHINXBUILD) -b json $(ALLSPHINXOPTS) $(BUILDDIR)/json @echo @echo "Build finished; now you can process the JSON files." htmlhelp: $(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) $(BUILDDIR)/htmlhelp @echo @echo "Build finished; now you can run HTML Help Workshop with the" \ ".hhp project file in $(BUILDDIR)/htmlhelp." qthelp: $(SPHINXBUILD) -b qthelp $(ALLSPHINXOPTS) $(BUILDDIR)/qthelp @echo @echo "Build finished; now you can run "qcollectiongenerator" with the" \ ".qhcp project file in $(BUILDDIR)/qthelp, like this:" @echo "# qcollectiongenerator $(BUILDDIR)/qthelp/RIPEAtlasToolsMagellan.qhcp" @echo "To view the help file:" @echo "# assistant -collectionFile $(BUILDDIR)/qthelp/RIPEAtlasToolsMagellan.qhc" devhelp: $(SPHINXBUILD) -b devhelp $(ALLSPHINXOPTS) $(BUILDDIR)/devhelp @echo @echo "Build finished." @echo "To view the help file:" @echo "# mkdir -p $$HOME/.local/share/devhelp/RIPEAtlasToolsMagellan" @echo "# ln -s $(BUILDDIR)/devhelp $$HOME/.local/share/devhelp/RIPEAtlasToolsMagellan" @echo "# devhelp" epub: $(SPHINXBUILD) -b epub $(ALLSPHINXOPTS) $(BUILDDIR)/epub @echo @echo "Build finished. The epub file is in $(BUILDDIR)/epub." latex: $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex @echo @echo "Build finished; the LaTeX files are in $(BUILDDIR)/latex." @echo "Run \`make' in that directory to run these through (pdf)latex" \ "(use \`make latexpdf' here to do that automatically)." latexpdf: $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex @echo "Running LaTeX files through pdflatex..." $(MAKE) -C $(BUILDDIR)/latex all-pdf @echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex." latexpdfja: $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex @echo "Running LaTeX files through platex and dvipdfmx..." $(MAKE) -C $(BUILDDIR)/latex all-pdf-ja @echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex." text: $(SPHINXBUILD) -b text $(ALLSPHINXOPTS) $(BUILDDIR)/text @echo @echo "Build finished. The text files are in $(BUILDDIR)/text." man: $(SPHINXBUILD) -b man $(ALLSPHINXOPTS) $(BUILDDIR)/man @echo @echo "Build finished. The manual pages are in $(BUILDDIR)/man." texinfo: $(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo @echo @echo "Build finished. The Texinfo files are in $(BUILDDIR)/texinfo." @echo "Run \`make' in that directory to run these through makeinfo" \ "(use \`make info' here to do that automatically)." info: $(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo @echo "Running Texinfo files through makeinfo..." make -C $(BUILDDIR)/texinfo info @echo "makeinfo finished; the Info files are in $(BUILDDIR)/texinfo." gettext: $(SPHINXBUILD) -b gettext $(I18NSPHINXOPTS) $(BUILDDIR)/locale @echo @echo "Build finished. The message catalogs are in $(BUILDDIR)/locale." changes: $(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) $(BUILDDIR)/changes @echo @echo "The overview file is in $(BUILDDIR)/changes." linkcheck: $(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) $(BUILDDIR)/linkcheck @echo @echo "Link check complete; look for any errors in the above output " \ "or in $(BUILDDIR)/linkcheck/output.txt." doctest: $(SPHINXBUILD) -b doctest $(ALLSPHINXOPTS) $(BUILDDIR)/doctest @echo "Testing of doctests in the sources finished, look at the " \ "results in $(BUILDDIR)/doctest/output.txt." xml: $(SPHINXBUILD) -b xml $(ALLSPHINXOPTS) $(BUILDDIR)/xml @echo @echo "Build finished. The XML files are in $(BUILDDIR)/xml." pseudoxml: $(SPHINXBUILD) -b pseudoxml $(ALLSPHINXOPTS) $(BUILDDIR)/pseudoxml @echo @echo "Build finished. The pseudo-XML files are in $(BUILDDIR)/pseudoxml." ripe-atlas-tools-1.2.2/docs/_static/000077500000000000000000000000001264541652500173035ustar00rootroot00000000000000ripe-atlas-tools-1.2.2/docs/_static/.keep000066400000000000000000000000001264541652500202160ustar00rootroot00000000000000ripe-atlas-tools-1.2.2/docs/_static/custom.css000066400000000000000000000005421264541652500213300ustar00rootroot00000000000000/* override table width restrictions */ @media screen and (min-width: 767px) { .wy-table-responsive table td { /* !important prevents the common CSS stylesheets from overriding this as on RTD they are loaded after this stylesheet */ white-space: normal !important; } .wy-table-responsive { overflow: visible !important; } } ripe-atlas-tools-1.2.2/docs/changelog.rst000066400000000000000000000000341264541652500203330ustar00rootroot00000000000000.. include:: ../CHANGES.rst ripe-atlas-tools-1.2.2/docs/conf.py000066400000000000000000000262241264541652500171620ustar00rootroot00000000000000# -*- coding: utf-8 -*- # # RIPE Atlas Tools (Magellan) documentation build configuration file, created by # sphinx-quickstart on Mon Oct 26 18:36:52 2015. # # This file is execfile()d with the current directory set to its # containing dir. # # Note that not all possible configuration values are present in this # autogenerated file. # # All configuration values have a default; values that are commented out # serve to show the default. import sys import os __version__ = None exec(open("../ripe/atlas/tools/version.py").read()) # Believe it or not, this is the officially sanctioned way to add custom CSS. def setup(app): app.add_stylesheet("custom.css") # If extensions (or modules to document with autodoc) are in another directory, # add these directories to sys.path here. If the directory is relative to the # documentation root, use os.path.abspath to make it absolute, like shown here. #sys.path.insert(0, os.path.abspath('.')) # -- General configuration ------------------------------------------------ # If your documentation needs a minimal Sphinx version, state it here. #needs_sphinx = '1.0' # Add any Sphinx extension module names here, as strings. They can be # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom # ones. extensions = [ 'sphinx.ext.autodoc', 'sphinx.ext.intersphinx', 'sphinx.ext.todo', 'sphinx.ext.pngmath', 'sphinx.ext.viewcode', ] # Add any paths that contain templates here, relative to this directory. templates_path = ['_templates'] # The suffix of source filenames. source_suffix = '.rst' # The encoding of source files. #source_encoding = 'utf-8-sig' # The master toctree document. master_doc = 'index' # General information about the project. project = u'RIPE Atlas Tools (Magellan)' copyright = u'2015, The RIPE Atlas Team' # The version info for the project you're documenting, acts as replacement for # |version| and |release|, also used in various other places throughout the # built documents. # # # If the build process ever explodes here, it's because you've set the version # number in ripe.atlas.tools.version to a string in a format other than x.y.z # # The short X.Y version. version = ".".join(__version__.split(".")[:2]) # The full version, including alpha/beta/rc tags. release = ".".join(__version__.split(".")[:3]) # The language for content autogenerated by Sphinx. Refer to documentation # for a list of supported languages. #language = None # There are two options for replacing |today|: either, you set today to some # non-false value, then it is used: #today = '' # Else, today_fmt is used as the format for a strftime call. #today_fmt = '%B %d, %Y' # List of patterns, relative to source directory, that match files and # directories to ignore when looking for source files. exclude_patterns = ['_build'] # The reST default role (used for this markup: `text`) to use for all # documents. #default_role = None # If true, '()' will be appended to :func: etc. cross-reference text. #add_function_parentheses = True # If true, the current module name will be prepended to all description # unit titles (such as .. function::). #add_module_names = True # If true, sectionauthor and moduleauthor directives will be shown in the # output. They are ignored by default. #show_authors = False # The name of the Pygments (syntax highlighting) style to use. pygments_style = 'sphinx' # A list of ignored prefixes for module index sorting. #modindex_common_prefix = [] # If true, keep warnings as "system message" paragraphs in the built documents. #keep_warnings = False # -- Options for HTML output ---------------------------------------------- # The theme to use for HTML and HTML Help pages. See the documentation for # a list of builtin themes. html_theme = 'default' # Theme options are theme-specific and customize the look and feel of a theme # further. For a list of options available for each theme, see the # documentation. #html_theme_options = {} # Add any paths that contain custom themes here, relative to this directory. html_theme_path = [] # The name for this set of Sphinx documents. If None, it defaults to # " v documentation". #html_title = None # A shorter title for the navigation bar. Default is the same as html_title. #html_short_title = None # The name of an image file (relative to this directory) to place at the top # of the sidebar. #html_logo = None # The name of an image file (within the static path) to use as favicon of the # docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32 # pixels large. #html_favicon = None # Add any paths that contain custom static files (such as style sheets) here, # relative to this directory. They are copied after the builtin static files, # so a file named "default.css" will overwrite the builtin "default.css". html_static_path = ['_static'] # Add any extra paths that contain custom files (such as robots.txt or # .htaccess) here, relative to this directory. These files are copied # directly to the root of the documentation. #html_extra_path = [] # If not '', a 'Last updated on:' timestamp is inserted at every page bottom, # using the given strftime format. #html_last_updated_fmt = '%b %d, %Y' # If true, SmartyPants will be used to convert quotes and dashes to # typographically correct entities. #html_use_smartypants = True # Custom sidebar templates, maps document names to template names. #html_sidebars = {} # Additional templates that should be rendered to pages, maps page names to # template names. #html_additional_pages = {} # If false, no module index is generated. #html_domain_indices = True # If false, no index is generated. #html_use_index = True # If true, the index is split into individual pages for each letter. #html_split_index = False # If true, links to the reST sources are added to the pages. #html_show_sourcelink = True # If true, "Created using Sphinx" is shown in the HTML footer. Default is True. #html_show_sphinx = True # If true, "(C) Copyright ..." is shown in the HTML footer. Default is True. #html_show_copyright = True # If true, an OpenSearch description file will be output, and all pages will # contain a tag referring to it. The value of this option must be the # base URL from which the finished HTML is served. #html_use_opensearch = '' # This is the file name suffix for HTML files (e.g. ".xhtml"). #html_file_suffix = None # Output file base name for HTML help builder. htmlhelp_basename = 'RIPEAtlasToolsMagellandoc' # # Attempt to use the ReadTheDocs theme. If it's not installed, fallback to # the default. # try: import sphinx_rtd_theme html_theme = "sphinx_rtd_theme" html_theme_path = [sphinx_rtd_theme.get_html_theme_path()] except ImportError: pass # -- Options for LaTeX output --------------------------------------------- latex_elements = { # The paper size ('letterpaper' or 'a4paper'). #'papersize': 'letterpaper', # The font size ('10pt', '11pt' or '12pt'). #'pointsize': '10pt', # Additional stuff for the LaTeX preamble. #'preamble': '', } # Grouping the document tree into LaTeX files. List of tuples # (source start file, target name, title, # author, documentclass [howto, manual, or own class]). latex_documents = [ ('index', 'RIPEAtlasToolsMagellan.tex', u'RIPE Atlas Tools (Magellan) Documentation', u'The RIPE Atlas Team', 'manual'), ] # The name of an image file (relative to this directory) to place at the top of # the title page. #latex_logo = None # For "manual" documents, if this is true, then toplevel headings are parts, # not chapters. #latex_use_parts = False # If true, show page references after internal links. #latex_show_pagerefs = False # If true, show URL addresses after external links. #latex_show_urls = False # Documents to append as an appendix to all manuals. #latex_appendices = [] # If false, no module index is generated. #latex_domain_indices = True # -- Options for manual page output --------------------------------------- # One entry per manual page. List of tuples # (source start file, name, description, authors, manual section). man_pages = [ ('index', 'ripeatlastoolsmagellan', u'RIPE Atlas Tools (Magellan) Documentation', [u'The RIPE Atlas Team'], 1) ] # If true, show URL addresses after external links. #man_show_urls = False # -- Options for Texinfo output ------------------------------------------- # Grouping the document tree into Texinfo files. List of tuples # (source start file, target name, title, author, # dir menu entry, description, category) texinfo_documents = [ ('index', 'RIPEAtlasToolsMagellan', u'RIPE Atlas Tools (Magellan) Documentation', u'The RIPE Atlas Team', 'RIPEAtlasToolsMagellan', 'One line description of project.', 'Miscellaneous'), ] # Documents to append as an appendix to all manuals. #texinfo_appendices = [] # If false, no module index is generated. #texinfo_domain_indices = True # How to display URL addresses: 'footnote', 'no', or 'inline'. #texinfo_show_urls = 'footnote' # If true, do not generate a @detailmenu in the "Top" node's menu. #texinfo_no_detailmenu = False # -- Options for Epub output ---------------------------------------------- # Bibliographic Dublin Core info. epub_title = u'RIPE Atlas Tools (Magellan)' epub_author = u'The RIPE Atlas Team' epub_publisher = u'The RIPE Atlas Team' epub_copyright = u'2015, The RIPE Atlas Team' # The basename for the epub file. It defaults to the project name. #epub_basename = u'RIPE Atlas Tools (Magellan)' # The HTML theme for the epub output. Since the default themes are not optimized # for small screen space, using the same theme for HTML and epub output is # usually not wise. This defaults to 'epub', a theme designed to save visual # space. #epub_theme = 'epub' # The language of the text. It defaults to the language option # or en if the language is not set. #epub_language = '' # The scheme of the identifier. Typical schemes are ISBN or URL. #epub_scheme = '' # The unique identifier of the text. This can be a ISBN number # or the project homepage. #epub_identifier = '' # A unique identification for the text. #epub_uid = '' # A tuple containing the cover image and cover page html template filenames. #epub_cover = () # A sequence of (type, uri, title) tuples for the guide element of content.opf. #epub_guide = () # HTML files that should be inserted before the pages created by sphinx. # The format is a list of tuples containing the path and title. #epub_pre_files = [] # HTML files shat should be inserted after the pages created by sphinx. # The format is a list of tuples containing the path and title. #epub_post_files = [] # A list of files that should not be packed into the epub file. epub_exclude_files = ['search.html'] # The depth of the table of contents in toc.ncx. #epub_tocdepth = 3 # Allow duplicate toc entries. #epub_tocdup = True # Choose between 'default' and 'includehidden'. #epub_tocscope = 'default' # Fix unsupported image types using the PIL. #epub_fix_images = False # Scale large images. #epub_max_image_width = 0 # How to display URL addresses: 'footnote', 'no', or 'inline'. #epub_show_urls = 'inline' # If false, no index is generated. #epub_use_index = True # Example configuration for intersphinx: refer to the Python standard library. intersphinx_mapping = {'http://docs.python.org/': None} ripe-atlas-tools-1.2.2/docs/contributing.rst000066400000000000000000000000411264541652500211110ustar00rootroot00000000000000.. include:: ../CONTRIBUTING.rst ripe-atlas-tools-1.2.2/docs/index.rst000066400000000000000000000012241264541652500175150ustar00rootroot00000000000000RIPE Atlas Tools (Magellan) =========================== The official command-line client for RIPE Atlas. .. _index-why-this-exists: Why This Exists =============== `RIPE Atlas`_ is a powerful Internet measurements platform that until recently was only accessible via the website and the RESTful API. The reality however is that a great many people using RIPE Atlas are most comfortable on the command-line, so this project is an attempt to fill that gap. .. _RIPE Atlas: https://atlas.ripe.net Contents ======== .. toctree:: :maxdepth: 2 quickstart installation use plugins contributing packaging troubleshooting changelog ripe-atlas-tools-1.2.2/docs/installation.rst000066400000000000000000000154271264541652500211210ustar00rootroot00000000000000.. _requirements-and-installation: Requirements & Installation *************************** This is a Linux-based tool, though it should work just fine in a BSD variant. Windows is definitely not supported. In terms of the actual installation, only Python's package manager (``pip``) is currently supported, and the installation process may require some system packages to be installed in order for everything to work. .. _requirements-and-installation-system-requirements: System Requirements =================== Some of the dependencies need to be compiled, so you'll need a compiler on your system, as well as the development libraries for Python. In the Linux world, this typically means a few packages need to be installed from your standard package manager, but in true Linux fashion, each distribution does things slightly differently. The most important thing to know is that you need Python 2.7 or 3. Python 2.6 will never be supported because it's old, ugly, and needs to die. .. _requirements-and-installation-distribution-specific-requirements: Distribution Specific Requirements ---------------------------------- .. note:: If you're running OpenBSD, you can skip this whole section. You can even skip the next one too. Just skip down to :ref:`Installation:OpenBSD ` and follow the instructions. Everything else is taken care of for you. .. _requirements-and-installation-distribution-specific-requirements-debian: Debian/Ubuntu ............. The following has been tested on Debian Jessie. Debian-based distributions require three system packages to be installed first: .. code:: bash sudo apt-get install python-dev libffi-dev libssl-dev You'll also need either ``virtualenv`` (recommended), or if you're not comfortable with that, at the very least, you'll need ``pip``: .. code:: bash sudo apt-get install python-virtualenv python-pip .. _requirements-and-installation-distribution-specific-requirements-centos: CentOS ...... This following has been tested on CentOS 7. Since we require Python's ``pip``, we first need to install the ``epel-release`` repository: .. code:: bash sudo yum install epel-release You'll also need the following system libraries: .. code:: bash sudo yum install gcc libffi-devel openssl-devel Once that's finished, you'll need access to ``virtualenv`` (recommended), or if you're not comfortable with that, at the very least, you'll need ``pip``: .. code:: bash sudo yum install python-virtualenv python-pip .. _requirements-and-installation-distribution-specific-requirements-gentoo: Gentoo ...... If you're a Gentoo user, you never have to worry about development libraries, but if you intend to use the bleeding-edge version of this package (and what self-respecting Gentoo user wouldn't?) then you'll probably want to make sure that git is built with curl support: .. code:: bash sudo USE="curl" emerge git If you're not going bleeding edge, or if you're just going to use SSH to get the code from GitHub, then Gentoo will have everything ready for you. .. _requirements-and-installation-distribution-specific-requirements-apple: Apple OSX ......... These instructions expect that you've got Python's ``pip`` installed, so if you have no idea what that is, or simply don't have it yet, you should be able to install pip with one easy command: .. code:: bash sudo easy_install pip Outside of that, a few of the Python dependencies require that you have a compiler on your system. For this, you need only get a free copy of `Xcode`_ from the app store, and from there you should be good to go. .. _Xcode: https://itunes.apple.com/us/app/xcode/id497799835 .. _installation-python-requirements: Python Requirements =================== Importantly, Magellan requires Python 2.7 or higher. For most desktop users, this shouldn't be a problem, but for some older servers like CentOS 6 and lower, this may cause some pain. Thankfully, for most such systems, there are usually work-arounds that allow you to install a more modern version of Python in parallel. Magellan depends on two other RIPE Atlas libraries, Cousteau and Sagan, which in turn depend on a reasonable number of Python libraries. Thankfully, Python's package manager, ``pip`` should handle all of these for you: * ripe.atlas.cousteau * ripe.atlas.sagan * tzlocal * pyyaml .. _installation: Installation ============ .. _installation-from-openbsd: OpenBSD ------- OpenBSD was the first platform to have a port for Magellan, so installation is easy: .. code:: bash sudo pkg_add py-ripe.atlas.tools .. _installation-from-freebsd: FreeBSD ------- FreeBSD has a port ready for you: .. code:: bash cd /usr/ports/net/py-ripe.atlas.tools make install .. _installation-from-gentoo: Gentoo ------ There's an ebuild for Magellan in Portage, so installation is as any other package: .. code:: bash sudo emerge ripe-atlas-tools .. _installation-from-pypi: From PyPi --------- Python's ``pip`` program can be used to install packages globally (not a good idea since it conflicts with your system package manager) or on a per-user basis. Typically, this is done with `virtualenv`_, but if you don't want to use that, you can always pass ``--user`` to the ``pip`` program and it'll install a user-based copy in ``${HOME}/.local/``. .. _virtualenv: https://pypi.python.org/pypi/virtualenv .. code:: bash # From within a virtualenv pip install ripe.atlas.tools # In your user's local environment pip install --user ripe.atlas.tools Or if you want to live on the edge and perhaps try submitting a pull request of your own: One day, we want this process to be as easy as installing any other command-line program, that is, with ``apt``, ``dfn``, or ``emerge``, but until that day, Python's standard package manager, ``pip`` does the job nicely. .. _installation-from-github: From GitHub ----------- If you're feeling a little more daring and want to go bleeding-edge and use our ``master`` branch on GitHub, you can have pip install right from there: .. code:: bash pip install git+https://github.com/RIPE-NCC/ripe-atlas-tools.git If you think you'd like to contribute back to the project, we recommend the use of pip's ``-e`` flag, which will place the Magellan code in a directory where you can edit it, and see the results without having to go through a new install procedure every time. Simply clone the repo on GitHub and install it like so: .. code:: bash pip install -e git+https://github.com/your-username/ripe-atlas-tools.git .. _installation-from-tarball: From a Tarball -------------- If for some reason you want to just download the source and install it manually, you can always do that too. Simply un-tar the file and run the following in the same directory as ``setup.py``: .. code:: bash python setup.py install ripe-atlas-tools-1.2.2/docs/packaging.rst000066400000000000000000000025721264541652500203410ustar00rootroot00000000000000Packaging ========= For those interested in packaging RIPE Atlas Tools for their favourite distro, this section is for you. Currently Supported ------------------- * OpenBSD * FreeBSD In Progress ----------- * `Gentoo Linux`_: Daniel Quinn has an `overlay`_ in development. * `Debian Linux`_: Apollon Oikonomopoulos is working with the RIPE Atlas team to ensure that everything conforms to Debian standards. .. _`Gentoo Linux`: https://gentoo.org/ .. _`overlay`: https://github.com/danielquinn/ripe-atlas-overlay .. _`Debian Linux`: https://www.debian.org/ Additional Distributions ------------------------ Is your distribution not listed? If you'd like to build a package for another distro or even if you're just someone who knows someone who can help us package and distribute this, please get in touch. Further Information ------------------- User Agent ~~~~~~~~~~ When packaging, it's good practise to manually set the user agent used within the toolkit so that we can get a rough idea of which distros are using this software. This is easily done by writing an arbitrary string to ``/ripe/atlas/tools/user-agent``. Something like this is recommended::: RIPE Atlas Tools (Magellan) [FreeBSD 10.2] 1.2 The only limitations to this file are that it should: * Only have one line in it (all other will be ignored) * That line should have a maximum of 128 characters in it ripe-atlas-tools-1.2.2/docs/plugins.rst000066400000000000000000000164661264541652500201050ustar00rootroot00000000000000.. _plugins: How to Create Your Own Plugins ============================== We built this toolkit for the community, and we knew going in that we couldn't possibly build every feature that every user could want, so we built this thing to be pluggable. You can write your own renderer(s) and use them seamlessly within your own environment, and if you think that others might benefit from your work, you can share your renderer as easy as posting a file online. Ready? So you have an idea now. You want to create a renderer called "awesomerenderer" and you want it to do some fancy things with traceroute measurement results. What do you have to do? .. _plugins-create: Create Your Renderer File ------------------------- As we've already covered, Magellan will look for renderers in very specific places, so you need to put your file(s) there. Additionally however, you have to make sure that you conform to Python norms, or stuff just won't work. Here's the basic commands to get you started: .. code:: bash $ mkdir -p ${HOME}/.config/ripe-atlas-tools/renderers $ touch ${HOME}/.config/ripe-atlas-tools/renderers/__init__.py $ touch ${HOME}/.config/ripe-atlas-tools/renderers/my_renderer.py The ``mkdir`` step there will create the renderers directory (if it doesn't exist already), and the ``touch`` commands will create the mandatory init file (for Python) and your renderer. Note that you can use whatever name you like for your renderer, so long as it consists only of letters, numbers, and the underscore and that it starts with a letter. Also, to be compliant with the rest of the project, it should be entirely lowercase. For our purposes though, ``my_renderer.py`` will suffice. .. _plugins-try-to-run: (Try to) Run It! ---------------- If you run this right now: .. code:: bash $ ripe-atlas report --help You should see ``my_renderer`` int he list of options for ``--renderer``. Pretty cool eh? However, if you try to run that, this'll happen: .. code:: bash $ ripe-atlas report 1000192 --renderer my_renderer The renderer you selected, "my_renderer" could not be found. Which kind of makes sense really. You've created a file called ``my_renderer``, but it's totally empty. Magellan found the file alright, but when it tried to import ``Renderer`` from it, everything exploded. .. _plugins-write: Actually Write a Renderer ------------------------- So now you know that we can see your renderer file, but you need to know what kind of code to put in there. Don't worry, we've got you covered: .. _plugins-write-anatomy: Anatomy of a Renderer ~~~~~~~~~~~~~~~~~~~~~ A "renderer" is simply a file located in a special place that contains some Python code defining a class called ``Renderer`` that subclasses ``ripe.atlas.tools.renderers.base.BaseRenderer``. Your class need only define one method: ``on_result()``, which is called every time a new result comes down the pipe. Let's look at a really simple example: .. code:: python from ripe.atlas.tools.renderers.base import Renderer as BaseRenderer class Renderer(BaseRenderer): # This renderer is capable of handling ping results only. RENDERS = [BaseRenderer.TYPE_PING] def on_result(self, result): """ on_result() only gets one argument, a result object, which is actually an instance of a RIPE Atlas result parsed with Sagan: https://ripe-atlas-sagan.readthedocs.org/ """ return "Packets received: {}".format(result.packets_received) As you can see, this renderer isn't very useful, but we're providing it here to give you a rough idea of what you get to play with when defining your own renderer. In the case of our PingPacketRenderer, we're doing the simplest of tasks: we're returning the number of packets in each result. The job of ``on_result()`` is to take a Sagan result object as input and return a string. **It should not print anything to standard out**, rather it should simply return a string that will get printed to standard out by the surrounding framework. .. _plugins-write-anatomy-additional: Additional Options :::::::::::::::::: It's likely that you will only ever need to work with ``on_result()``, but in the event that you'd like to get more complicated, there are options: ``header()``, ``additional()``, and ``footer()``. Note however that these other methods are currently only available to the ``report`` command. Streaming only makes use of ``on_result()``. .. _plugins-write-anatomy-additional-header: header() ........ The value returned from this method is printed to standard out before any results are captured. By default it returns an empty string. .. _plugins-write-anatomy-additional-additional: additional() ............ Typically used for summary logic, this is executed after the last result is rendered. A common pattern is to override ``__init__()`` to set some collector properties, update them via ``on_result()``, and then print out said properties in a summary via this method. For an example, let's update our ``Renderer`` class: .. code:: python from ripe.atlas.tools.renderers.base import Renderer as BaseRenderer class Renderer(BaseRenderer): RENDERS = [BaseRenderer.TYPE_PING] def __init__(self, *args, **kwargs): self.packet_total = 0 BaseRenderer.__init__(self, *args, **kwargs) def on_result(self, result): self.packet_total += result.packets_received return "Packets received: {}\n".format(result.packets_received) def additional(self, results): return "\nTotal packets received: {}\n".format(self.packet_total) Note that the passed-in value of ``results`` is the list of Sagan Result objects that were previously looped over for on_result(). You can do some interesting things with that. .. _plugins-write-anatomy-additional-footer: footer() ........ Much the same as ``header()``, this should return a string, but unlike ``header()``, the output of this method is rendered after everything else. .. _plugins-run: Run It! ------- Now that you've written your renderer and the file is stored where it's supposed to be, it should be ready to go: .. code:: bash $ ripe-atlas report --help You should see ``my_renderer`` in the list of options for ``--renderer`` just as before, but now when you actually try to execute it... .. code:: bash $ ripe-atlas report 1000192 --renderer my_renderer Packets received: 3 Packets received: 3 Packets received: 3 Packets received: 3 Packets received: 3 Packets received: 3 Total packets received: 18 It's not very interesting, but it's a start! .. _plugins-contributing: Contributing ------------ We love it when people write stuff that talks to our stuff. If you think your stuff is useful, it'd be awesome if you could do any of these: * Post to the [ripe-atlas mailing list](https://www.ripe.net/mailman/listinfo/ripe-atlas) about it. You can also solicit feedback from the RIPE Atlas developers or the wider community on this list. * Write a blog post about your plugin, what makes it useful, etc. * Tweet about it. Feel free to mention [@RIPE_Atlas](https://twitter.com/ripe_atlas) and we might even retweet it. * Create a [pull request](https://github.com/RIPE-NCC/ripe-atlas-tools/pulls) for this project to get your plugin added to core. ripe-atlas-tools-1.2.2/docs/quickstart.rst000066400000000000000000000027441264541652500206100ustar00rootroot00000000000000Quickstart ========== This is a very fast break down of everything you need to start using Ripe Atlas on the command line. Viewing public data is quick & easy, while creation is a little more complicated, since you need to setup your authorisation key. Viewing Public Data ------------------- 1. :ref:`Install ` the toolkit. 2. View help with: ``ripe-atlas --help`` 3. View a basic report for a public measurement: ``ripe-atlas report `` 4. View the live stream for a measurement: ``ripe-atlas stream `` 5. Get a list of probes in ASN 3333: ``ripe-atlas probes --asn 3333`` 6. Get a list of measurements with the word "wikipedia" in them: ``ripe-atlas measurements --search wikipedia`` Creating a Measurement ---------------------- 1. Log into `RIPE Atlas`_. If you don't have an account, you can create one there for free. 2. Visit the `API Keys`_ page and create a new key with the permission ``Create a new user defined measurement`` 3. Install the toolkit as below. 4. Configure the toolkit to use your key with ``ripe-atlas configure --set authorisation.create=MY_API_KEY`` 5. View the help for measurement creation with ``ripe-atlas measure --help`` 6. Create a measurement with ``ripe-atlas measure ping --target example.com`` .. _`RIPE Atlas`: https://atlas.ripe.net/ .. _`API Keys`: https://atlas.ripe.net/keys/ Advanced Use ------------ Refer to the :ref:`complete usage documentation ` for more advanced options. ripe-atlas-tools-1.2.2/docs/troubleshooting.rst000066400000000000000000000057461264541652500216520ustar00rootroot00000000000000.. _troubleshooting: Troubleshooting =============== Sometimes things don't go as planned. In these cases, this page is here to help. .. _troubleshooting-insecureplatformwarning: InsecurePlatformWarning ----------------------- On older systems (running Python versions <2.7.10), you may be presented with a warning message that looks like this:: /path/to/lib/python2.7/site-packages/requests/packages/urllib3/util/ssl_.py:100: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning. InsecurePlatformWarning This is due to the insecure way older versions of Python handle secure connections and a visit to the above URL will tell you that the fix is one of three options: * Upgrade to a modern version of Python * Install three Python packages: ``pyopenssl``, ``ndg-httpsclient``, and ``pyasn1`` * `Suppress the warnings`_. Don't do that though. .. _Suppress the warnings: https://urllib3.readthedocs.org/en/latest/security.html#disabling-warnings .. _troubleshooting-saganopensslosx: Sagan, OpenSSL, and OSX ----------------------- If you're using Mac OSX, the installation of Sagan, (one of Magellan's dependencies) may give you trouble, especially in how Apple handles PyOpenSSL on their machines. Workarounds and proper fixes for this issue can be found in the `Sagan installation documentation`_. .. _Sagan installation documentation: https://ripe-atlas-sagan.readthedocs.org/en/latest/installation.html#troubleshooting .. _troubleshooting-libyaml: Complaints from libyaml ----------------------- During the installation, you may see something like this scroll by: Running setup.py install for pyyaml checking if libyaml is compilable x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/python2.7 -c build/temp.linux-x86_64-2.7/check_libyaml.c -o build/temp.linux-x86_64-2.7/check_libyaml.o build/temp.linux-x86_64-2.7/check_libyaml.c:2:18: fatal error: yaml.h: No such file or directory #include ^ compilation terminated. libyaml is not found or a compiler error: forcing --without-libyaml (if libyaml is installed correctly, you may need to specify the option --include-dirs or uncomment and modify the parameter include_dirs in setup.cfg) Don't worry. This is just the installation script noticing that you don't have libyaml installed and it's complaining because it's good to have around for performance reasons. However, since we're only using YAML for configuration, performance isn't an issue, and the fallback option will be sufficient. If however, you don't like these sorts of errors, make sure that libyaml is installed for your distribution before attempting to install this toolkit. ripe-atlas-tools-1.2.2/docs/use.rst000066400000000000000000000744601264541652500172160ustar00rootroot00000000000000.. _use: How to Use the RIPE Atlas Toolkit ********************************* .. _use-configure: Configuration ============= For most features, Magellan will work out-of-the-box, but if you'd like to customise the experience, or if you want to use this tool to create a measurement of your own, then you'll need to configure it. Thankfully, configuration is easy by way of the ``configure`` command::: $ ripe-atlas configure --help .. _use-configure-options: Options ------- ================== ================= ======================================== Option Arguments Explanation ================== ================= ======================================== ``--editor`` Invoke ${EDITOR} to edit the configuration directly ``--set`` path=value Permanently set a configuration value so it can be used in the future. ``--init`` Create a configuration file and save it into your home directory at: ``${HOME}/.config/ripe-atlas-tools/rc`` ================== ================= ======================================== .. _use-configure-examples: Examples -------- Create a standard configuration file. Note that this typically isn't necessary:: $ ripe-atlas configure --init Invoke your editor of choice to manually fiddle with the configuration file:: $ ripe-atlas configure --editor Set an arbitrary value within the configuration file. You can use dot-separated notation to dictation the value you wish to change:: $ ripe-atlas configure --set authorisation.create=YOUR_API_KEY .. _use-go: Quick Measurement Information ============================= For the impatient, and for those looking to see how they might write their own plugins, we have a simple ``go`` command::: $ ripe-atlas go This will open a web browser and take you to the detail page for the measurement id provided. .. _use-measurements: Measurement Querying ==================== A querying tool for finding existing measurements in the RIPE Atlas database. You can request a table-formatted list of measurements based on search-string lookups, type, start time, etc. .. _use-measurements-options: Options ------- ============================ ================== ============================== Option Arguments Explanation ============================ ================== ============================== ``--search`` A free-form string This could match the target or description. ``--status`` One of: scheduled, The measurement status. stopped, ongoing ``--af`` One of: 4, 6 The address family. ``--type`` One of: ping, The measurement type. traceroute, dns, sslcert, ntp, http ``--field`` One of: status, The field(s) to display. target, url, type, Invoke multiple times for id, description multiple fields. The default is id, type, description, and status. ``--ids-only`` Display a list of measurement ids matching your filter criteria. ``--limit`` An integer The number of measurements to return. The number must be between 1 and 1000 ``--started-before`` An ISO timestamp Filter for measurements that started before a specific date. The format required is YYYY-MM-DDTHH:MM:SS ``--started-after`` An ISO timestamp Filter for measurements that started after a specific date. The format required is YYYY-MM-DDTHH:MM:SS ``--stopped-before`` An ISO timestamp Filter for measurements that stopped before a specific date. The format required is YYYY-MM-DDTHH:MM:SS ``--stopped-after`` An ISO timestamp Filter for measurements that stopped after a specific date. The format required is YYYY-MM-DDTHH:MM:SS ============================ ================== ============================== .. _use-measurements-examples: Examples -------- Get a list of measurements:: $ ripe-atlas measurements Filter that list by ``status=ongoing``:: $ ripe-atlas measurements --status ongoing Further filter it by getting measurements that conform to IPv6:: $ ripe-atlas measurements --status ongoing --af 6 Get that same list, but strip out everything but the measurement ids:: $ ripe-atlas measurements --status ongoing --af 6 --ids-only Limit that list to 200 entries:: $ ripe-atlas measurements --status ongoing --af 6 --limit 200 Get that list, but show only the id, url and target fields: $ ripe-atlas measurements --status ongoing --af 6 \ --field id --field url --field target Filter for measurements of type ``dns`` that started after January 1, 2015:: $ ripe-atlas measurements --type dns --started-after 2015-01-01 .. _use-probes: Probe Querying ============== Just like the ``measurements`` command, but for probes, and a lot more powerful. You can use this command to find probes within an ASN, prefix, or geographical region, and then aggregate by country, ASN, and/or prefix. .. _use-probes-options: Options ------- ============================ ================== ============================== Option Arguments Explanation ============================ ================== ============================== ``--limit`` An integer Return limited number of probes. ``--field`` One of: status, The field(s) to display. description, Invoke multiple times for address_v6, multiple fields. The default address_v4, is id, asn_v4, asn_v6, asn_v4, is_public, country, and status. asn_v6, id, prefix_v4, prefix_v6, is_anchor, country, coordinates ``--aggregate-by`` country, asn_v4, Aggregate list of probes based asn_v6, on all specified aggregations. prefix_v4, Multiple aggregations prefix_v6 supported. ``--all`` Fetch *ALL* probes. That will give you a loooong list. ``--max-per-aggregation`` An integer Maximum number of probes per aggregated bucket. ``--ids-only`` Print only IDs of probes. Useful to pipe it to another command. ``--asn`` An integer Filter the list by an ASN ``--asnv4`` An integer Filter the list by an ASN ``--asnv6`` An integer Filter the list by an ASN ``--prefix`` A prefix string Filter the list by a prefix ``--prefixv4`` A prefix string Filter the list by a prefix ``--prefixv6`` A prefix string Filter the list by a prefix ``--location`` A free-form string The location of probes as a string i.e. 'Amsterdam' ``--center`` A pair of Location as geographic ,-string, i.e. coordinates "48.45,9.16" ``--radius`` An integer Radius in km from specified center/point. ``--country`` A two-letter The country in which the ISO country code probes are located. ============================ ================== ============================== .. _use-probes-examples: Examples -------- Get a list of probes within ASN 3333:: $ ripe-atlas probes --asn 3333 Further filter that list to show only probes in ASN 3333 from the Netherlands:: $ ripe-atlas probes --asn 3333 --country nl Change the limit from the default of 25 to 200:: $ ripe-atlas probes --asn 3333 --limit 200 Aggregate the probes by country, and then by ASN:: $ ripe-atlas probes --asn 3333 --aggregate-by country --aggregate-by asn Show the id, url, target, description, and whether the probe is public or not:: $ ripe-atlas probes --asn 3333 --field id --field url --field description \ --field is_public .. _use-report: Result Reporting ================ A means to generate a simple text-based report based on the results from a measurement. Typically, this is used to get the latest results of a measurement in a human-readable format, but with the ``--start-time`` and ``--stop-time`` options, you can get results from any time range you like. .. _use-report-options: Options ------- ================== ================== ======================================== Option Arguments Explanation ================== ================== ======================================== ``--probes`` A comma-separated Limit the report to only results list of probe ids obtained from specific probes. ``--probe-asns`` A comma-separated Limit the report to only results list of ASNs obtained from probes belonging to specific ASNs. ``--renderer`` One of: dns, http, The renderer you want to use. If this ntp, ping, raw, isn't defined, an appropriate renderer ssl_consistency, will be selected. sslcert, traceroute, traceroute_aspath, aggregate_ping ``--aggregate-by`` One of: status, Tell the rendering engine to aggregate prefix_v4, the results by the selected option. Note prefix_v6, that if you opt for aggregation, no country, output will be generated until all rtt-median, results are received. asn_v4, asn_v6 ``--start-time`` An ISO timestamp The start time of the report. The format should conform to YYYY-MM-DDTHH:MM:SS ``--stop-time`` An ISO timestamp The stop time of the report. The format should conform to YYYY-MM-DDTHH:MM:SS ================== ================== ======================================== .. _use-report-examples: Examples -------- Get the latest results of measurement 1001:: $ ripe-atlas report 1001 The same, but specifically request the ping renderer:: $ ripe-atlas report 1001 --renderer ping Aggregate those results by country:: $ ripe-atlas report 1001 --aggregate-by country Get results from the same measurement, but show all results from the first week of 2015:: $ ripe-atlas report 1001 --start-time 2015-01-01 --stop-time 2015-01-07 Get results from the first day of 2015 until right now:: $ ripe-atlas report 1001 --start-time 2015-01-01 .. _use-stream: Result Streaming ================ Connect to the streaming API and render the results in real-time as they come in. .. _use-stream-options: Options ------- ================== ================== ======================================== Option Arguments Explanation ================== ================== ======================================== ``--limit`` A number < 1000 The maximum number of results you want to stream. The default is to stream forever until you hit ``Ctrl+C``. ``--renderer`` One of: dns, http, The renderer you want to use. If this ntp, ping, raw, isn't defined, an appropriate renderer ssl_consistency, will be selected. sslcert, traceroute, traceroute_aspath, aggregate_ping ================== ================== ======================================== .. _use-stream-examples: Examples -------- Stream the results from measurement #1001:: $ ripe-atlas stream 1001 Limit those results to 500:: $ ripe-atlas stream 1001 --limit 500 Specify a renderer:: $ ripe-atlas stream 1001 --renderer ping Combine for fun and profit:: $ ripe-atlas stream 1001 --renderer ping --limit 500 .. _use-render: Result Rendering ================ Sometimes you already have a large collection of measurement results and you just want Magellan to render them nicely for you. In these cases, ``render`` is your friend. You can use the ``--renderer`` flag to target specific renderers too if the default isn't enough for you. .. _use-render-options: Options ------- ================== ================== ======================================== Option Arguments Explanation ================== ================== ======================================== ``--renderer`` One of: dns, http, The renderer you want to use. If this ntp, ping, raw, isn't defined, an appropriate renderer ssl_consistency, will be selected. sslcert, traceroute, traceroute_aspath, aggregate_ping ``--probes`` A comma-separated Limit the results to those returned from list of probe ids specific probes ``--from-file`` A file path The source of the data to be rendered. If nothing is specified, we assume "-" or, standard in (the default). ``--aggregate-by`` One of: country, Tell the rendering engine to aggregate asn_v4, asn_v6, the results by the selected option. Note prefix_v4, that if you opt for aggregation, no prefix_v6 output will be generated until all results are received, and if large data sets may explode your system. ================== ================== ======================================== .. _use-render-examples: Examples -------- Pipe the contents of an arbitrary file file into the renderer. The rendering engine will be guessed from the first line of input:: $ cat /path/to/file/full/of/results | ripe-atlas render The same, but point Magellan to a file deliberately rather than using a pipe:: $ ripe-atlas render --from-file /path/to/file/full/of/results Specify a particular renderer:: $ cat /path/to/file/full/of/results | ripe-atlas render --renderer ping Aggregate the output by country:: $ cat /path/to/file/full/of/results | ripe-atlas render --aggregate-by country .. _use-measure: Measurement Creation ==================== The most complicated command we have, this will create a measurement (given a plethora of options) and begin streaming the results back to you in a standardised rendered form. It's invoked by using a special positional argument that dictates the type of measurement you want to create. This also unlocks special options, specific to that type. See the :ref:`examples ` for more information. .. _use-measure-options: Options ------- All measurements share a base set of options. ====================== ================== ==================================== Option Arguments Explanation ====================== ================== ==================================== ``--renderer`` One of: dns, http, The renderer you want to use. If ntp, ping, raw, this isn't defined, an appropriate ssl_consistency, renderer will be selected. sslcert, traceroute, traceroute_aspath, aggregate_ping ``--dry-run`` Do not create the measurement, only show its definition. ``--auth`` An API key The API key you want to use to create the measurement. ``--af`` One of: 4, 6 The address family, either 4 or 6. The default is a guess based on the target, favouring 6. ``--description`` A free-form string The description/name of your new measurement. ``--target`` A domain or IP The target, either a domain name or IP address. If creating a DNS measurement, the absence of this option will imply that you wish to use the probe's resolver. ``--no-report`` Don't wait for a response from the measurement, just return the URL at which you can later get information about the measurement. ``--interval`` An integer Rather than run this measurement as a one-off (the default), create this measurement as a recurring one, with an interval of n seconds between attempted measurements. This option implies ``--no-report``. ``--from-area`` One of: WW, West, The area from which you'd like to North-Central, select your probes. South-Central, North-East, South-East ``--from-country`` A two-letter ISO The country from which you'd like to country code select your probes. ``--from-prefix`` A prefix string The prefix from which you'd like to select your probes. ``--from-asn`` An ASN number The ASN from which you'd like to select your probes. ``--from-probes`` A comma-separated Probes you want to use in your list of probe ids measurement. ``--from-measurement`` A measurement id A measurement id which you want to use as the basis for probe selection in your new measurement. This is a handy way to re-create a measurement under conditions similar to another measurement. ``--probes`` An integer The number of probes you want to use. ``--include-tag`` A tag name Include only probes that are marked with this tag. Note that this option may be repeated. ``--exclude-tag`` A tag name Exclude probes that are marked with this tag. Note that this option may be repeated. ====================== ================== ==================================== .. _use-measure-options-ping: Ping-Specific Options ::::::::::::::::::::: ====================== ================== ==================================== Option Arguments Explanation ====================== ================== ==================================== ``--packets`` An integer The number of packets sent ``--size`` An integer The size of packets sent ``--packet-interval`` An integer ====================== ================== ==================================== .. _use-measure-options-traceroute: Traceroute-Specific Options ::::::::::::::::::::::::::: ============================= ================== ==================================== Option Arguments Explanation ============================= ================== ==================================== ``--packets`` An integer The number of packets sent ``--size`` An integer The size of packets sent ``--protocol`` One of: ICMP, UDP, The protocol used. For DNS TCP measurements, this is limited to UDP and TCP, but traceroutes may use ICMP as well. ``--timeout`` An integer The timeout per-packet ``--dont-fragment`` Don't Fragment the packet ``--paris`` An integer Use Paris. Value must be between 0 and 64.If 0, a standard traceroute will be performed. ``--first-hop`` An integer Value must be between 1 and 255. ``--max-hops`` An integer Value must be between 1 and 255. ``--port`` An integer Destination port, valid for TCP only. ``--destination-option-size`` An integer IPv6 destination option header. ``--hop-by-hop-option-size`` An integer IPv6 hop by hop option header. ============================= ================== ==================================== .. _use-measure-options-dns: DNS-Specific Options :::::::::::::::::::: ============================ ================== ============================== Option Arguments Explanation ============================ ================== ============================== ``--query-class`` One of: IN, CHAOS The query class. The default is "IN" ``--query-type`` One of: A, SOA, The query type. The default TXT, SRV, SSHFP, is "A" TLSA, NSEC, DS, AAAA, CNAME, DNSKEY, NSEC3, PTR, HINFO, NSEC3PARAM, NS, MX, RRSIG, ANY ``--query-argument`` A string The DNS label to query. ``--set-cd-bit`` Set the DNSSEC Checking Disabled flag (RFC4035) ``--set-do-bit`` Set the DNSSEC OK flag (RFC3225) ``--set-nsid-bit`` Include an EDNS name server. ID request with the query. ``--udp-payload-size`` An integer May be any integer between 512 and 4096 inclusive. ``--set-rd-bit`` Set the Recursion Desired flag. ``--retry`` An integer Number of times to retry. ============================ ================== ============================== .. _use-measure-options-sslcert: SSL Certificate-Specific Options :::::::::::::::::::::::::::::::: ============================ ================== ============================== Option Arguments Explanation ============================ ================== ============================== ``--port`` An integer The port to query ============================ ================== ============================== .. _use-measure-options-http: HTTP-Specific Options ::::::::::::::::::::: ============================ ================== ============================== Option Arguments Explanation ============================ ================== ============================== ``--header-bytes`` An integer The maximum number of bytes to retrieve from the header ``--version`` A string The HTTP version to use ``--method`` A string The HTTP method to use ``--path`` A string The path on the webserver ``--query-string`` A string An arbitrary query string ``--user-agent`` A string An arbitrary user agent ``--body-bytes`` An integer The maximum number of bytes to retrieve from the body ``--timing-verbosity`` One of: 0, 1, 2 The amount of timing information you want returned. 1 returns the time to read, to connect, and to first byte, 2 returns timing information per read system call. 0 (default) returns no additional timing information. ============================ ================== ============================== .. _use-measure-options-ntp: NTP-Specific Options :::::::::::::::::::: ============================ ================== ============================== Option Arguments Explanation ============================ ================== ============================== ``--packets`` An integer The number of packets sent ``--timeout`` An integer The timeout per-packet ============================ ================== ============================== .. _use-measure-examples: Examples -------- The simplest of measurements. Create a ping with 50 probes to example.com:: $ ripe-atlas measure ping --target example.com The same, but don't actually create it, just show what would be done:: $ ripe-atlas measure ping --target example.com --dry-run Be more specific about which address family you want to target:: $ ripe-atlas measure ping --target example.com --af 6 Ask for 20 probes from Canada:: $ ripe-atlas measure ping --target example.com --probes 20 --from-country ca Or ask for 20 Canadian probes that definitely support IPv6:: $ ripe-atlas measure ping --target example.com --probes 20 \ --from-country ca --include-tag system-ipv6-works Rather than creating a one-off create a recurring measurement:: $ ripe-atlas measure ping --target example.com --interval 3600 Moving onto DNS measurements, do a lookup for example.com. Since we're not specifying ``--target`` here, this implies that we want to use the probe's resolver:: $ ripe-atlas measure dns --query-argument example.com Getting a little more complicated, let's set a few special bits and make a more complex query:: $ ripe-atlas measure dns --query-type AAAA --query-argument example.com \ --set-nsid-bit --set-rd-bit --set-do-bit --set-cd-bit .. _use-shortcuts: Shortcuts --------- If you're creating a lot of measurements in a short time, typing out ``ripe-atlas measure traceroute`` a whole bunch of times can be tiresome, so we've added a few shortcut scripts for you: ================================= ========================== Where you'd typically write You could use this instead ================================= ========================== ``ripe-atlas measure ping`` ``aping`` ``ripe-atlas measure traceroute`` ``atraceroute`` ``ripe-atlas measure dns`` ``adig`` ``ripe-atlas measure sslcert`` ``asslcert`` ``ripe-atlas measure http`` ``ahttp`` ``ripe-atlas measure ntp`` ``antp`` ================================= ========================== So for example, these two commands are the same:: $ ripe-atlas measure ping --target example.com $ aping --target example.com If you want to streamline your typing process even more than this, we recommend the use of your shell's ``alias`` feature, which is both powerful and customisable for your needs. ripe-atlas-tools-1.2.2/ripe/000077500000000000000000000000001264541652500156645ustar00rootroot00000000000000ripe-atlas-tools-1.2.2/ripe/__init__.py000066400000000000000000000015031264541652500177740ustar00rootroot00000000000000# Copyright (c) 2015 RIPE NCC # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . try: __import__('pkg_resources').declare_namespace(__name__) except ImportError: from pkgutil import extend_path __path__ = extend_path(__path__, __name__) ripe-atlas-tools-1.2.2/ripe/atlas/000077500000000000000000000000001264541652500167705ustar00rootroot00000000000000ripe-atlas-tools-1.2.2/ripe/atlas/__init__.py000066400000000000000000000015031264541652500211000ustar00rootroot00000000000000# Copyright (c) 2015 RIPE NCC # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . try: __import__('pkg_resources').declare_namespace(__name__) except ImportError: from pkgutil import extend_path __path__ = extend_path(__path__, __name__) ripe-atlas-tools-1.2.2/ripe/atlas/tools/000077500000000000000000000000001264541652500201305ustar00rootroot00000000000000ripe-atlas-tools-1.2.2/ripe/atlas/tools/__init__.py000066400000000000000000000000001264541652500222270ustar00rootroot00000000000000ripe-atlas-tools-1.2.2/ripe/atlas/tools/aggregators/000077500000000000000000000000001264541652500224355ustar00rootroot00000000000000ripe-atlas-tools-1.2.2/ripe/atlas/tools/aggregators/__init__.py000066400000000000000000000014631264541652500245520ustar00rootroot00000000000000# Copyright (c) 2015 RIPE NCC # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . from .base import RangeKeyAggregator, ValueKeyAggregator, aggregate __all__ = [ "aggregate", "RangeKeyAggregator", "ValueKeyAggregator", ] ripe-atlas-tools-1.2.2/ripe/atlas/tools/aggregators/base.py000066400000000000000000000067421264541652500237320ustar00rootroot00000000000000# Copyright (c) 2015 RIPE NCC # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . from ..helpers.rendering import SaganSet class ValueKeyAggregator(object): """Aggregator based on tha actual value of the key/attribute""" def __init__(self, key, prefix=None): self.aggregation_keys = key.split('.') self.key_prefix = prefix or self.aggregation_keys[-1].upper() def get_key_value(self, entity): """ Returns the value of the key/attribute the aggregation will use to bucketize probes/results """ attribute = entity for key in self.aggregation_keys: attribute = getattr(attribute, key) return attribute def get_bucket(self, entity): """ Returns the bucket the specific entity belongs to based on the give key/attribute """ return "{0}: {1}".format(self.key_prefix, self.get_key_value(entity)) def insert2bucket(self, buckets, bucket, entity): if bucket in buckets: buckets[bucket].append(entity) else: buckets[bucket] = [entity] class RangeKeyAggregator(ValueKeyAggregator): """ Aggregator based on where the position of the value of the key/attribute is in the given range """ def __init__(self, key, ranges): ValueKeyAggregator.__init__(self, key) self.aggregation_ranges = sorted(ranges, reverse=True) def get_bucket(self, entity): """ Returns the bucket the specific entity belongs to based on the give key/attribute """ bucket = "{0}: < {1}".format( self.key_prefix, self.aggregation_ranges[-1]) key_value = self.get_key_value(entity) for index, krange in enumerate(self.aggregation_ranges): if key_value > krange: if index == 0: bucket = "{0}: > {1}".format(self.key_prefix, krange) else: bucket = "{0}: {1}-{2}".format( self.key_prefix, krange, self.aggregation_ranges[index - 1] ) break return bucket def aggregate(entities, aggregators): """ This is doing the len(aggregators) level aggregation of the entities. Caution: being recursive is a bit hard to read/understand, if you change something make sure you run tests. """ if not aggregators: return entities if isinstance(entities, (list, SaganSet)): aggregator = aggregators.pop(0) buckets = {} for entity in entities: bucket = aggregator.get_bucket(entity) aggregator.insert2bucket(buckets, bucket, entity) return aggregate(buckets, aggregators) elif isinstance(entities, dict): for k, v in entities.items(): entities[k] = aggregate(entities[k], aggregators[:]) return entities ripe-atlas-tools-1.2.2/ripe/atlas/tools/cache.py000066400000000000000000000104421264541652500215460ustar00rootroot00000000000000# Copyright (c) 2015 RIPE NCC # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . try: import cPickle as pickle except ImportError: import pickle import datetime import dbm import functools import os import sys class LocalCache(object): """ Simple caching engine, making use of the built-in dbm support. This will create a file called cache.db in ripe-atlas-tools config directory and dump stuff in there for use later. """ def __init__(self): self._now = datetime.datetime.now() self._db = dbm.open(self._get_or_create_db_path(), "c") def __contains__(self, key): return key in self._db def __getitem__(self, key): return self.get(key) def __setitem__(self, key, value, expires=None): self._db[key] = pickle.dumps((expires, value)) def __delitem__(self, key): if key not in self._db: raise KeyError del self._db[key] def keys(self): return self._db.keys() def items(self): for key in self.keys(): yield key, self._db[key] def get(self, key, default=None): if key in self._db: expires, value = pickle.loads(self._db[key]) if not expires or expires > self._now: return value else: del(self._db[key]) return default def set(self, key, value, expires=None): return self.__setitem__( key, value, self._now + datetime.timedelta(seconds=expires)) def clear(self, key=None): """ Removes a specific key from the cache manually, or will wipe the entire cache if you don't specify `key`. Note that this shouldn't be necessary unless you've cached something with an inappropriately long expire time. """ if key: if key in self._db: del(self._db[key]) else: for key in self.keys(): del(self._db[key]) def expire(self): """ Clears out should-be-expired values from the cache. Note that this happens automatically whenever you call `.get()` so you should never really need to run this. """ for key in self.keys(): self.get(key) @staticmethod def _get_or_create_db_path(): v = sys.version_info file_name = "cache-{}.{}.{}.db".format(v.major, v.minor, v.micro) db_path = os.path.join("/", "tmp", file_name) if "HOME" in os.environ: db_path = os.path.join( os.environ["HOME"], ".config", "ripe-atlas-tools", file_name) try: os.makedirs(os.path.dirname(db_path)) except OSError: pass # Better to ask forgiveness than permission return db_path cache = LocalCache() class Memoiser(object): """ Enabling class for the @memoised decorator """ def __init__(self, function, cache_time): self._function = function self._cache_time = cache_time def __call__(self, *args, **kwargs): key = pickle.dumps([args, kwargs]) value = cache[key] if value: return value value = self._function(*args, **kwargs) cache.set(key, value, self._cache_time) return value def __get__(self, obj, objtype): """Support instance methods.""" return functools.partial(self.__call__, obj) def memoised(cache_time): """ Decorate a method or function with this to cache the result of said method for n seconds: @memoised(60 * 60) # Cache for one hour my_function(some_arguments): ... return whatever """ def _wrap(function): return Memoiser(function, cache_time=cache_time) return _wrap ripe-atlas-tools-1.2.2/ripe/atlas/tools/commands/000077500000000000000000000000001264541652500217315ustar00rootroot00000000000000ripe-atlas-tools-1.2.2/ripe/atlas/tools/commands/__init__.py000066400000000000000000000000001264541652500240300ustar00rootroot00000000000000ripe-atlas-tools-1.2.2/ripe/atlas/tools/commands/base.py000066400000000000000000000136431264541652500232240ustar00rootroot00000000000000# Copyright (c) 2015 RIPE NCC # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . import argparse import os import re import six import sys from datetime import datetime from ..helpers.colours import colourise from ..version import __version__ class RipeHelpFormatter(argparse.RawTextHelpFormatter): def _format_usage(self, *args): r = argparse.RawTextHelpFormatter._format_usage( self, *args).capitalize() return "\n\n{}\n".format(r) class Command(object): NAME = "" DESCRIPTION = "" # Define this in the subclass def __init__(self, *args, **kwargs): self.arguments = None self.parser = argparse.ArgumentParser( formatter_class=RipeHelpFormatter, description=self.DESCRIPTION, prog="ripe-atlas {}".format(self.NAME) ) self.user_agent = self._get_user_agent() def init_args(self, args=None): """ Initialises all parse arguments and makes them available to the class. """ if args is None: args = sys.argv[1:] self.arguments = self.parser.parse_args( self._modify_parser_args(args)) def run(self): raise NotImplemented() def add_arguments(self): """ A hook that's executed in the __init__, you can make use of `self.parser` here to add arguments to the command: self.parser.add_argument( "measurement_id", type=int, help="The measurement id you want to use" ) """ pass def _modify_parser_args(self, args): """ A modifier hook that can be overridden in the child class to allow that class to manipulate the arguments before being parsed. The common use-case we're trying to solve here is popping a secondary argument off of the list and/or appending `--help` in some circumstances. """ self.add_arguments() return args def ok(self, message): sys.stdout.write("\n{}\n\n".format(colourise(message, "green"))) @staticmethod def _get_user_agent(): """ Allow packagers to change the user-agent to whatever they like by placing a file called `user-agent` into the `tools` directory. If no file is found, we go with a sensible default + the version. """ try: custom = os.path.join(os.path.dirname(__file__), "..", "user-agent") with open(custom) as f: return f.readline().strip()[:128] except IOError: pass # We go with the default return "RIPE Atlas Tools (Magellan) {}".format(__version__) class TabularFieldsMixin(object): """ A handy mixin to dump into classes that are expected to render tabular data. It expects both that COLUMNS is defined by the subclass and that --field is set in the add_arguments() method. """ def _get_line_format(self): """ Loop over the field arguments and generate a string that makes use of Python's string format mini language. We later use this string to format the values for each row. """ r = u"" for field in self.arguments.field: if r: r += u" " r += (u"{!s:" + u"{}{}".format(*self.COLUMNS[field]) + u"}") return r def _get_header_names(self): return [_.capitalize() for _ in self.arguments.field] def _get_header(self): """ Generates a header by using the line formatter and the list of field arguments. """ return self._get_line_format().format(*self._get_header_names()) def _get_horizontal_rule(self): """ A bit of a hack: We get a formatted line for no other reason than to determine the width of that line. Then we use a regex to overwrite that line with "=". """ return re.sub( r".", "=", self._get_line_format().format(*self.arguments.field)) def _get_line_items(self, measurement): raise NotImplementedError("This needs to be defined in the subclass.") def _get_filter_display(self, filters): if not filters: return "" r = colourise("\nFilters:\n", "white") for k, v in filters.items(): if k not in ("search",): v = str(v).capitalize() r += colourise( " {}: {}\n".format(*self._get_filter_key_value_pair(k, v)), "cyan" ) return r def _get_filter_key_value_pair(self, k, v): return k.capitalize().replace("__", " "), v class MetaDataMixin(object): @staticmethod def _prettify_boolean(boolean): checkmark = u"\u2714" x = u"\u2718" if six.PY2: checkmark = checkmark.encode("utf-8") x = x.encode("utf-8") if boolean: return colourise(checkmark, "green") return colourise(x, "red") @staticmethod def _prettify_time(timestamp): return "{} UTC".format( datetime.fromtimestamp(timestamp).isoformat().replace("T", " ")) @staticmethod def _render_line(header, value): print("{} {}".format( colourise("{:25}".format(header), "bold"), value)) class Factory(object): @classmethod def build(cls, *args, **kwargs): return object() ripe-atlas-tools-1.2.2/ripe/atlas/tools/commands/configure.py000066400000000000000000000102641264541652500242670ustar00rootroot00000000000000# Copyright (c) 2015 RIPE NCC # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . import functools import os from ..exceptions import RipeAtlasToolsException from ..settings import Configuration, conf from .base import Command as BaseCommand class Command(BaseCommand): NAME = "configure" EDITOR = os.environ.get("EDITOR", "/usr/bin/vim") DESCRIPTION = "An easy way to configure this tool. Alternatively, you " \ "can always just create/edit {}".format(Configuration.USER_RC) def add_arguments(self): self.parser.add_argument( "--editor", action="store_true", help="Invoke {0} to edit the configuration directly".format( self.EDITOR) ) self.parser.add_argument( "--set", action="store", help="Permanently set a configuration value so it can be used in " "the future. Example: --set authorisation.create=MY_API_KEY" ) self.parser.add_argument( "--init", action="store_true", help="Create a configuration file and save it into your home " "directory at: {}".format(Configuration.USER_RC) ) def run(self): if not self.arguments.init: if not self.arguments.editor: if not self.arguments.set: raise RipeAtlasToolsException( "Run this with --help for more information") self._create_if_necessary() if self.arguments.editor: os.system("{0} {1}".format(self.EDITOR, Configuration.USER_RC)) if self.arguments.init or self.arguments.editor: return self.ok( "Configuration file writen to {}".format(Configuration.USER_RC)) if self.arguments.set: if "=" not in self.arguments.set: raise RipeAtlasToolsException( "Invalid format. Execute with --help for more information.") path, value = self.arguments.set.split("=") self.set(path.split("."), value) def set(self, path, value): try: required_type = type(self._get_from_dict(conf, path)) except KeyError: raise RipeAtlasToolsException( 'Invalid configuration key: "{}"'.format(".".join(path))) if value.isdigit(): value = int(value) if not isinstance(value, required_type): raise RipeAtlasToolsException( 'Invalid configuration value: "{}". You must supply a {} for ' 'this key'.format(value, required_type.__name__) ) self._set_in_dict(conf, path, value) Configuration.write(conf) @staticmethod def _create_if_necessary(): if os.path.exists(Configuration.USER_RC): return if not os.path.exists(Configuration.USER_CONFIG_DIR): os.makedirs(Configuration.USER_CONFIG_DIR) Configuration.write(conf) @staticmethod def _get_from_dict(data, path): return functools.reduce(lambda d, k: d[k], path, data) @classmethod def _set_in_dict(cls, data, path, value): cls._get_from_dict(data, path[:-1])[path[-1]] = value @staticmethod def cast_value(value): # Booleans are a pain in the ass to cast if value.lower() == "true": return True if value.lower() == "false": return False try: return int(value) except ValueError: try: return float(value) except ValueError: return str(value) ripe-atlas-tools-1.2.2/ripe/atlas/tools/commands/go.py000066400000000000000000000025441264541652500227150ustar00rootroot00000000000000# Copyright (c) 2015 RIPE NCC # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . import webbrowser from .base import Command as BaseCommand class Command(BaseCommand): NAME = "go" DESCRIPTION = "Visit the web page for a specific measurement" URL = "https://atlas.ripe.net/measurements/{0}/" def add_arguments(self): self.parser.add_argument( "measurement_id", type=int, help="The measurement id you want reported" ) def run(self): url = self.URL.format(self.arguments.measurement_id) if not webbrowser.open(url): self.ok( "It looks like your system doesn't have a web browser " "available. You'll have to go there manually: {0}".format(url) ) ripe-atlas-tools-1.2.2/ripe/atlas/tools/commands/measure/000077500000000000000000000000001264541652500233725ustar00rootroot00000000000000ripe-atlas-tools-1.2.2/ripe/atlas/tools/commands/measure/__init__.py000066400000000000000000000045241264541652500255100ustar00rootroot00000000000000# Copyright (c) 2015 RIPE NCC # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . from __future__ import print_function, absolute_import import sys from ...exceptions import RipeAtlasToolsException from ..base import Factory as BaseFactory from .ping import PingMeasureCommand from .traceroute import TracerouteMeasureCommand from .dns import DnsMeasureCommand from .sslcert import SslcertMeasureCommand from .http import HttpMeasureCommand from .ntp import NtpMeasureCommand class Factory(BaseFactory): TYPES = { "ping": PingMeasureCommand, "traceroute": TracerouteMeasureCommand, "dns": DnsMeasureCommand, "sslcert": SslcertMeasureCommand, "http": HttpMeasureCommand, "ntp": NtpMeasureCommand, } def __init__(self): self.build_class = None if len(sys.argv) >= 2: self.build_class = self.TYPES.get(sys.argv[1].lower()) if not self.build_class: self.raise_log() def raise_log(self): """Depending on the input raise with different log message.""" # cases: 1) ripe-atlas measure 2) ripe-atlas measure --help/-h if ( len(sys.argv) == 1 or (len(sys.argv) == 2 and sys.argv[1] in ("--help", "-h")) ): log = ( "For extended options for a specific measurement type, " "try ripe-atlas measure --help." ) # cases: ripe-atlas measure bla else: log = ( "The measurement type you requested is invalid. " "Please choose one of {}." ).format(", ".join(self.TYPES.keys())) raise RipeAtlasToolsException(log) def create(self, *args, **kwargs): return self.build_class(*args, **kwargs) ripe-atlas-tools-1.2.2/ripe/atlas/tools/commands/measure/base.py000066400000000000000000000341511264541652500246620ustar00rootroot00000000000000# Copyright (c) 2015 RIPE NCC # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . from __future__ import print_function, absolute_import import re from collections import OrderedDict from ripe.atlas.cousteau import ( Ping, Traceroute, Dns, Sslcert, Http, Ntp, AtlasSource, AtlasCreateRequest) from ...exceptions import RipeAtlasToolsException from ...helpers.colours import colourise from ...helpers.validators import ArgumentType from ...renderers import Renderer from ...settings import conf from ...streaming import Stream, CaptureLimitExceeded from ..base import Command as BaseCommand class Command(BaseCommand): NAME = "measure" DESCRIPTION = "Create a measurement and optionally wait for the results" CREATION_CLASSES = OrderedDict(( ("ping", Ping), ("traceroute", Traceroute), ("dns", Dns), ("sslcert", Sslcert), ("http", Http), ("ntp", Ntp) )) def __init__(self, *args, **kwargs): self._type = None self._is_oneoff = True BaseCommand.__init__(self, *args, **kwargs) def _modify_parser_args(self, args): kinds = self.CREATION_CLASSES.keys() error = ( "Usage: ripe-atlas measure <{}> [options]\n" "\n" " Example: ripe-atlas measure ping --target example.com" "".format("|".join(kinds)) ) if not args: raise RipeAtlasToolsException(error) if args[0] not in self.CREATION_CLASSES.keys(): raise RipeAtlasToolsException(error) self._type = args.pop(0) if not args: args.append("--help") return BaseCommand._modify_parser_args(self, args) def add_arguments(self): self.parser.add_argument( "--renderer", choices=Renderer.get_available(), help="The renderer you want to use. If this isn't defined, an " "appropriate renderer will be selected." ) self.parser.add_argument( "--dry-run", action="store_true", help="Do not create the measurement, only show its definition." ) # Standard for all types self.parser.add_argument( "--auth", type=str, default=conf["authorisation"]["create"], help="The API key you want to use to create the measurement" ) self.parser.add_argument( "--af", type=int, choices=(4, 6), help="The address family, either 4 or 6" ) self.parser.add_argument( "--description", type=str, default=conf["specification"]["description"], help="A free-form description" ) self.parser.add_argument( # Most types "--target", type=ArgumentType.ip_or_domain, help="The target, either a domain name or IP address. If creating " "a DNS measurement, the absence of this option will imply " "that you wish to use the probe's resolver." ) self.parser.add_argument( "--no-report", action="store_true", help="Don't wait for a response from the measurement, just return " "the URL at which you can later get information about the " "measurement." ) self.parser.add_argument( "--interval", type=int, help="Rather than run this measurement as a one-off (the default), " "create this measurement as a recurring one, with an interval " "of n seconds between attempted measurements. This option " "implies --no-report." ) origins = self.parser.add_mutually_exclusive_group() origins.add_argument( "--from-area", type=str, choices=("WW", "West", "North-Central", "South-Central", "North-East", "South-East"), help="The area from which you'd like to select your probes." ) origins.add_argument( "--from-country", type=ArgumentType.country_code, metavar="COUNTRY", help="The two-letter ISO code for the country from which you'd " "like to select your probes. Example: --from-country=GR" ) origins.add_argument( "--from-prefix", type=str, metavar="PREFIX", help="The prefix from which you'd like to select your probes. " "Example: --from-prefix=82.92.0.0/14" ) origins.add_argument( "--from-asn", # http://www.iana.org/assignments/as-numbers/as-numbers.xhtml type=ArgumentType.integer_range(1, 2 ** 32 - 2), metavar="ASN", help="The ASN from which you'd like to select your probes. " "Example: --from-asn=3333" ) origins.add_argument( "--from-probes", type=ArgumentType.comma_separated_integers(minimum=1), metavar="PROBES", help="A comma-separated list of probe-ids you want to use in your " "measurement. Example: --from-probes=1,2,34,157,10006" ) origins.add_argument( "--from-measurement", type=ArgumentType.integer_range(minimum=1), metavar="MEASUREMENT_ID", help="A measurement id which you want to use as the basis for " "probe selection in your new measurement. This is a handy " "way to re-create a measurement under conditions similar to " "another measurement. Example: --from-measurement=1000192" ) self.parser.add_argument( "--probes", type=ArgumentType.integer_range(minimum=1), default=None, help="The number of probes you want to use. Defaults to {}," "unless --from-probes is invoked, in which case the number of " "probes selected is used.".format(conf["specification"]["source"]["requested"]) ) self.parser.add_argument( "--include-tag", type=ArgumentType.regex(r"^[a-z_\-]+$"), action="append", metavar="TAG", help="Include only probes that are marked with these tags. " "Example: --include-tag=system-ipv6-works" ) self.parser.add_argument( "--exclude-tag", type=ArgumentType.regex(r"^[a-z_\-]+$"), action="append", metavar="TAG", help="Exclude probes that are marked with these tags. " "Example: --exclude-tag=system-ipv6-works" ) def run(self): self._account_for_selected_probes() if self.arguments.dry_run: return self.dry_run() is_success, response = self.create() if not is_success: self._handle_api_error(response) # Raises an exception pk = response["measurements"][0] url = "{0}/measurements/{1}/".format(conf["ripe-ncc"]["endpoint"], pk) self.ok( "Looking good! Your measurement was created and details about " "it can be found here:\n\n {0}".format(url) ) if not self.arguments.no_report: self.stream(pk, url) def dry_run(self): print(colourise("\nDefinitions:\n{}".format("=" * 80), "bold")) for param, val in self._get_measurement_kwargs().items(): print(colourise("{:<25} {}".format(param, val), "cyan")) print(colourise("\nSources:\n{}".format("=" * 80), "bold")) for param, val in self._get_source_kwargs().items(): if param == "tags": print(colourise("tags\n include{}{}\n exclude{}{}\n".format( " " * 17, ", ".join(val["include"]), " " * 17, ", ".join(val["exclude"]) ), "cyan")) continue print(colourise("{:<25} {}".format(param, val), "cyan")) def create(self): creation_class = self.CREATION_CLASSES[self._type] return AtlasCreateRequest( server=conf["ripe-ncc"]["endpoint"].replace("https://", ""), key=self.arguments.auth, user_agent=self.user_agent, measurements=[creation_class(**self._get_measurement_kwargs())], sources=[AtlasSource(**self._get_source_kwargs())], is_oneoff=self._is_oneoff ).create() def stream(self, pk, url): self.ok("Connecting to stream...") try: Stream(capture_limit=self.arguments.probes, timeout=300).stream( self.arguments.renderer, self._type, pk) except (KeyboardInterrupt, CaptureLimitExceeded): pass # User said stop, so we fall through to the finally block. finally: self.ok("Disconnecting from stream\n\nYou can find details " "about this measurement here:\n\n {0}".format(url)) def clean_target(self): if not self.arguments.target: raise RipeAtlasToolsException( "You must specify a target for that kind of measurement" ) return self.arguments.target def clean_description(self): if self.arguments.description: return self.arguments.description if conf["specification"]["description"]: return conf["specification"]["description"] return "{} measurement to {}".format( self._type.capitalize(), self.arguments.target) def _get_measurement_kwargs(self): # This is kept apart from the r = {} because dns measurements don't # require a target attribute target = self.clean_target() r = { "af": self._get_af(), "description": self.clean_description(), } spec = conf["specification"] # Shorter names are easier to read if self.arguments.interval or spec["times"]["interval"]: r["interval"] = self.arguments.interval self._is_oneoff = False self.arguments.no_report = True elif not spec["times"]["one-off"]: raise RipeAtlasToolsException( "Your configuration file appears to be setup to not create " "one-offs, but also offers no interval value. Without one of " "these, a measurement cannot be created." ) if target: r["target"] = target return r def _get_source_kwargs(self): r = conf["specification"]["source"] r["requested"] = self.arguments.probes if self.arguments.from_country: r["type"] = "country" r["value"] = self.arguments.from_country elif self.arguments.from_area: r["type"] = "area" r["value"] = self.arguments.from_area elif self.arguments.from_prefix: r["type"] = "prefix" r["value"] = self.arguments.from_prefix elif self.arguments.from_asn: r["type"] = "asn" r["value"] = self.arguments.from_asn elif self.arguments.from_probes: r["type"] = "probes" r["value"] = ",".join([str(_) for _ in self.arguments.from_probes]) elif self.arguments.from_measurement: r["type"] = "msm" r["value"] = self.arguments.from_measurement r["tags"] = { "include": self.arguments.include_tag or [], "exclude": self.arguments.exclude_tag or [] } af = "ipv{}".format(self._get_af()) kind = self._type spec = conf["specification"] for clude in ("in", "ex"): clude += "clude" if not r["tags"][clude]: r["tags"][clude] += spec["tags"][af][kind][clude] r["tags"][clude] += spec["tags"][af]["all"][clude] return r def _get_af(self): """ Returns the specified af, or a guessed one, or the configured one. In that order. """ if self.arguments.af: return self.arguments.af if self.arguments.target: if ":" in self.arguments.target: return 6 if re.match(r"^\d+\.\d+\.\d+\.\d+$", self.arguments.target): return 4 return conf["specification"]["af"] def _account_for_selected_probes(self): """ If the user has used --from-probes, there's a little extra magic we need to do. """ # We can't use argparse's mutually_exclusive_group() method here because # that library doesn't allow partial overlap. if self.arguments.from_probes and self.arguments.probes: raise RipeAtlasToolsException( "Explicit probe selection (--from-probes) in incompatible with " "a --probes argument." ) configured = conf["specification"]["source"]["requested"] if not self.arguments.probes: self.arguments.probes = configured if self.arguments.from_probes: self.arguments.probes = len(self.arguments.from_probes) @staticmethod def _handle_api_error(response): error_detail = response if isinstance(response, dict) and "detail" in response: error_detail = response["detail"] message = ( "There was a problem communicating with the RIPE Atlas " "infrastructure. The message given was:\n\n {}" ).format(error_detail) raise RipeAtlasToolsException(message) ripe-atlas-tools-1.2.2/ripe/atlas/tools/commands/measure/dns.py000066400000000000000000000116511264541652500245340ustar00rootroot00000000000000# Copyright (c) 2015 RIPE NCC # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . from __future__ import print_function, absolute_import from ripe.atlas.sagan.dns import Message from ...exceptions import RipeAtlasToolsException from ...helpers.validators import ArgumentType from ...settings import conf from .base import Command class DnsMeasureCommand(Command): def add_arguments(self): Command.add_arguments(self) specific = self.parser.add_argument_group("DNS-specific Options") specific.add_argument( "--protocol", type=str, choices=("UDP", "TCP"), default=conf["specification"]["types"]["dns"]["protocol"], help="The protocol used." ) specific.add_argument( "--query-class", type=str, choices=("IN", "CHAOS"), default=conf["specification"]["types"]["dns"]["query-class"], help='The query class. The default is "{}"'.format( conf["specification"]["types"]["dns"]["query-class"] ) ) specific.add_argument( "--query-type", type=str, choices=list(Message.ANSWER_CLASSES.keys()) + ["ANY"], # The only ones we can parse default=conf["specification"]["types"]["dns"]["query-type"], help='The query type. The default is "{}"'.format( conf["specification"]["types"]["dns"]["query-type"] ) ) specific.add_argument( "--query-argument", type=str, default=conf["specification"]["types"]["dns"]["query-argument"], help="The DNS label to query" ) specific.add_argument( "--set-cd-bit", action="store_true", default=conf["specification"]["types"]["dns"]["set-cd-bit"], help="Set the DNSSEC Checking Disabled flag (RFC4035)" ) specific.add_argument( "--set-do-bit", action="store_true", default=conf["specification"]["types"]["dns"]["set-do-bit"], help="Set the DNSSEC OK flag (RFC3225)" ) specific.add_argument( "--set-nsid-bit", action="store_true", default=conf["specification"]["types"]["dns"]["set-nsid-bit"], help="Include an EDNS name server ID request with the query" ) specific.add_argument( "--set-rd-bit", action="store_true", default=conf["specification"]["types"]["dns"]["set-rd-bit"], help="Set the Recursion Desired flag" ) specific.add_argument( "--retry", type=ArgumentType.integer_range(minimum=1), default=conf["specification"]["types"]["dns"]["retry"], help="Number of times to retry" ) specific.add_argument( "--udp-payload-size", type=ArgumentType.integer_range(minimum=1), default=conf["specification"]["types"]["dns"]["udp-payload-size"], help="May be any integer between 512 and 4096 inclusive" ) def clean_target(self): """ Targets aren't required for this type """ return self.arguments.target def clean_description(self): if self.arguments.target: return Command.clean_description(self) return "DNS measurement for {}".format(self.arguments.query_argument) def _get_measurement_kwargs(self): r = Command._get_measurement_kwargs(self) for opt in ("class", "type", "argument"): if not getattr(self.arguments, "query_{0}".format(opt)): raise RipeAtlasToolsException( "At a minimum, DNS measurements require a query argument.") r["query_class"] = self.arguments.query_class r["query_type"] = self.arguments.query_type r["query_argument"] = self.arguments.query_argument r["set_cd_bit"] = self.arguments.set_cd_bit r["set_do_bit"] = self.arguments.set_do_bit r["set_rd_bit"] = self.arguments.set_rd_bit r["set_nsid_bit"] = self.arguments.set_nsid_bit r["protocol"] = self.arguments.protocol r["retry"] = self.arguments.retry r["udp_payload_size"] = self.arguments.udp_payload_size r["use_probe_resolver"] = "target" not in r return r ripe-atlas-tools-1.2.2/ripe/atlas/tools/commands/measure/http.py000066400000000000000000000070661264541652500247340ustar00rootroot00000000000000# Copyright (c) 2015 RIPE NCC # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . from __future__ import print_function, absolute_import from ...helpers.validators import ArgumentType from ...settings import conf from .base import Command class HttpMeasureCommand(Command): def add_arguments(self): Command.add_arguments(self) spec = conf["specification"]["types"]["http"] specific = self.parser.add_argument_group("HTTP-specific Options") specific.add_argument( "--header-bytes", type=ArgumentType.integer_range(minimum=0, maximum=2048), default=spec["header-bytes"], help="The maximum number of bytes to retrieve from the header" ) specific.add_argument( "--version", type=str, default=spec["version"], help="The HTTP version to use" ) specific.add_argument( "--method", type=str, default=spec["method"], help="The HTTP method to use" ) specific.add_argument( "--port", type=ArgumentType.integer_range(minimum=1, maximum=2**16), default=spec["port"], help="Destination port" ) specific.add_argument( "--path", type=str, default=spec["path"], help="" ) specific.add_argument( "--query-string", type=str, default=spec["query-string"], help="" ) specific.add_argument( "--user-agent", type=str, default=spec["user-agent"], help="The user agent used when performing the request" ) specific.add_argument( "--body-bytes", type=ArgumentType.integer_range(minimum=1, maximum=1020048), default=spec["body-bytes"], help="The maximum number of bytes to retrieve from the body" ) specific.add_argument( "--timing-verbosity", type=int, choices=(0, 1, 2), default=spec["timing-verbosity"], help="The amount of timing information you want returned. 1 " "returns the time to read, to connect, and to first byte, 2 " "returns timing information per read system call. 0 " "(default) returns no additional timing information." ) def _get_measurement_kwargs(self): r = Command._get_measurement_kwargs(self) keys = ( "header_bytes", "version", "method", "port", "path", "query_string", "user_agent" ) for key in keys: r[key] = getattr(self.arguments, key) if self.arguments.timing_verbosity > 0: r["extended_timing"] = True if self.arguments.timing_verbosity > 1: r["more_extended_timing"] = True r["max_bytes_read"] = self.arguments.body_bytes return r ripe-atlas-tools-1.2.2/ripe/atlas/tools/commands/measure/ntp.py000066400000000000000000000032021264541652500245420ustar00rootroot00000000000000# Copyright (c) 2015 RIPE NCC # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . from __future__ import print_function, absolute_import from ...helpers.validators import ArgumentType from ...settings import conf from .base import Command class NtpMeasureCommand(Command): def add_arguments(self): Command.add_arguments(self) spec = conf["specification"]["types"]["ntp"] specific = self.parser.add_argument_group("NTP-specific Options") specific.add_argument( "--packets", type=ArgumentType.integer_range(minimum=1), default=spec["packets"], help="The number of packets sent" ) specific.add_argument( "--timeout", type=ArgumentType.integer_range(minimum=1), default=spec["timeout"], help="The timeout per-packet" ) def _get_measurement_kwargs(self): r = Command._get_measurement_kwargs(self) r["packets"] = self.arguments.packets r["timeout"] = self.arguments.timeout return r ripe-atlas-tools-1.2.2/ripe/atlas/tools/commands/measure/ping.py000066400000000000000000000035471264541652500247120ustar00rootroot00000000000000# Copyright (c) 2015 RIPE NCC # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . from __future__ import print_function, absolute_import from ...helpers.validators import ArgumentType from ...settings import conf from .base import Command class PingMeasureCommand(Command): def add_arguments(self): Command.add_arguments(self) spec = conf["specification"]["types"]["ping"] specific = self.parser.add_argument_group("Ping-specific Options") specific.add_argument( "--packets", type=ArgumentType.integer_range(minimum=1), default=spec["packets"], help="The number of packets sent" ) specific.add_argument( "--size", type=ArgumentType.integer_range(minimum=1), default=spec["size"], help="The size of packets sent" ) specific.add_argument( "--packet-interval", type=ArgumentType.integer_range(minimum=1), default=spec["packet-interval"], ) def _get_measurement_kwargs(self): r = Command._get_measurement_kwargs(self) r["packets"] = self.arguments.packets r["packet_interval"] = self.arguments.packet_interval r["size"] = self.arguments.size return r ripe-atlas-tools-1.2.2/ripe/atlas/tools/commands/measure/sslcert.py000066400000000000000000000026441264541652500254310ustar00rootroot00000000000000# Copyright (c) 2015 RIPE NCC # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . from __future__ import print_function, absolute_import from ...helpers.validators import ArgumentType from ...settings import conf from .base import Command class SslcertMeasureCommand(Command): def add_arguments(self): Command.add_arguments(self) spec = conf["specification"]["types"]["sslcert"] specific = self.parser.add_argument_group( "SSL Certificate-specific Options") specific.add_argument( "--port", type=ArgumentType.integer_range(minimum=1, maximum=2**16), default=spec["port"], help="Destination port" ) def _get_measurement_kwargs(self): r = Command._get_measurement_kwargs(self) r["port"] = self.arguments.port return r ripe-atlas-tools-1.2.2/ripe/atlas/tools/commands/measure/traceroute.py000066400000000000000000000075541264541652500261340ustar00rootroot00000000000000# Copyright (c) 2015 RIPE NCC # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . from __future__ import print_function, absolute_import from ...helpers.validators import ArgumentType from ...settings import conf from .base import Command class TracerouteMeasureCommand(Command): def add_arguments(self): Command.add_arguments(self) spec = conf["specification"]["types"]["traceroute"] specific = self.parser.add_argument_group( "Traceroute-specific Options") specific.add_argument( "--packets", type=ArgumentType.integer_range(minimum=1), default=spec["packets"], help="The number of packets sent" ) specific.add_argument( "--size", type=ArgumentType.integer_range(minimum=1), default=spec["size"], help="The size of packets sent" ) specific.add_argument( "--protocol", type=str, choices=("ICMP", "UDP", "TCP"), default=spec["protocol"], help="The protocol used." ) specific.add_argument( "--timeout", type=ArgumentType.integer_range(minimum=1), default=spec["timeout"], help="The timeout per-packet" ) specific.add_argument( "--dont-fragment", action="store_true", default=spec["dont-fragment"], help="Don't Fragment the packet" ) specific.add_argument( "--paris", type=ArgumentType.integer_range(minimum=0, maximum=64), default=spec["paris"], help="Use Paris. Value must be between 0 and 64." "If 0, a standard traceroute will be performed" ) specific.add_argument( "--first-hop", type=ArgumentType.integer_range(minimum=1, maximum=255), default=spec["first-hop"], help="Value must be between 1 and 255" ) specific.add_argument( "--max-hops", type=ArgumentType.integer_range(minimum=1, maximum=255), default=spec["max-hops"], help="Value must be between 1 and 255" ) specific.add_argument( "--port", type=ArgumentType.integer_range(minimum=1, maximum=2**16), default=spec["port"], help="Destination port, valid for TCP only" ) specific.add_argument( "--destination-option-size", type=ArgumentType.integer_range(minimum=1), default=spec["destination-option-size"], help="IPv6 destination option header" ) specific.add_argument( "--hop-by-hop-option-size", type=ArgumentType.integer_range(minimum=1), default=spec["hop-by-hop-option-size"], help=" IPv6 hop by hop option header" ) def _get_measurement_kwargs(self): r = Command._get_measurement_kwargs(self) keys = ( "destination_option_size", "dont_fragment", "first_hop", "hop_by_hop_option_size", "max_hops", "packets", "paris", "port", "protocol", "size", "timeout" ) for key in keys: r[key] = getattr(self.arguments, key) return r ripe-atlas-tools-1.2.2/ripe/atlas/tools/commands/measurement.py000066400000000000000000000155261264541652500246410ustar00rootroot00000000000000# Copyright (c) 2015 RIPE NCC # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . from __future__ import print_function, absolute_import from ripe.atlas.cousteau import Measurement from ripe.atlas.cousteau.exceptions import APIResponseError from .base import Command as BaseCommand, MetaDataMixin from ..exceptions import RipeAtlasToolsException from ..helpers.colours import colourise from ..helpers.sanitisers import sanitise class Command(MetaDataMixin, BaseCommand): NAME = "measurement" DESCRIPTION = ( "Returns the meta data for one measurement" ) def add_arguments(self): self.parser.add_argument("id", type=int, help="The measurement id") def run(self): try: measurement = Measurement( id=self.arguments.id, user_agent=self.user_agent) except APIResponseError: raise RipeAtlasToolsException( "That measurement does not appear to exist") self.render_basic(measurement) getattr(self, "render_{}".format(measurement.type.lower()))(measurement) @classmethod def render_basic(cls, measurement): url_template = "https://atlas.ripe.net/measurements/{}/" cls._render(measurement, ( ("id", "ID"), ("id", "URL", lambda _: colourise(url_template.format(_), "cyan")), ("type", "Type", lambda _: cls._prettify_type(_["name"])), ("status", "Status", lambda _: _["name"]), ("description", "Description", sanitise), ("af", "Address Family"), ("is_public", "Public?", cls._prettify_boolean), ("is_oneoff", "One-off?", cls._prettify_boolean), ("destination_name", "Destination Name", sanitise), ("destination_address", "Destination Address", sanitise), ("destination_asn", "Destination ASN"), ("interval", "Interval"), ("spread", "Spread"), ("creation_time", "Created", cls._prettify_time), ("start_time", "Started", cls._prettify_time), ("stop_time", "Stopped", cls._prettify_time), ("probes_requested", "Probes Requested"), ("probes_scheduled", "Probes Scheduled"), ("probes_currently_involved", "Probes Involved"), ("participant_count", "Participant Count"), ("is_all_scheduled", "Fully Scheduled?", cls._prettify_boolean), ("resolved_ips", "Resolved IPs", lambda _: ", ".join(_)), ("resolve_on_probe", "Resolve on the Probe", cls._prettify_boolean), )) @classmethod def render_ping(cls, measurement): cls._render(measurement, ( ("packets", "Packets"), ("size", "Size"), )) @classmethod def render_traceroute(cls, measurement): cls._render(measurement, ( ("packets", "Packets"), ("protocol", "Protocol"), ("dont_fragment", "Don't Fragment", cls._prettify_boolean), ("paris", "Paris"), ("first_hop", "First Hop"), ("max_hops", "Maximum Hops"), ("timeout", "Timeout"), ("size", "Size"), ("destination_option_size", "Destination Option Size"), ("hop_by_hop_option_size", "Hop-by-hop Option Size"), ("gap_limit", "Gap Limit"), )) @classmethod def render_dns(cls, measurement): cls._render(measurement, ( ("query", "Query", cls._prettify_query), ("retry", "Retry Times"), ("include_qbuf", "Include the Qbuf?", cls._prettify_boolean), ("include_abuf", "Include the Abuf?", cls._prettify_boolean), ("protocol", "Protocol"), ("prepend_probe_id", "Prepend the Probe ID?"), ("udp_payload_size", "UDP Payload Size"), ( "use_probe_resolver", "Use the Probe's Resolver?", cls._prettify_boolean ), ("set_do_bit", "Set the DO Bit?", cls._prettify_boolean), ("set_nsid_bit", "Set the NSID Bit?", cls._prettify_boolean), ("set_rd_bit", "Set the RD Bit?", cls._prettify_boolean), ("set_cd_bit", "Set the CD Bit?", cls._prettify_boolean), )) @classmethod def render_sslcert(cls, measurement): cls._render(measurement, ( ("port", "Port"), )) @classmethod def render_http(cls, measurement): cls._render(measurement, ( ("header_bytes", "Header Bytes"), ("version", "Version"), ("method", "Method"), ("port", "Port"), ("path", "Path", sanitise), ("query_string", "Query String", sanitise), ("user_agent", "User-Agent"), ("max_bytes_read", "Body Bytes"), )) timing_verbosity = 0 if "extended_timing" in measurement.meta_data: if measurement.meta_data["extended_timing"]: timing_verbosity = 1 if "more_extended_timing" in measurement.meta_data: if measurement.meta_data["more_extended_timing"]: timing_verbosity = 2 cls._render_line("Timing Verbosity", timing_verbosity) @classmethod def render_ntp(cls, measurement): cls._render(measurement, ( ("packets", "Packets"), ("timeout", "Timeout"), )) @staticmethod def _prettify_type(kind): types = { "ping": "Ping", "traceroute": "Traceroute", "dns": "DNS", "sslcert": "SSL Certificate", "http": "HTTP", "ntp": "NTP" } if kind in types: return colourise(colourise(types[kind], "bold"), "blue") return colourise("Unknown", "red") @staticmethod def _prettify_query(query): return sanitise( "{} {} {}".format(query["class"], query["type"], query["value"])) @classmethod def _render(cls, measurement, keys): for prop in keys: value = None if prop[0] in measurement.meta_data: value = measurement.meta_data[prop[0]] if value is None: value = "-" elif len(prop) == 3: value = prop[2](value) cls._render_line(prop[1], value) ripe-atlas-tools-1.2.2/ripe/atlas/tools/commands/measurements.py000066400000000000000000000163471264541652500250260ustar00rootroot00000000000000# Copyright (c) 2015 RIPE NCC # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . from __future__ import print_function, absolute_import import itertools from ripe.atlas.cousteau import MeasurementRequest from .base import Command as BaseCommand, TabularFieldsMixin from ..helpers.colours import colourise from ..helpers.sanitisers import sanitise from ..helpers.validators import ArgumentType class Command(TabularFieldsMixin, BaseCommand): NAME = "measurements" LIMITS = (1, 1000) STATUS_SPECIFIED = 0 STATUS_SCHEDULED = 1 STATUS_ONGOING = 2 STATUS_STOPPED = 4 STATUS_FORCED_STOP = 5 STATUS_NO_SUITABLE_PROBES = 6 STATUS_FAILED = 7 STATUSES = { "scheduled": (STATUS_SPECIFIED, STATUS_SCHEDULED,), "ongoing": (STATUS_ONGOING,), "stopped": ( STATUS_STOPPED, STATUS_FORCED_STOP, STATUS_NO_SUITABLE_PROBES, STATUS_FAILED, ) } # Column name: (alignment, width) COLUMNS = { "id": ("<", 7), "type": ("<", 10), "description": ("<", 45), "status": (">", 18), "target": ("<", 45), "url": ("<", 45), } DESCRIPTION = ( "Fetches and prints measurements fulfilling specified criteria based " "on given filters." ) def add_arguments(self): self.parser.add_argument( "--search", type=str, help="A search string. This could match the target or description." ) self.parser.add_argument( "--status", type=str, choices=self.STATUSES.keys(), help="The measurement status." ) self.parser.add_argument( "--af", type=int, choices=(4, 6), help="The address family." ) self.parser.add_argument( "--type", type=str, choices=("ping", "traceroute", "dns", "sslcert", "ntp", "http"), help="The measurement type." ) self.parser.add_argument( "--field", type=str, action="append", choices=self.COLUMNS.keys(), default=[], help="The field(s) to display. Invoke multiple times for multiple " "fields. The default is id, type, description, and status." ) self.parser.add_argument( "--ids-only", action="store_true", default=False, help="Display a list of measurement ids matching your filter " "criteria." ) timing = self.parser.add_argument_group("Timing") for position in ("started", "stopped"): for chrono in ("before", "after"): timing.add_argument( "--{}-{}".format(position, chrono), type=ArgumentType.datetime, help="Filter for measurements that {} {} a specific date. " "The format required is YYYY-MM-DDTHH:MM:SS".format( position, chrono) ) self.parser.add_argument( "--limit", type=ArgumentType.integer_range(self.LIMITS[0], self.LIMITS[1]), default=50, help="The number of measurements to return. The number must be " "between {} and {}".format(self.LIMITS[0], self.LIMITS[1]) ) def run(self): if not self.arguments.field: self.arguments.field = ("id", "type", "description", "status") filters = self._get_filters() measurements = MeasurementRequest( return_objects=True, user_agent=self.user_agent, **filters) truncated_measurements = itertools.islice( measurements, self.arguments.limit) if self.arguments.ids_only: for measurement in truncated_measurements: print(measurement.id) return hr = self._get_horizontal_rule() print(self._get_filter_display(filters)) print(self._get_header()) print(colourise(hr, "bold")) for measurement in truncated_measurements: print(colourise(self._get_line_format().format( *self._get_line_items(measurement) ), self._get_colour_from_status(measurement.status_id))) print(colourise(hr, "bold")) # Print total count of found measurements print(("{:>" + str(len(hr)) + "}\n").format( "Showing {} of {} total measurements".format( min(self.arguments.limit, measurements.total_count), measurements.total_count ) )) def _get_line_items(self, measurement): r = [] for field in self.arguments.field: if field == "url": r.append("https://atlas.ripe.net/measurements/{}/".format( measurement.id )) elif field == "type": r.append(measurement.type.lower()) continue elif field == "target": r.append(sanitise( measurement.destination_name or measurement.destination_address or "-" )[:self.COLUMNS["target"][1]]) elif field == "description": description = sanitise(measurement.description) or "" r.append(description[:self.COLUMNS["description"][1]]) else: r.append(sanitise(getattr(measurement, field))) return r def _get_filters(self): r = {} if self.arguments.search: r["search"] = self.arguments.search if self.arguments.status: r["status__in"] = self.STATUSES[self.arguments.status] if self.arguments.af: r["af"] = self.arguments.af if self.arguments.type: r["type"] = self.arguments.type if self.arguments.started_before: r["start_time__lt"] = self.arguments.started_before if self.arguments.started_after: r["start_time__gt"] = self.arguments.started_after if self.arguments.stopped_before: r["stop_time__lt"] = self.arguments.stopped_before if self.arguments.stopped_after: r["stop_time__gt"] = self.arguments.stopped_after return r def _get_colour_from_status(self, status): if status in self.STATUSES["ongoing"]: return "green" if status == self.STATUS_STOPPED: return "yellow" if status in self.STATUSES["stopped"]: return "red" if status in self.STATUSES["scheduled"]: return "blue" return "white" ripe-atlas-tools-1.2.2/ripe/atlas/tools/commands/probe.py000066400000000000000000000055001264541652500234120ustar00rootroot00000000000000# Copyright (c) 2015 RIPE NCC # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . from __future__ import print_function, absolute_import from ripe.atlas.cousteau import Probe from ripe.atlas.cousteau.exceptions import APIResponseError from .base import Command as BaseCommand, MetaDataMixin from ..exceptions import RipeAtlasToolsException from ..helpers.colours import colourise from ..helpers.sanitisers import sanitise class Command(MetaDataMixin, BaseCommand): NAME = "probe" DESCRIPTION = "Returns the meta data for one probe" def add_arguments(self): self.parser.add_argument("id", type=int, help="The probe id") def run(self): try: probe = Probe(id=self.arguments.id, user_agent=self.user_agent) except APIResponseError: raise RipeAtlasToolsException( "That probe does not appear to exist") url_template = "https://atlas.ripe.net/probes/{}/" keys = ( ("id", "ID"), ("id", "URL", lambda _: colourise(url_template.format(_), "cyan")), ("is_public", "Public?", self._prettify_boolean), ("is_anchor", "Anchor?", self._prettify_boolean), ("country_code", "Country"), ("description", "Description", sanitise), ("asn_v4", "ASN (IPv4)"), ("asn_v6", "ASN (IPv6)"), ("address_v4", "Address (IPv4)"), ("address_v6", "Address (IPv6)"), ("prefix_v4", "Prefix (IPv4)"), ("prefix_v6", "Prefix (IPv6)"), ("geometry", "Coordinates", self._prettify_coordinates), ("status", "Status"), ) for key in keys: value = getattr(probe, key[0]) if value is None: value = "-" elif len(key) == 3: value = key[2](value) self._render_line(key[1], value) print(colourise("Tags", "bold")) for tag in probe.tags: print(" {}".format(tag["slug"])) @staticmethod def _prettify_coordinates(geometry): if geometry and "coordinates" in geometry and geometry["coordinates"]: return "{},{}".format( geometry["coordinates"][1], geometry["coordinates"][0] ) ripe-atlas-tools-1.2.2/ripe/atlas/tools/commands/probes.py000066400000000000000000000365451264541652500236120ustar00rootroot00000000000000# Copyright (c) 2015 RIPE NCC # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . from __future__ import print_function, absolute_import import itertools import six import requests from ripe.atlas.cousteau import ProbeRequest from ripe.atlas.tools.aggregators import ValueKeyAggregator, aggregate from .base import Command as BaseCommand, TabularFieldsMixin from ..exceptions import RipeAtlasToolsException from ..helpers.colours import colourise from ..helpers.sanitisers import sanitise class Command(TabularFieldsMixin, BaseCommand): NAME = "probes" DESCRIPTION = ( "Fetches and prints probes fulfilling specified criteria based on " "given filters." ) # Column name: (alignment, width) COLUMNS = { "id": ("<", 5), "asn_v4": ("<", 6), "asn_v6": ("<", 6), "country": ("^", 7), "status": ("<", 15), "prefix_v4": ("<", 18), "prefix_v6": ("<", 18), "coordinates": ("<", 19), "is_public": ("^", 6), "description": ("<", 30), "address_v4": ("<", 15), "address_v6": ("<", 39), "is_anchor": ("^", 6), } def __init__(self, *args, **kwargs): BaseCommand.__init__(self, *args, **kwargs) self.aggregators = [] self.first_line_padding = False def add_arguments(self): """Adds all commands line arguments for this command.""" asn = self.parser.add_argument_group("ASN") asn.add_argument( "--asn", type=int, help="ASN" ) asn.add_argument( "--asnv4", type=int, help="ASNv4" ) asn.add_argument( "--asnv6", type=int, help="ASNv6" ) prefix = self.parser.add_argument_group("Prefix") prefix.add_argument( "--prefix", type=str, help="Prefix" ) prefix.add_argument( "--prefixv4", type=str, help="Prefixv4" ) prefix.add_argument( "--prefixv6", type=str, help="Prefixv6" ) area = self.parser.add_argument_group("Area") geo_location = area.add_mutually_exclusive_group() geo_location.add_argument( "--location", type=str, help="The location of probes as a string i.e. 'Amsterdam'" ) geo_location.add_argument( "--center", type=str, help="location as ,-string, i.e. '48.45,9.16'" ) geo_location.add_argument( "--country", type=str, help="The country code of probes." ) area.add_argument( "--radius", type=int, help="Radius in km from specified center/point." ) self.parser.add_argument( "--limit", type=int, default=25, help="Return limited number of probes" ) self.parser.add_argument( "--field", type=str, action="append", choices=self.COLUMNS.keys(), default=[], help="The field(s) to display. Invoke multiple times for multiple " "fields. The default is id, asn_v4, asn_v6, country, and " "status." ) self.parser.add_argument( "--aggregate-by", type=str, choices=[ 'country', 'asn_v4', 'asn_v6', 'prefix_v4', 'prefix_v6' ], action="append", help=( "Aggregate list of probes based on all specified aggregations." " Multiple aggregations supported." ) ) self.parser.add_argument( "--all", action='store_true', help="Fetch *ALL* probes. That will give you a loooong list." ) self.parser.add_argument( "--max-per-aggregation", type=int, help="Maximum number of probes per aggregated bucket." ) self.parser.add_argument( "--ids-only", action='store_true', help=( "Print only IDs of probes. Useful to pipe it to another " "command." ) ) def run(self): if not self.arguments.field: self.arguments.field = ( "id", "asn_v4", "asn_v6", "country", "status") if self.arguments.all: self.arguments.limit = None filters = self.build_request_args() if not filters and not self.arguments.all: raise RipeAtlasToolsException(colourise( "Typically you'd want to run this with some arguments to " "filter the probe \nlist, as fetching all of the probes can " "take a Very Long Time. However, if you \ndon't care about " "the wait, you can use --all and go get yourself a coffee.", "blue" )) self.set_aggregators() probes = ProbeRequest( return_objects=True, user_agent=self.user_agent, **filters) if self.arguments.limit: truncated_probes = itertools.islice(probes, self.arguments.limit) else: truncated_probes = probes if self.arguments.ids_only: for probe in truncated_probes: print(probe.id) return hr = self._get_horizontal_rule() print(self._get_filter_display(filters)) print(colourise(self._get_header(), "bold")) print(colourise(hr, "bold")) if self.arguments.aggregate_by: buckets = aggregate(list(truncated_probes), self.aggregators) self.render_aggregation(buckets) else: for probe in truncated_probes: print(self._get_line(probe)) print(colourise(hr, "bold")) # Print total count of found measurements print(("{:>" + str(len(hr)) + "}\n").format( "Showing {} of {} total probes".format( min(self.arguments.limit, probes.total_count) or "all", probes.total_count ) )) def render_aggregation(self, aggregation_data, indent=0): """ Recursively traverses through aggregation data and print them indented. """ if isinstance(aggregation_data, dict): for k, v in aggregation_data.items(): if not indent: if self.first_line_padding: print("") else: self.first_line_padding = True print((u" " * indent) + colourise(k, "bold")) self.render_aggregation(v, indent=indent + 1) elif isinstance(aggregation_data, list): for index, probe in enumerate(aggregation_data): print(" ", end="") print(self._get_line(probe)) if self.arguments.max_per_aggregation: if index >= self.arguments.max_per_aggregation - 1: break def build_request_args(self): """ Builds the request arguments from parser arguments and returns a dict that can be used with ATLAS API. """ if self.arguments.all: return {} return self._clean_request_args() def _clean_request_args(self): """Cleans all arguments for the API request and checks for sanity.""" args = {} if any( [self.arguments.asn, self.arguments.asnv4, self.arguments.asnv6] ): args.update(self._clean_asn()) if any([ self.arguments.prefix, self.arguments.prefixv4, self.arguments.prefixv6 ]): args.update(self._clean_prefix()) if self.arguments.location: args.update(self._clean_location()) if self.arguments.center: args.update(self._clean_center()) if self.arguments.country: args.update(self._clean_country_code()) return args def _clean_asn(self): """Make sure ASN arguments don't conflict and make sense.""" asn = self.arguments.asn asnv4 = self.arguments.asnv4 asnv6 = self.arguments.asnv6 if asn and (asnv4 or asnv6): exc_log = ( "Specifying argument --asn together with --asnv4/--asnv6 " "doesn't make sense" ) raise RipeAtlasToolsException(exc_log) if asn: return {"asn": asn} asn_args = {} if asnv4: asn_args["asn_v4"] = asnv4 if asnv6: asn_args["asn_v6"] = asnv6 return asn_args def _clean_prefix(self): """Make sure ASN arguments don't conflict and make sense.""" prefix = self.arguments.prefix prefixv4 = self.arguments.prefixv4 prefixv6 = self.arguments.prefixv6 if prefix and (prefixv4 or prefixv6): exc_log = ( "Specifying argument --prefix together with " "--prefixv4/--prefixv6 doesn't make sense" ) raise RipeAtlasToolsException(exc_log) if prefix: return {"prefix": prefix} prefix_args = {} if prefixv4: prefix_args["prefix_v4"] = prefixv4 if prefixv6: prefix_args["prefix_v6"] = prefixv6 return prefix_args def _clean_location(self): """Make sure location argument are sane.""" lat, lng = self.location2degrees() if self.arguments.radius: location_args = { "radius": "{0},{1}:{2}".format(lat, lng, self.arguments.radius) } else: location_args = {"latitude": lat, "longitude": lng} return location_args def location2degrees(self): """Fetches degrees based on the given location.""" error_log = ( "Following error occured while trying to fetch lat/lon" "for location <{}>:\n{}" ) goole_api_url = "http://maps.googleapis.com/maps/api/geocode/json" try: result = requests.get(goole_api_url, params={ "sensor": "false", "address": self.arguments.location }) except ( requests.ConnectionError, requests.HTTPError, requests.Timeout, ) as e: error_log = error_log.format(self.arguments.location, e) raise RipeAtlasToolsException(error_log) result = result.json() try: lat = result["results"][0]["geometry"]["location"]["lat"] lng = result["results"][0]["geometry"]["location"]["lng"] except (KeyError, IndexError) as e: error = error_log.format(self.arguments.location, e) raise RipeAtlasToolsException(error) return str(lat), str(lng) def _clean_center(self): """Make sure center argument are sane.""" try: lat, lng = self.arguments.center.split(",") except ValueError: raise RipeAtlasToolsException( "Point argument should be in format." ) if self.arguments.radius: center_args = { "radius": "{0},{1}:{2}".format(lat, lng, self.arguments.radius) } else: center_args = {"latitude": lat, "longitude": lng} return center_args def _clean_country_code(self): """Make sure country_code argument are sane.""" return {"country_code": self.arguments.country} def set_aggregators(self): """ Builds and returns the key aggregators that will be used in the aggregation. """ self.aggregators = [] if not self.arguments.aggregate_by: return for key in self.arguments.aggregate_by: if key == "country": self.aggregators.append(ValueKeyAggregator( key="country_code", prefix="Country")) else: self.aggregators.append(ValueKeyAggregator(key=key)) def _get_line_items(self, probe): r = [] for field in self.arguments.field: if field == "country": r.append((probe.country_code or "").lower()) elif field in ("asn_v4", "asn_v6"): r.append(getattr(probe, field) or "") elif field == "description": description = sanitise(probe.description) or "" r.append(description[:self.COLUMNS["description"][1]]) elif field == "coordinates": r.append(u"{},{}".format( probe.geometry["coordinates"][1], probe.geometry["coordinates"][0], )) elif field in ("is_public", "is_anchor"): if getattr(probe, field): r.append(u"\u2714") # Check mark else: r.append(u"\u2718") # X else: r.append(sanitise(getattr(probe, field))) return r @staticmethod def _get_colour_from_status(status): if status == "Connected": return "green" if status == "Disconnected": return "yellow" if status == "Abandoned": return "red" return "white" def _get_line_format(self): r = TabularFieldsMixin._get_line_format(self) if not self.aggregators: return r return (u" " * len(self.aggregators)) + r def _get_header_names(self): r = [] for field in self.arguments.field: if field == "id": r.append("ID") elif field == "is_public": r.append("Public") elif field == "is_anchor": r.append("Anchor") else: r.append(field.capitalize()) return r def _get_line(self, probe): """ Python 2 and 3 differ on how to render strings with non-ascii characters in them, so we have to accommodate both here. """ if six.PY2: return colourise( self._get_line_format().format( *self._get_line_items(probe) ).encode("utf-8"), self._get_colour_from_status(probe.status) ) return colourise( self._get_line_format().format(*self._get_line_items(probe)), self._get_colour_from_status(probe.status) ) def _get_filter_key_value_pair(self, k, v): if k == "country_code": return "Country", v.upper() if k == "asn": return "ASN", v return TabularFieldsMixin._get_filter_key_value_pair(self, k, v) ripe-atlas-tools-1.2.2/ripe/atlas/tools/commands/render.py000066400000000000000000000131421264541652500235630ustar00rootroot00000000000000# Copyright (c) 2015 RIPE NCC # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . from __future__ import print_function import sys import itertools try: import ujson as json except ImportError: import json from ripe.atlas.sagan import Result from ..aggregators import RangeKeyAggregator, ValueKeyAggregator, aggregate from ..helpers.rendering import SaganSet, Rendering from ..helpers.validators import ArgumentType from ..renderers import Renderer from .base import Command as BaseCommand class Command(BaseCommand): NAME = "render" DESCRIPTION = "Render the contents of an arbitrary file.\n\nExample:\n" \ " cat /my/file | ripe-atlas render\n" AGGREGATORS = { "country": ["probe.country_code", ValueKeyAggregator], "rtt-median": [ "rtt_median", RangeKeyAggregator, [10, 20, 30, 40, 50, 100, 200, 300] ], "status": ["probe.status", ValueKeyAggregator], "asn_v4": ["probe.asn_v4", ValueKeyAggregator], "asn_v6": ["probe.asn_v6", ValueKeyAggregator], "prefix_v4": ["probe.prefix_v4", ValueKeyAggregator], "prefix_v6": ["probe.prefix_v6", ValueKeyAggregator], } def __init__(self, *args, **kwargs): BaseCommand.__init__(self, *args, **kwargs) self.file = None def add_arguments(self): self.parser.add_argument( "--renderer", choices=Renderer.get_available(), help="The renderer you want to use. If this isn't defined, an " "appropriate renderer will be selected." ) self.parser.add_argument( "--probes", type=ArgumentType.comma_separated_integers_or_file, help="Either a comma-separated list of probe ids you want to see " "exclusively, a path to a file containing probe ids (one on " "each line), or \"-\" for standard input in the same format." ) self.parser.add_argument( "--from-file", type=ArgumentType.path, default="-", help='The source of the data to be rendered. If nothing is ' 'specified, we assume "-" or, standard in (the default).' ) self.parser.add_argument( "--aggregate-by", type=str, choices=self.AGGREGATORS.keys(), action="append", help="Tell the rendering engine to aggregate the results by the " "selected option. Note that if you opt for aggregation, no " "output will be generated until all results are received, and " "if large data sets may explode your system." ) def run(self): using_regular_file = self.arguments.from_file != "-" sample, source = self._get_sample_result_and_source(using_regular_file) results = SaganSet(iterable=source, probes=self.arguments.probes) if self.arguments.aggregate_by: results = aggregate(results, self.get_aggregators()) renderer = Renderer.get_renderer( self.arguments.renderer, Result.get(sample).type)() Rendering(renderer=renderer, payload=results).render() if using_regular_file: self.file.close() def get_aggregators(self): """ Return aggregators list based on user input """ aggregation_keys = [] for aggr_key in self.arguments.aggregate_by: # Get class and aggregator key aggregation_class = self.AGGREGATORS[aggr_key][1] key = self.AGGREGATORS[aggr_key][0] if aggr_key == "rtt": # Get range for the aggregation key_range = self.AGGREGATORS[aggr_key][2] aggregation_keys.append( aggregation_class(key=key, ranges=key_range) ) else: aggregation_keys.append(aggregation_class(key=key)) return aggregation_keys def _get_sample_result_and_source(self, using_regular_file): """ We need to get the first result from the source in order to detect the type. Additionally, if the source is actually one great big JSON list, then we need to parse it so we iterate over the results since there's no newline characters. """ self.file = sys.stdin if using_regular_file: self.file = open(self.arguments.from_file) # Pop the first line off the source stack. This may very well be a Very # Large String and cause a memory explosion, but we like to let our # users shoot themselves in the foot. sample = self.file.next() # Re-attach the line back onto the iterable so we don't lose anything source = itertools.chain([sample], self.file) # In the case of the Very Large String, we parse out the JSON here if sample.startswith("["): source = json.loads("".join(source)) sample = source[0] # Reassign sample to an actual result return sample, source ripe-atlas-tools-1.2.2/ripe/atlas/tools/commands/report.py000066400000000000000000000137751264541652500236330ustar00rootroot00000000000000# Copyright (c) 2015 RIPE NCC # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . from __future__ import print_function from ripe.atlas.cousteau import ( AtlasLatestRequest, AtlasResultsRequest, Measurement, APIResponseError) from ..aggregators import RangeKeyAggregator, ValueKeyAggregator, aggregate from ..exceptions import RipeAtlasToolsException from ..helpers.rendering import SaganSet, Rendering from ..helpers.validators import ArgumentType from ..renderers import Renderer from .base import Command as BaseCommand from ..filters import FilterFactory, filter_results class Command(BaseCommand): NAME = "report" DESCRIPTION = "Report the results of a measurement.\n\nExample:\n" \ " ripe-atlas report 1001 --probes 157,10006\n" AGGREGATORS = { "country": ["probe.country_code", ValueKeyAggregator], "rtt-median": [ "rtt_median", RangeKeyAggregator, [10, 20, 30, 40, 50, 100, 200, 300] ], "status": ["probe.status", ValueKeyAggregator], "asn_v4": ["probe.asn_v4", ValueKeyAggregator], "asn_v6": ["probe.asn_v6", ValueKeyAggregator], "prefix_v4": ["probe.prefix_v4", ValueKeyAggregator], "prefix_v6": ["probe.prefix_v6", ValueKeyAggregator], } def add_arguments(self): self.parser.add_argument( "measurement_id", type=int, help="The measurement id you want reported." ) self.parser.add_argument( "--probes", type=ArgumentType.comma_separated_integers_or_file, help="Either a comma-separated list of probe ids you want to see " "exclusively, a path to a file containing probe ids (one on " "each line), or \"-\" for standard input in the same format." ) self.parser.add_argument( "--renderer", choices=Renderer.get_available(), help="The renderer you want to use. If this isn't defined, an " "appropriate renderer will be selected." ) self.parser.add_argument( "--aggregate-by", type=str, choices=self.AGGREGATORS.keys(), action="append", help="Tell the rendering engine to aggregate the results by the " "selected option. Note that if you opt for aggregation, no " "output will be generated until all results are received." ) self.parser.add_argument( "--probe-asns", type=ArgumentType.comma_separated_integers( minimum=1, # http://www.iana.org/assignments/as-numbers/as-numbers.xhtml maximum=2 ** 32 - 2 ), help="A comma-separated list of probe ASNs you want to see " "exclusively." ) self.parser.add_argument( "--start-time", type=ArgumentType.datetime, help="The start time of the report." ) self.parser.add_argument( "--stop-time", type=ArgumentType.datetime, help="The stop time of the report." ) def _get_request(self): kwargs = { "msm_id": self.arguments.measurement_id, "user_agent": self.user_agent } if self.arguments.probes: kwargs["probe_ids"] = self.arguments.probes if self.arguments.start_time: kwargs["start"] = self.arguments.start_time if self.arguments.stop_time: kwargs["stop"] = self.arguments.stop_time if "start" in kwargs or "stop" in kwargs: return AtlasResultsRequest(**kwargs) return AtlasLatestRequest(**kwargs) def run(self): try: measurement = Measurement( id=self.arguments.measurement_id, user_agent=self.user_agent) except APIResponseError: raise RipeAtlasToolsException("That measurement does not exist") renderer = Renderer.get_renderer( self.arguments.renderer, measurement.type.lower())() results = self._get_request().get()[1] if not results: raise RipeAtlasToolsException( "There aren't any results available for that measurement") results = SaganSet(iterable=results, probes=self.arguments.probes) if self.arguments.probe_asns: asn_filters = set([]) for asn in self.arguments.probe_asns: asn_filters.add(FilterFactory.create("asn", asn)) results = filter_results(asn_filters, list(results)) if self.arguments.aggregate_by: results = aggregate(results, self.get_aggregators()) Rendering(renderer=renderer, payload=results).render() def get_aggregators(self): """Return aggregators list based on user input""" aggregation_keys = [] for aggr_key in self.arguments.aggregate_by: # Get class and aggregator key aggregation_class = self.AGGREGATORS[aggr_key][1] key = self.AGGREGATORS[aggr_key][0] if aggr_key == "rtt-median": # Get range for the aggregation key_range = self.AGGREGATORS[aggr_key][2] aggregation_keys.append( aggregation_class(key=key, ranges=key_range) ) else: aggregation_keys.append(aggregation_class(key=key)) return aggregation_keys ripe-atlas-tools-1.2.2/ripe/atlas/tools/commands/shibboleet.py000066400000000000000000000116141264541652500244260ustar00rootroot00000000000000# Copyright (c) 2015 RIPE NCC # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . from __future__ import absolute_import, print_function import random import requests from ..cache import cache from ..helpers.colours import colourise from ..helpers.sanitisers import sanitise from .base import Command as BaseCommand class Command(BaseCommand): DESCRIPTION = "https://xkcd.com/806/" HEADERS = { "Accept": "application/vnd.github.v3+json", "User-Agent": "RIPE Atlas Tools (Magellan)", } URLS = { "root": "https://api.github.com", "statistics": [ "/repos/RIPE-NCC/ripe.atlas.sagan/stats/contributors", "/repos/RIPE-NCC/ripe-atlas-cousteau/stats/contributors", "/repos/RIPE-NCC/ripe-atlas-tools/stats/contributors", ], "users": "/users" } SPACING = ( 61, 61, 61, 53, 7, 53, 6, 52, 5, 52, 49, 48, 47, 46, 47, 47, 43, 41, 38, 39, 42, 46 ) BOAT = ( "\n{}|\n{}|\n{}|\n{}|{}|\n{}|{}---\n{}---{}'-'\n{}'-' ____|_____\n{}__" "__|__/ | /\n{}/ | / | /\n{}/ |( | (\n{}( " " | \ | \\\n{}\ | \____|____\ /|\n{}/\____|___`---.----` ." "' |\n{}.-'/ | \ |__.--' \\\n{}.'/ ( | \ |. " " \\\n{}_ /_/ \ | \ | `. \\\n{}`-.' \.--._|.--" "-` | `-._______\\\n{}``-.-------'-------'------------/\n{}`'.______" "_________________.'\n" ).format(*[" " * _ for _ in SPACING]) WATER = ("~" * 80) def __init__(self, *args, **kwargs): BaseCommand.__init__(self, *args, **kwargs) self.statistics = {} def run(self): r = ( "\nThanks for using RIPE Atlas!\n\nThis toolkit " "(Magellan) is a group effort, spearheaded by the team at the " "RIPE\nNCC, but supported by members of the community from all " "over. If you're\ncurious about who we are and what sorts of " "stuff we work on, here's a break\ndown of our contributions to " "date.\n\nName Changes URL\n{}\n" ).format("-" * 79) for contributor in self.get_contributors(): r += u"{name:20} {changes:10} {url}\n".format(**contributor) print(u"{}{}{}\n".format( r, colourise(colourise(self.BOAT, "black"), "bold"), colourise(self.WATER, "blue") )) def get_contributors(self): cache_key = "github:statistics" self.statistics = cache.get(cache_key, {}) if not self.statistics: for url in self.URLS["statistics"]: self._update_statistics_from_url(url) cache.set(cache_key, self.statistics, 60 * 10) r = [] for k, v in self.statistics.items(): r.append({ "name": sanitise(k), "changes": v["changes"], "url": v["url"] }) random.shuffle(r) return r def _update_statistics_from_url(self, url): response = requests.get( "{}{}".format(self.URLS["root"], url), headers=self.HEADERS) contributors = response.json() # Sometimes, GitHub just returns nothing if not contributors: return self._update_statistics_from_url(url) for contributor in response.json(): user = self.get_user(contributor["author"]["login"]) name = user["name"] or contributor["author"]["login"] if name not in self.statistics: self.statistics[name] = { "changes": 0, "url": contributor["author"]["html_url"] } for week in contributor["weeks"]: self.statistics[name]["changes"] += week["a"] + week["d"] def get_user(self, username): cache_key = "github-user:{}".format(username) user = cache.get(cache_key) if user: return user cache.set( cache_key, requests.get( "{}{}/{}".format( self.URLS["root"], self.URLS["users"], username ), headers=self.HEADERS ).json(), 60 * 60 * 24 * 365 ) return self.get_user(username) ripe-atlas-tools-1.2.2/ripe/atlas/tools/commands/stream.py000066400000000000000000000044211264541652500235770ustar00rootroot00000000000000# Copyright (c) 2015 RIPE NCC # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . from __future__ import print_function, absolute_import from ripe.atlas.cousteau import Measurement, APIResponseError from ..exceptions import RipeAtlasToolsException from ..renderers import Renderer from ..streaming import Stream, CaptureLimitExceeded from .base import Command as BaseCommand class Command(BaseCommand): NAME = "stream" DESCRIPTION = "Report the results of a measurement" URLS = { "detail": "/api/v2/measurements/{0}.json", } def add_arguments(self): self.parser.add_argument( "measurement_id", type=int, help="The measurement id you want streamed" ) self.parser.add_argument( "--limit", type=int, help="The maximum number of results you want to stream" ) self.parser.add_argument( "--renderer", choices=Renderer.get_available(), help="The renderer you want to use. If this isn't defined, an " "appropriate renderer will be selected." ) def run(self): try: measurement = Measurement( id=self.arguments.measurement_id, user_agent=self.user_agent) except APIResponseError: raise RipeAtlasToolsException("That measurement does not exist") try: Stream(capture_limit=self.arguments.limit).stream( self.arguments.renderer, measurement.type.lower(), self.arguments.measurement_id ) except (KeyboardInterrupt, CaptureLimitExceeded): self.ok("Disconnecting from the stream") ripe-atlas-tools-1.2.2/ripe/atlas/tools/exceptions.py000066400000000000000000000016461264541652500226720ustar00rootroot00000000000000# Copyright (c) 2015 RIPE NCC # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . import sys class RipeAtlasToolsException(Exception): RED = "\033[1;31m" RESET = "\033[0m" def write(self): r = str(self) if sys.stderr.isatty(): r = self.RED + r + self.RESET sys.stderr.write("\n{0}\n\n".format(r)) ripe-atlas-tools-1.2.2/ripe/atlas/tools/filters.py000066400000000000000000000047161264541652500221620ustar00rootroot00000000000000# Copyright (c) 2015 RIPE NCC # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . from .exceptions import RipeAtlasToolsException class FilterFactory(object): @staticmethod def create(key, value): """Create new filter class based on the key""" if key == "asn": return ASNFilter(value) else: return Filter(key, value) class Filter(object): """ Class that represents filter for results. For now supports only attributes of probes property of Result property. It could be extended for any property of Result easily. """ def __init__(self, key, value): self.key = key self.value = value def filter(self, result): """ Decide if given result should be filtered (False) or remain on the pile of results. """ try: attr_value = getattr(result.probe, self.key) except AttributeError: log = ( "Cousteau's Probe class does not have an attribute " "called: <{}>" ).format(self.key) raise RipeAtlasToolsException(log) if attr_value == self.value: return True return False class ASNFilter(Filter): """Class thar represents filter by probes that belong to given ASN.""" def __init__(self, value): key = "asn" super(ASNFilter, self).__init__(key, value) def filter(self, result): asn_v4 = getattr(result.probe, "asn_v4") asn_v6 = getattr(result.probe, "asn_v6") if self.value in (asn_v4, asn_v6): return True return False def filter_results(filters, results): """docstring for filter""" new_results = [] for result in results: for rfilter in filters: if rfilter.filter(result): new_results.append(result) break return new_results ripe-atlas-tools-1.2.2/ripe/atlas/tools/helpers/000077500000000000000000000000001264541652500215725ustar00rootroot00000000000000ripe-atlas-tools-1.2.2/ripe/atlas/tools/helpers/__init__.py000066400000000000000000000000001264541652500236710ustar00rootroot00000000000000ripe-atlas-tools-1.2.2/ripe/atlas/tools/helpers/colours.py000066400000000000000000000032531264541652500236350ustar00rootroot00000000000000# Copyright (c) 2015 RIPE NCC # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . import sys class Colour(object): @classmethod def _colourise(cls, text, colour): return "{}[{}m{}{}[0m".format(chr(0x1b), colour, text, chr(0x1b)) @classmethod def black(cls, text): return cls._colourise(text, 30) @classmethod def red(cls, text): return cls._colourise(text, 31) @classmethod def green(cls, text): return cls._colourise(text, 32) @classmethod def yellow(cls, text): return cls._colourise(text, 33) @classmethod def blue(cls, text): return cls._colourise(text, 34) @classmethod def mangenta(cls, text): return cls._colourise(text, 35) @classmethod def cyan(cls, text): return cls._colourise(text, 36) @classmethod def white(cls, text): return cls._colourise(text, 37) @classmethod def bold(cls, text): return cls._colourise(text, 1) def colourise(text, colour): if sys.stdout.isatty(): return getattr(Colour, colour)(text) return text ripe-atlas-tools-1.2.2/ripe/atlas/tools/helpers/rendering.py000066400000000000000000000074721264541652500241330ustar00rootroot00000000000000# Copyright (c) 2015 RIPE NCC # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . from __future__ import print_function from ripe.atlas.sagan import Result as SaganResult from ripe.atlas.sagan import ResultParseError from ..probes import Probe from ..renderers.base import Result as MagellanResult class SaganSet(object): """ We need something that doesn't take up a lot of memory while it's being constructed, but that will also spread out into a handy string when we need it to. """ def __init__(self, iterable=None, probes=()): self._probes = probes self._iterable = iterable def __iter__(self): sagans = [] for line in self._iterable: # line may be a dictionary (parsed JSON) if hasattr(line, "strip"): line = line.strip() # Break out when there's nothing left if not line: break try: sagan = SaganResult.get( line, on_error=SaganResult.ACTION_IGNORE, on_warning=SaganResult.ACTION_IGNORE ) if not self._probes or sagan.probe_id in self._probes: sagans.append(sagan) if len(sagans) > 100: for sagan in self._attach_probes(sagans): yield sagan sagans = [] except ResultParseError: pass # Probably garbage in the file for sagan in self._attach_probes(sagans): yield sagan def __next__(self): return iter(self).next() def next(self): return self.__next__() @staticmethod def _attach_probes(sagans): probes = dict( [(p.id, p) for p in Probe.get_many(s.probe_id for s in sagans)] ) for sagan in sagans: sagan.probe = probes[sagan.probe_id] yield sagan class Rendering(object): def __init__(self, renderer=None, header="", footer="", payload=()): self.renderer = renderer self.header = header + "\n" if header else "" self.footer = footer + "\n" if footer else "" self.payload = payload def render(self): if self.renderer.SHOW_DEFAULT_HEADER: print(self.header, end="") print(self.renderer.header(), end="") self._smart_render(self.payload) print(self.renderer.additional(self.payload), end="") print(self.renderer.footer(), end="") if self.renderer.SHOW_DEFAULT_FOOTER: print(self.footer, end="") def _get_rendered_results(self, data): for sagan in data: yield MagellanResult(self.renderer.on_result(sagan), sagan.probe_id) def _smart_render(self, data, indent=""): """ Traverses the aggregation data and prints everything nicely indented. """ if not data: return if isinstance(data, (list, SaganSet)): for line in self._get_rendered_results(data): print(indent + line, end="") elif isinstance(data, dict): for k, v in data.items(): print("{}{}".format(indent, k)) self._smart_render(v, indent=indent + " ") ripe-atlas-tools-1.2.2/ripe/atlas/tools/helpers/sanitisers.py000066400000000000000000000021701264541652500243300ustar00rootroot00000000000000# Copyright (c) 2015 RIPE NCC # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . import six FORBIDDEN = dict((i, None) for i in list(range(0, 32)) + [127]) def sanitise(s, strip_newlines=True): """ Strip out control characters to prevent people from screwing with the output """ if not isinstance(s, six.string_types): return s if six.PY2: s = unicode(s) if not strip_newlines: return s.translate( dict((k, v) for k, v in FORBIDDEN.items() if not k == 10)) return s.translate(FORBIDDEN) ripe-atlas-tools-1.2.2/ripe/atlas/tools/helpers/validators.py000066400000000000000000000126731264541652500243250ustar00rootroot00000000000000# Copyright (c) 2015 RIPE NCC # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . from __future__ import print_function, absolute_import import argparse import os import re import sys from dateutil import parser class ArgumentType(object): @staticmethod def path(string): if not os.path.exists(string) and not string == "-": raise argparse.ArgumentTypeError( 'The file name specified, "{}" does not appear to exist'.format( string ) ) return string @staticmethod def country_code(string): if not re.match(r"^[a-zA-Z][a-zA-Z]$", string): raise argparse.ArgumentTypeError( "Countries must be defined with a two-letter ISO code") return string.upper() @staticmethod def datetime(string): try: return parser.parse(string) except: raise argparse.ArgumentTypeError( "Times must be specified in ISO 8601 format. For example: " "2010-10-01T00:00:00 or a portion thereof. All times are in " "UTC." ) @staticmethod def ip_or_domain(string): message = '"{}" does not appear to be an IP address or host ' \ 'name'.format(string) if " " in string: raise argparse.ArgumentTypeError(message) if "." not in string and ":" not in string: if not re.match(r"^\w+$", string): raise argparse.ArgumentTypeError(message) return string @classmethod def comma_separated_integers_or_file(cls, string): """ Allow a list of comma-separated integers, or a file containing a newline-separated list of integers, OR "-" which implies standard out. """ if re.match(r"^((\d+,?)+)$", string): return cls.comma_separated_integers()(string) f = sys.stdin if not string == "-": if not os.path.exists(string): raise argparse.ArgumentTypeError("Cannot find file: {}".format( string )) f = open(string) try: return [int(_) for _ in f.readlines()] except ValueError: raise argparse.ArgumentTypeError( "The contents of the file presented does not conform to input " "standards. Please ensure that every line in the file " "consists of a single integer." ) class integer_range(object): def __init__(self, minimum=float("-inf"), maximum=float("inf")): self.minimum = minimum self.maximum = maximum def __call__(self, string): message = "The integer must be between {} and {}.".format( self.minimum, self.maximum) if self.maximum == float("inf"): message = "The integer must be greater than {}.".format( self.minimum) try: integer = int(string) if integer < self.minimum or integer > self.maximum: raise argparse.ArgumentTypeError(message) except ValueError: raise argparse.ArgumentTypeError( "An integer must be specified." ) return integer class comma_separated_integers(object): def __init__(self, minimum=float("-inf"), maximum=float("inf")): self.minimum = minimum self.maximum = maximum def __call__(self, string): r = [] for i in string.split(","): try: i = int(i) except ValueError: raise argparse.ArgumentTypeError( "The ids supplied were not in the correct format. Note " "that you must specify them as a list of " "comma-separated integers without spaces. Example: " "1,2,34,157,10006" ) if i < self.minimum: raise argparse.ArgumentTypeError( "{} is lower than the minimum permitted value of " "{}.".format(i, self.minimum) ) if i > self.maximum: raise argparse.ArgumentTypeError( "{} exceeds the maximum permitted value of {}.".format( i, self.maximum) ) r.append(i) return r class regex(object): def __init__(self, regex): self.regex = re.compile(regex) def __call__(self, string): if not self.regex.match(string): raise argparse.ArgumentTypeError( '"{}" does not appear to be valid.'.format(string)) return string ripe-atlas-tools-1.2.2/ripe/atlas/tools/ipdetails.py000066400000000000000000000101451264541652500224610ustar00rootroot00000000000000# Copyright (c) 2015 RIPE NCC # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . import requests import IPy from .cache import cache class IP(object): RIPESTAT_URL = "https://stat.ripe.net/data/prefix-overview/data.json?resource={ip}" CACHE_EXPIRATION_TIME = 60 * 60 * 24 * 7 def __init__(self, address): self.cached_prefix_found = False self.ip_object = IPy.IP(address) self.address = self.ip_object.strFullsize() self.asn = None self.holder = None self.prefix = None self.not_querable_types = [ 'RESERVED', 'UNSPECIFIED', 'LOOPBACK', 'UNASSIGNED', 'DOCUMENTATION', 'ULA', 'LINKLOCAL', 'PRIVATE' ] details = self._get_details() if details: self.asn = details["ASN"] self.holder = details["Holder"] self.prefix = details["Prefix"] def _get_details(self): details = None if not self.is_querable(): return details details = cache.get("IPDetails:{}".format(self.address)) if details: return details details = self.get_from_cached_prefix() if not details: details = self.query_stat() if details: self.update_cache(details) return details def is_querable(self): """Determines if address is worth querable.""" return (self.ip_object.iptype() not in self.not_querable_types) def get_from_cached_prefix(self): """Search cache for existing cached Prefix""" details = None for cache_entry in cache.keys(): if not cache_entry.startswith("IPDetailsPrefix:"): continue prefix_details = cache.get(cache_entry) # data could exist but expired if not prefix_details: continue prefix = IPy.IP(prefix_details["Prefix"]) if self.ip_object in prefix: details = prefix_details self.cached_prefix_found = True break return details def query_stat(self): """Query RIPE Stat to get address details.""" URL = self.RIPESTAT_URL.format(ip=self.address) details = {} try: response = requests.get(URL) if not response.ok: return details res = response.json() except (requests.exceptions.RequestException, ValueError): # Catch all requests exception + not valid json ones. return details if res.get("status") == "ok": try: details = { "ASN": str(res["data"]["asns"][0]["asn"]), "Holder": res["data"]["asns"][0]["holder"], "Prefix": res["data"]["resource"] } except ( # Protect from any kind of malformed json response AttributeError, ValueError, KeyError, IndexError, TypeError ): pass return details def update_cache(self, details): """Update cache for the address and prefix if needed.""" if not self.cached_prefix_found: key = "IPDetailsPrefix:{}".format(details["Prefix"]) cache.set(key, details, self.CACHE_EXPIRATION_TIME) key = "IPDetails:{}".format(self.address) cache.set(key, details, self.CACHE_EXPIRATION_TIME) def __str__(self): return "IP {}, ASN {}, Holder {}".format( self.address, self.asn, self.holder ) ripe-atlas-tools-1.2.2/ripe/atlas/tools/probes/000077500000000000000000000000001264541652500214225ustar00rootroot00000000000000ripe-atlas-tools-1.2.2/ripe/atlas/tools/probes/__init__.py000066400000000000000000000042411264541652500235340ustar00rootroot00000000000000# Copyright (c) 2015 RIPE NCC # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . from ..cache import cache from ripe.atlas.cousteau import ProbeRequest from ripe.atlas.cousteau import Probe as CProbe class Probe(object): """ A crude representation of the data we get from the API via Cousteau """ EXPIRE_TIME = 60 * 60 * 24 * 30 @classmethod def get(cls, pk): """ Given a single id, attempt to fetch a probe object from the cache. If that fails, do an API call to get it. Don't use this for multiple probes unless you know they're all in the cache, or you'll be in for a long wait. """ r = cache.get("probe:{}".format(pk)) if not r: probe = CProbe(id=pk) cache.set("probe:{}".format(probe.id), probe, cls.EXPIRE_TIME) return probe @classmethod def get_many(cls, ids): """ Given a list of ids, attempt to get probe objects out of the local cache. Probes that cannot be found will be fetched from the API and cached for future use. """ r = [] fetch_ids = [] for pk in ids: probe = cache.get("probe:{}".format(pk)) if probe: r.append(probe) else: fetch_ids.append(str(pk)) if fetch_ids: kwargs = {"id__in": fetch_ids} for probe in [p for p in ProbeRequest(return_objects=True, **kwargs)]: cache.set("probe:{}".format(probe.id), probe, cls.EXPIRE_TIME) r.append(probe) return r ripe-atlas-tools-1.2.2/ripe/atlas/tools/renderers/000077500000000000000000000000001264541652500221215ustar00rootroot00000000000000ripe-atlas-tools-1.2.2/ripe/atlas/tools/renderers/__init__.py000066400000000000000000000013161264541652500242330ustar00rootroot00000000000000# Copyright (c) 2015 RIPE NCC # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . from .base import Renderer __all__ = ["Renderer"] ripe-atlas-tools-1.2.2/ripe/atlas/tools/renderers/aggregate_ping.py000066400000000000000000000075561264541652500254530ustar00rootroot00000000000000# Copyright (c) 2015 RIPE NCC # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . from ..helpers.sanitisers import sanitise from .base import Renderer as BaseRenderer class Renderer(BaseRenderer): """ This is meant to be a stub example for what an aggregate renderer might look like. If you have ideas as to how to make this better, feel free to send along a pull request. """ RENDERS = [BaseRenderer.TYPE_PING] def __init__(self): self.target = "" self.packet_loss = 0 self.sent_packets = 0 self.received_packets = 0 self.rtts = [] self.rtts_min = [] self.rtts_max = [] self.rtt_types_map = { "min": self.rtts_min, "max": self.rtts_max } def header(self): return "Collecting results...\n" def additional(self, results): self.collect_stats(results) self.packet_loss = self.calculate_loss() return self.render( "reports/aggregate_ping.txt", target=sanitise(self.target), sent=self.sent_packets, received=self.received_packets, packet_loss=self.packet_loss, min=min(self.rtts_min), median=self.median(), mean=self.mean(), max=max(self.rtts_max) ) def collect_stats(self, results): """ Calculates, stores and collects all stats we want from the given results. """ for result in results: self.set_target(result) self.sent_packets += result.packets_sent self.received_packets += result.packets_received self.collect_min_max_rtts("min", result.rtt_min) self.collect_min_max_rtts("max", result.rtt_max) self.collect_packets_rtt(result.packets) def set_target(self, result): """Sets the target of the measurement if not set.""" if not self.target: self.target = result.destination_name def collect_min_max_rtts(self, rtt_type, rtt): """ Stores the given rtt in the corresponding list (min/max) if rtt is set. """ rtt = rtt if not rtt: rtt = 0 self.rtt_types_map[rtt_type].append(rtt) def collect_packets_rtt(self, packets): """ Collects all the rrts of given packets and stores them in our rtts list. """ for packet in packets: rtt = packet.rtt if not packet.rtt: rtt = 0 self.rtts.append(rtt) def calculate_loss(self): """Calculates the total loss between received and sent packets.""" if not self.sent_packets: return 0 return (1 - float(self.received_packets) / self.sent_packets) * 100 def mean(self): """Calculates the mean of the collected rtts""" return round( float(sum(self.rtts)) / max(len(self.rtts), 1), 3 ) def median(self): """Calculates the median of the collected rtts""" sorted_rtts = sorted(self.rtts) index = (len(self.rtts) - 1) // 2 if len(self.rtts) % 2: return sorted_rtts[index] else: return (sorted_rtts[index] + sorted_rtts[index + 1]) / 2.0 def on_result(self, result): return "" ripe-atlas-tools-1.2.2/ripe/atlas/tools/renderers/base.py000066400000000000000000000112131264541652500234030ustar00rootroot00000000000000# Copyright (c) 2015 RIPE NCC # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . import importlib import os import pkgutil import sys from ..exceptions import RipeAtlasToolsException class Renderer(object): TYPE_PING = "ping" TYPE_TRACEROUTE = "traceroute" TYPE_DNS = "dns" TYPE_SSLCERT = "sslcert" TYPE_HTTP = "http" TYPE_NTP = "ntp" RENDERS = () SHOW_DEFAULT_HEADER = True SHOW_DEFAULT_FOOTER = True def __init__(self, *args, **kwargs): pass @staticmethod def get_available(): """ Return a list of renderers available to be used. """ paths = [os.path.dirname(__file__)] if "HOME" in os.environ: path = os.path.join( os.environ["HOME"], ".config", "ripe-atlas-tools") sys.path.append(path) paths += [os.path.join(path, "renderers")] r = [package_name for _, package_name, _ in pkgutil.iter_modules(paths)] r.remove("base") return r @staticmethod def render(template, **kwargs): """ A crude templating engine. """ template = os.path.join( os.path.dirname(__file__), "templates", template) with open(template) as f: return str(f.read()).format(**kwargs) @classmethod def get_renderer(cls, name=None, kind=None): """ Using the name if you've asked for it specifically, or attempting to guess the appropriate renderer based on the kind of measurement, this will return a Renderer subclass or None if nothing can be found. """ renderer = None if name: renderer = cls.get_renderer_by_name(name) if not renderer and kind: renderer = cls.get_renderer_by_kind(kind) if kind: cls._test_renderer_accepts_kind(renderer, kind) return renderer @classmethod def get_renderer_by_name(cls, name): error_message = ( 'The renderer you selected, "{}" could not be found.' ).format(name) try: # User-defined, user-supplied r = cls.import_renderer("renderers", name) except ImportError: try: # User-defined, officially-supported r = cls.import_renderer("ripe.atlas.tools.renderers", name) except ImportError: raise RipeAtlasToolsException(error_message) return r @classmethod def get_renderer_by_kind(cls, kind): error_message = ( 'The selected renderer, "{}" could not be found.' ).format(kind) try: r = cls.import_renderer("ripe.atlas.tools.renderers", kind) except ImportError: raise RipeAtlasToolsException(error_message) return r @staticmethod def import_renderer(package, name): return getattr( importlib.import_module("{}.{}".format(package, name)), "Renderer" ) def header(self): """ Override this to add an additional header. """ return "" def additional(self, results): """ Override this for summary logic. """ return "" def footer(self): """ Override this to add an additional footer. """ return "" @staticmethod def _test_renderer_accepts_kind(renderer, kind): if kind not in renderer.RENDERS: raise RipeAtlasToolsException( 'The renderer selected does not appear to support measurements ' 'of type "{}"'.format(kind) ) def on_result(self, result): """ This must be defined in the subclass, and must return a string, even if that string is "". """ raise NotImplementedError() class Result(str): """ A string-like object that we can use to render results, but that contains enough information to be used by the aggregators if need be. """ def __new__(cls, value, probe_id): obj = str.__new__(cls, value) obj.probe_id = probe_id return obj ripe-atlas-tools-1.2.2/ripe/atlas/tools/renderers/dns.py000066400000000000000000000100231264541652500232530ustar00rootroot00000000000000# Copyright (c) 2015 RIPE NCC # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . from tzlocal import get_localzone from ..helpers.colours import colourise from ..helpers.sanitisers import sanitise from .base import Renderer as BaseRenderer class Renderer(BaseRenderer): RENDERS = [BaseRenderer.TYPE_DNS] TIME_FORMAT = "%a %b %d %H:%M:%S %Z %Y" def on_result(self, result): created = result.created.astimezone(get_localzone()) probe_id = result.probe_id r = "\n\nProbe #{0}\n{1}\n".format(probe_id, "=" * 79) if result.responses: for response in result.responses: r += self.get_formatted_response(probe_id, created, response) else: r += "\n {}\n".format(colourise("No response found", "red")) return r @classmethod def get_formatted_response(cls, probe_id, created, response): if not response.abuf: return "\n- {0} -\n\n No abuf found.\n".format( response.response_id) header_flags = [] for flag in ("aa", "ad", "cd", "qr", "ra", "rd",): if getattr(response.abuf.header, flag): header_flags.append(flag) edns = "" if response.abuf.edns0: edns = "\n ;; OPT PSEUDOSECTION:\n ; EDNS: version: {0}, " \ "flags:; udp: {1}\n".format( response.abuf.edns0.version, response.abuf.edns0.udp_size ) question = "" if response.abuf.questions: question = response.abuf.questions[0].name return cls._colourise_by_response(response, cls.render( "reports/dns.txt", # Older measurements don't have a response_id response_id=response.response_id or 1, probe=probe_id, question_name=sanitise(question), header_opcode=response.abuf.header.opcode, header_return_code=response.abuf.header.return_code, header_id=response.abuf.header.id, header_flags=" ".join(header_flags), edns=edns, question_count=len(response.abuf.questions), answer_count=len(response.abuf.answers), authority_count=len(response.abuf.authorities), additional_count=len(response.abuf.additionals), question=sanitise(cls.get_section( "question", response.abuf.questions), strip_newlines=False), answers=sanitise(cls.get_section( "answer", response.abuf.answers), strip_newlines=False), authorities=sanitise(cls.get_section( "authority", response.abuf.authorities), strip_newlines=False), additionals=sanitise(cls.get_section( "additional", response.abuf.additionals), strip_newlines=False), response_time=response.response_time, response_size=response.response_size, created=created.strftime(cls.TIME_FORMAT), destination_address=sanitise(response.destination_address), )) @staticmethod def get_section(header, data): if not data: return "" return "\n ;; {0} SECTION:\n{1}\n".format( header.upper(), "\n".join([" {0}".format(_) for _ in data]) ) @staticmethod def _colourise_by_response(response, output): colour = "red" if response.is_error else "green" return colourise(output, colour) ripe-atlas-tools-1.2.2/ripe/atlas/tools/renderers/dst_asn.py000066400000000000000000000035401264541652500241300ustar00rootroot00000000000000# Copyright (c) 2015 RIPE NCC # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . from .base import Renderer as BaseRenderer from collections import Counter from ..helpers.sanitisers import sanitise from ..ipdetails import IP class Renderer(BaseRenderer): RENDERS = [BaseRenderer.TYPE_PING] SHOW_DEFAULT_HEADER = False SHOW_DEFAULT_FOOTER = False def __init__(self, *args, **kwargs): BaseRenderer.__init__(self, *args, **kwargs) # Keys are timestamps, data struct captures ASN membership self.asns = Counter() self.asn2name = {} def on_result(self, result): dst = result.destination_address if dst is not None: ip = IP(dst) if ip.asn: self.asns[ip.asn] += 1 self.asn2name[ip.asn] = sanitise(ip.holder) else: self.asns[''] += 1 self.asn2name[''] = 'unknown' return "" return "" def additional(self, results): total = sum(self.asns.values()) r = "" for asn, count in self.asns.most_common(): r += "AS%s %.2f%% (%s)" % ( asn, 100.0 * count / total, self.asn2name[asn] ) return r ripe-atlas-tools-1.2.2/ripe/atlas/tools/renderers/http.py000066400000000000000000000043771264541652500234650ustar00rootroot00000000000000# Copyright (c) 2015 RIPE NCC # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . from ..helpers.colours import colourise from .base import Renderer as BaseRenderer class Renderer(BaseRenderer): """ We're abusing the Extended Log File Format here to render the result, amending it to include a few things not originally specified in the W3C Working Draft: http://www.w3.org/TR/WD-logfile.html Namely: http-version, header-bytes, and body-bytes """ RENDERS = [BaseRenderer.TYPE_HTTP] COLOURS = { "2": "green", "3": "blue", "4": "yellow", "5": "red" } def on_result(self, result, probes=None): r = "#Version: 1.0\n#Date: {}\n#Fields: {}\n".format( result.created.strftime("%Y-%m-%d %H:%M:%S"), "cs-method cs-uri c-ip s-ip sc-status time-taken http-version " "header-bytes body-bytes" ) for response in result.responses: r += self._colourise_by_status( "{} {} {} {} {} {} {} {} {}\n".format( result.method, result.uri, response.source_address, response.destination_address, response.code, response.response_time, response.version, response.head_size, response.body_size ), response.code ) return r + "\n" def _colourise_by_status(self, output, status): try: return colourise(output, self.COLOURS[str(status)[0]]) except (IndexError, KeyError): return colourise(output, "red") ripe-atlas-tools-1.2.2/ripe/atlas/tools/renderers/ntp.py000066400000000000000000000015351264541652500233000ustar00rootroot00000000000000# Copyright (c) 2015 RIPE NCC # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . from .base import Renderer as BaseRenderer class Renderer(BaseRenderer): RENDERS = [BaseRenderer.TYPE_NTP] def on_result(self, result, probes=None): return "Not ready yet\n" ripe-atlas-tools-1.2.2/ripe/atlas/tools/renderers/ping.py000066400000000000000000000027521264541652500234360ustar00rootroot00000000000000# Copyright (c) 2015 RIPE NCC # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . from .base import Renderer as BaseRenderer class Renderer(BaseRenderer): RENDERS = [BaseRenderer.TYPE_PING] def on_result(self, result): packets = result.packets if not packets: return "No packets found\n" # Because the origin value is more reliable as "from" in v4 and as # "packet.source_address" in v6. origin = result.origin if ":" in origin: origin = packets[0].source_address line = "{} bytes from probe #{:<5} {:15} to {} ({}): ttl={} times:{}\n" return line.format( result.packet_size, result.probe_id, origin, result.destination_name, result.destination_address, packets[0].ttl, " ".join(["{:8}".format(str(_.rtt) + ",") for _ in packets]) ) ripe-atlas-tools-1.2.2/ripe/atlas/tools/renderers/raw.py000066400000000000000000000023611264541652500232660ustar00rootroot00000000000000# Copyright (c) 2015 RIPE NCC # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . from __future__ import print_function import json from .base import Renderer as BaseRenderer from ..helpers.sanitisers import sanitise class Renderer(BaseRenderer): RENDERS = [ BaseRenderer.TYPE_PING, BaseRenderer.TYPE_TRACEROUTE, BaseRenderer.TYPE_DNS, BaseRenderer.TYPE_SSLCERT, BaseRenderer.TYPE_HTTP, BaseRenderer.TYPE_NTP ] SHOW_DEFAULT_HEADER = False SHOW_DEFAULT_FOOTER = False def on_result(self, result, probes=None): return sanitise( json.dumps(result.raw_data, separators=(",", ":"))) + "\n" ripe-atlas-tools-1.2.2/ripe/atlas/tools/renderers/ssl_consistency.py000066400000000000000000000074761264541652500257330ustar00rootroot00000000000000# Copyright (c) 2015 RIPE NCC # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . from ..helpers.sanitisers import sanitise from .base import Renderer as BaseRenderer THRESHOLD = 80 # % class Renderer(BaseRenderer): RENDERS = [BaseRenderer.TYPE_SSLCERT] def __init__(self, *args, **kwargs): BaseRenderer.__init__(self, *args, **kwargs) self.uniqcerts = {} self.blob_list = [] def additional(self, results): self.gather_unique_certs(results) most_seen_cert = self.get_nprobes_ofpopular_cert() for cert_id in sorted( self.uniqcerts, key=lambda pk: self.uniqcerts[pk]["cnt"], reverse=True ): self.blob_list.append(self.render_certificate(cert_id)) if self.uniqcerts[cert_id]["cnt"] < most_seen_cert * THRESHOLD / 100: self.blob_list.extend(self.render_below_threshold(cert_id)) return "\n".join(self.blob_list) def gather_unique_certs(self, results): for result in results: self.bucketize_result_cert(result) def bucketize_result_cert(self, result): for certificate in result.certificates: cert_id = certificate.checksum_sha256 if cert_id not in self.uniqcerts: self.uniqcerts[cert_id] = { "cert": None, "cnt": 0, "probes": [] } self.uniqcerts[cert_id]["cert"] = certificate self.uniqcerts[cert_id]["cnt"] += 1 self.uniqcerts[cert_id]["probes"].append(result.probe) def get_nprobes_ofpopular_cert(self): """ Gets the number of probes that have seen the most popular (in terms of probes) cert. """ return max( [self.uniqcerts[cert_id]["cnt"] for cert_id in self.uniqcerts] ) def render_certificate(self, cert_id): """Renders the specific certificate""" certificate = self.uniqcerts[cert_id]["cert"] return self.render( "reports/ssl_consistency.txt", issuer_c=sanitise(certificate.issuer_c), issuer_o=sanitise(certificate.issuer_o), issuer_cn=sanitise(certificate.issuer_cn), subject_c=sanitise(certificate.subject_c), subject_o=sanitise(certificate.subject_o), subject_cn=sanitise(certificate.subject_cn), sha256fp=certificate.checksum_sha256, seenby=self.uniqcerts[cert_id]["cnt"], s="s" if self.uniqcerts[cert_id]["cnt"] > 1 else "" ) def render_below_threshold(self, cert_id): """ Print information about the given cert that is below our threshold of visibility. """ blob_list = [ " Below the threshold ({0}%)".format(THRESHOLD), " Probes that saw it: ", ] for probe in self.uniqcerts[cert_id]["probes"]: log = ( " ID: {id}, country code: {cc}, ASN (v4/v6): {asn4}/{asn6}" ).format( id=probe.id, cc=probe.country_code, asn4=probe.asn_v4, asn6=probe.asn_v6 ) blob_list.append(log) return blob_list def on_result(self, result): return "" ripe-atlas-tools-1.2.2/ripe/atlas/tools/renderers/sslcert.py000066400000000000000000000044461264541652500241620ustar00rootroot00000000000000# Copyright (c) 2015 RIPE NCC # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . import OpenSSL from ..helpers.sanitisers import sanitise from .base import Renderer as BaseRenderer class Renderer(BaseRenderer): RENDERS = [BaseRenderer.TYPE_SSLCERT] def on_result(self, result): r = "" for certificate in result.certificates: r += self.get_formatted_response(certificate) return "\nProbe #{0}\n{1}\n".format(result.probe_id, r) @classmethod def get_formatted_response(cls, certificate): x509 = OpenSSL.crypto.load_certificate( OpenSSL.crypto.FILETYPE_PEM, certificate.raw_data.replace("\\/", "/").replace("\n\n", "\n") ) pkey_type = x509.get_pubkey().type() # TODO: to be improved if pkey_type == 6: pkey_type_descr = "rsaEncryption" else: pkey_type_descr = pkey_type return cls.render( "reports/sslcert.txt", issuer_c=sanitise(certificate.issuer_c), issuer_o=sanitise(certificate.issuer_o), issuer_cn=sanitise(certificate.issuer_cn), not_before=certificate.valid_from, not_after=certificate.valid_until, subject_c=sanitise(certificate.subject_c), subject_o=sanitise(certificate.subject_o), subject_cn=sanitise(certificate.subject_cn), version=x509.get_version(), serial_number=x509.get_serial_number(), signature_algorithm=x509.get_signature_algorithm(), pkey_type=pkey_type_descr, pkey_bits=x509.get_pubkey().bits(), sha1fp=certificate.checksum_sha1, sha256fp=certificate.checksum_sha256 ) ripe-atlas-tools-1.2.2/ripe/atlas/tools/renderers/templates/000077500000000000000000000000001264541652500241175ustar00rootroot00000000000000ripe-atlas-tools-1.2.2/ripe/atlas/tools/renderers/templates/reports/000077500000000000000000000000001264541652500256155ustar00rootroot00000000000000ripe-atlas-tools-1.2.2/ripe/atlas/tools/renderers/templates/reports/aggregate_ping.txt000066400000000000000000000002331264541652500313170ustar00rootroot00000000000000-- {target} ping statistics --- {sent} packets transmitted, {received} received, {packet_loss}% loss rtt min/med/avg/max = {min}/{median}/{mean}/{max} ms ripe-atlas-tools-1.2.2/ripe/atlas/tools/renderers/templates/reports/dns.txt000066400000000000000000000010231264541652500271360ustar00rootroot00000000000000 - {response_id} - ; <<>> RIPE Atlas Tools <<>> {question_name} ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: {header_opcode}, status: {header_return_code}, id: {header_id} ;; flags: {header_flags}; QUERY: 1, ANSWER: {answer_count}, AUTHORITY: {authority_count}, ADDITIONAL: {additional_count} {edns}{question}{answers}{authorities}{additionals} ;; Query time: {response_time} msec ;; SERVER: {destination_address}#53({destination_address}) ;; WHEN: {created} ;; MSG SIZE rcvd: {response_size} ripe-atlas-tools-1.2.2/ripe/atlas/tools/renderers/templates/reports/ssl_consistency.txt000066400000000000000000000002701264541652500315770ustar00rootroot00000000000000Certificate: Issuer: C={issuer_c}, O={issuer_o}, CN={issuer_cn} Subject: C={subject_c}, O={subject_o}, CN={subject_cn} SHA256 Fingerprint={sha256fp} Seen by {seenby} probe{s} ripe-atlas-tools-1.2.2/ripe/atlas/tools/renderers/templates/reports/sslcert.txt000066400000000000000000000007771264541652500300500ustar00rootroot00000000000000 Certificate: Data: Version: {version} Serial Number: {serial_number} Signature Algorithm: {signature_algorithm} Issuer: C={issuer_c}, O={issuer_o}, CN={issuer_cn} Validity Not Before: {not_before} Not After : {not_after} Subject: C={subject_c}, O={subject_o}, CN={subject_cn} Subject Public Key Info: Public Key Algorithm: {pkey_type} Public-Key: ({pkey_bits} bit) SHA1 Fingerprint={sha1fp} SHA256 Fingerprint={sha256fp} ripe-atlas-tools-1.2.2/ripe/atlas/tools/renderers/traceroute.py000066400000000000000000000032311264541652500246470ustar00rootroot00000000000000# Copyright (c) 2015 RIPE NCC # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . from .base import Renderer as BaseRenderer from ..helpers.colours import colourise from ..helpers.sanitisers import sanitise class Renderer(BaseRenderer): RENDERS = [BaseRenderer.TYPE_TRACEROUTE] def on_result(self, result): r = "" for hop in result.hops: if hop.is_error: r += "{}\n".format( colourise(sanitise(hop.error_message), "red")) continue name = "" rtts = [] for packet in hop.packets: name = name or packet.origin or "*" if packet.rtt: rtts.append("{:8} ms".format(packet.rtt)) else: rtts.append(" *") r += "{:>3} {:39} {}\n".format( hop.index, sanitise(name), " ".join(rtts) ) return "\n{}\n\n{}".format( colourise("Probe #{}".format(result.probe_id), "bold"), r ) ripe-atlas-tools-1.2.2/ripe/atlas/tools/renderers/traceroute_aspath.py000066400000000000000000000060461264541652500262160ustar00rootroot00000000000000# Copyright (c) 2015 RIPE NCC # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . from ..ipdetails import IP from .base import Renderer as BaseRenderer class Renderer(BaseRenderer): RENDERS = [BaseRenderer.TYPE_TRACEROUTE] def __init__(self, *args, **kwargs): BaseRenderer.__init__(self, *args, **kwargs) self.paths = {} # Number of different ASs starting from the end of the traceroute path. # # TODO: if a method to pass options to renderers will be defined # this could be set by user input. self.RADIUS = 2 @staticmethod def _get_asns_for_output(asns, radius): asns_with_padding = [""] * radius + asns asns_with_padding = asns_with_padding[-radius:] return " ".join( ["{:>8}".format("AS{}".format(asn) if asn else "") for asn in asns_with_padding] ) def on_start(self): return "For each traceroute path toward the target, the " \ "last {} ASNs will be shown\n\n".format(self.RADIUS) def on_result(self, result): ip_hops = [] for hop in result.hops: for packet in hop.packets: if packet.origin: ip_hops.append(packet.origin) break asns = [] # starting from the last hop's IP, get up to ASNs for address in reversed(ip_hops): ip = IP(address) if ip.asn and ip.asn not in asns: asns.append(ip.asn) if len(asns) == self.RADIUS: break as_path = self._get_asns_for_output(list(reversed(asns)), self.RADIUS) if as_path not in self.paths: self.paths[as_path] = {} self.paths[as_path]['cnt'] = 0 self.paths[as_path]['responded'] = 0 self.paths[as_path]['cnt'] += 1 if result.destination_ip_responded: self.paths[as_path]['responded'] += 1 return "Probe #{:<5}: {}, {}completed\n".format( result.probe_id, as_path, "NOT " if not result.destination_ip_responded else "" ) def additional(self, results): s = "\nNumber of probes for each AS path:\n\n" for as_path in self.paths: s += " {}: {} probe{}, {} completed\n".format( as_path, self.paths[as_path]['cnt'], "s" if self.paths[as_path] > 1 else "", self.paths[as_path]['responded'] ) return s ripe-atlas-tools-1.2.2/ripe/atlas/tools/settings/000077500000000000000000000000001264541652500217705ustar00rootroot00000000000000ripe-atlas-tools-1.2.2/ripe/atlas/tools/settings/__init__.py000066400000000000000000000170531264541652500241070ustar00rootroot00000000000000# Copyright (c) 2015 RIPE NCC # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . import collections import copy import os import re import yaml class Configuration(object): """ A singleton configuration class that's smart enough to create a config out of defaults + yaml """ USER_CONFIG_DIR = os.path.join( os.environ["HOME"], ".config", "ripe-atlas-tools") USER_RC = os.path.join(USER_CONFIG_DIR, "rc") DEFAULT = { "authorisation": { "fetch": "", "create": "", }, "specification": { "af": 4, "description": "", "source": { "type": "area", "value": "WW", "requested": 50, }, "times": { "one-off": True, "interval": None, "start": None, "stop": None, }, "types": { "ping": { "packets": 3, "packet-interval": 1000, "size": 48 }, "traceroute": { "packets": 3, "size": 48, "protocol": "ICMP", "dont-fragment": False, "paris": 0, "first-hop": 1, "max-hops": 255, "port": 80, "destination-option-size": None, "hop-by-hop-option-size": None, "timeout": 4000 }, "sslcert": { "port": 443 }, "ntp": { "packets": 3, "timeout": 4000 }, "dns": { "set-cd-bit": False, "set-do-bit": False, "protocol": "UDP", "query-class": "IN", "query-type": "A", "query-argument": None, "set-nsid-bit": False, "udp-payload-size": 512, "set-rd-bit": True, "retry": 0 }, "http": { "header-bytes": 0, "version": "1.1", "method": "GET", "port": 80, "path": "/", "query-string": None, "user-agent": "RIPE ATLAS: https://atlas.ripe.net/", "body-bytes": None, "timing-verbosity": 0, }, }, "tags": { "ipv4": { "ping": { "include": [], "exclude": [] }, "traceroute": { "include": [], "exclude": [] }, "dns": { "include": [], "exclude": [] }, "sslcert": { "include": [], "exclude": [] }, "http": { "include": [], "exclude": [] }, "ntp": { "include": [], "exclude": [] }, "all": { "include": ["system-ipv4-works"], "exclude": [] }, }, "ipv6": { "ping": { "include": [], "exclude": [] }, "traceroute": { "include": [], "exclude": [] }, "dns": { "include": [], "exclude": [] }, "sslcert": { "include": [], "exclude": [] }, "http": { "include": [], "exclude": [] }, "ntp": { "include": [], "exclude": [] }, "all": { "include": ["system-ipv6-works"], "exclude": [] } } } }, "ripe-ncc": { "endpoint": "https://atlas.ripe.net", "version": 0, } } def get(self): r = copy.deepcopy(self.DEFAULT) if os.path.exists(self.USER_RC): with open(self.USER_RC) as y: custom = yaml.load(y) if custom: r = self.deep_update(r, custom) return r @classmethod def deep_update(cls, d, u): """ Updates a dictionary with another dictionary, only it goes deep. Stolen from http://stackoverflow.com/questions/3232943/ """ for k, v in u.items(): if isinstance(v, collections.Mapping): r = cls.deep_update(d.get(k, {}), v) d[k] = r else: d[k] = u[k] return d @staticmethod def write(config): """ PyYaml is incapable of preserving comments, or even specifying them as an argument to `.dump()` (http://pyyaml.org/ticket/114), so we have to do some regex gymnastics here to make sure that the config file remains easy for n00bs to read. """ template = os.path.join( os.path.dirname(__file__), "templates", "base.yaml") authorisation = re.compile("^authorisation:$", re.MULTILINE) tags = re.compile("^ tags:$", re.MULTILINE) specification = re.compile("^specification:$", re.MULTILINE) ripe = re.compile("^ripe-ncc:$", re.MULTILINE) with open(template) as t: payload = str(t.read()).format( payload=yaml.dump( config, default_flow_style=False ) ) payload = ripe.sub( "\n# Don't mess with these, or Bad Things may happen\n" "ripe-ncc:", payload ) payload = authorisation.sub( "# Authorisation\n" "authorisation:", payload ) payload = specification.sub( "\n# Measurement Creation\n" "specification:", payload ) payload = tags.sub( " # Tags added to probes selection\n" " tags:", payload ) with open(Configuration.USER_RC, "w") as rc: rc.write(payload) conf = Configuration().get() ripe-atlas-tools-1.2.2/ripe/atlas/tools/settings/templates/000077500000000000000000000000001264541652500237665ustar00rootroot00000000000000ripe-atlas-tools-1.2.2/ripe/atlas/tools/settings/templates/base.yaml000066400000000000000000000025611264541652500255700ustar00rootroot00000000000000# # This is the config file for the ripe-atlas command. It follows the YAML # formatting standard found here: http://yaml.org/ -- but the layout should be # rather intuitive. # # Lines that start with "#" (like this one) are considered comments and # are ignored by the parser. Beyond that, the configuration data is laid out # in a hierarchical fashion # # section_name: # variable_name: "some-value" # some_other_variable_name: 123 # a_subsection: # something: "something else" # # Where possible, we've commented this file to help you out, but you can also # use the ripe-atlas script to modify it for you. Simply type: # # $ ripe-atlas configure --help # # And the help text that appears will walk you through it. # # One good thing to put in here is your API key. This makes it easy to # generate measurements quickly. Defining your API key looks like this: # # authorisation: # create: "YOUR-API-KEY" # # Leaving this file empty won't hurt, but it will mean that you'll have to # use the --auth=YOUR-API-KEY flag every time you try to create a # measurement. # # What follows is a complete break down of all possible options, set initially # to their defaults. If you wish to change a default, simply change the value # to whatever you like and save this file. # ############################################################################ {payload} ripe-atlas-tools-1.2.2/ripe/atlas/tools/streaming.py000066400000000000000000000036201264541652500224740ustar00rootroot00000000000000# Copyright (c) 2015 RIPE NCC # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . from __future__ import absolute_import import sys from ripe.atlas.cousteau import AtlasStream from ripe.atlas.sagan import Result from .renderers import Renderer class CaptureLimitExceeded(Exception): pass class Stream(object): def __init__(self, capture_limit=None, timeout=None): self.captured = 0 self.capture_limit = capture_limit self.timeout = timeout def stream(self, renderer_name, kind, pk): renderer = Renderer.get_renderer(name=renderer_name, kind=kind)() def on_result_response(result, *args): sys.stdout.write(renderer.on_result(Result.get( result, on_error=Result.ACTION_IGNORE, on_malformation=Result.ACTION_IGNORE ))) self.captured += 1 if self.capture_limit and self.captured >= self.capture_limit: raise CaptureLimitExceeded() stream = AtlasStream() stream.connect() stream.bind_channel("result", on_result_response) try: stream.start_stream(stream_type="result", msm=pk) stream.timeout(self.timeout) except (KeyboardInterrupt, CaptureLimitExceeded) as e: stream.disconnect() raise e ripe-atlas-tools-1.2.2/ripe/atlas/tools/version.py000066400000000000000000000012611264541652500221670ustar00rootroot00000000000000# Copyright (c) 2015 RIPE NCC # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . __version__ = "1.2.2" ripe-atlas-tools-1.2.2/scripts/000077500000000000000000000000001264541652500164145ustar00rootroot00000000000000ripe-atlas-tools-1.2.2/scripts/adig000077700000000000000000000000001264541652500212222ripe-atlasustar00rootroot00000000000000ripe-atlas-tools-1.2.2/scripts/ahttp000077700000000000000000000000001264541652500214362ripe-atlasustar00rootroot00000000000000ripe-atlas-tools-1.2.2/scripts/antp000077700000000000000000000000001264541652500212602ripe-atlasustar00rootroot00000000000000ripe-atlas-tools-1.2.2/scripts/aping000077700000000000000000000000001264541652500214142ripe-atlasustar00rootroot00000000000000ripe-atlas-tools-1.2.2/scripts/asslcert000077700000000000000000000000001264541652500221362ripe-atlasustar00rootroot00000000000000ripe-atlas-tools-1.2.2/scripts/atraceroute000077700000000000000000000000001264541652500226342ripe-atlasustar00rootroot00000000000000ripe-atlas-tools-1.2.2/scripts/ripe-atlas000066400000000000000000000053461264541652500204100ustar00rootroot00000000000000#!/usr/bin/env python import importlib import os import pkgutil import re import sys from ripe.atlas.tools import commands from ripe.atlas.tools.exceptions import RipeAtlasToolsException class RipeAtlas(object): def __init__(self): self.command = None self.args = [] self.kwargs = {} @staticmethod def _get_available_commands(): """ Get a list of commands that we can execute. By default, we have a fixed that we make available in this directory, but the user can create her own plugins and store them at ~/.config/ripe-atlas-tools/commands/. If we find any files there, we add them to the list here. """ paths = [os.path.dirname(commands.__file__)] if "HOME" in os.environ: paths += [os.path.join( os.environ["HOME"], ".config", "ripe-atlas-tools", "commands")] r = [package_name for _, package_name, _ in pkgutil.iter_modules(paths)] r.remove("base") return r def _setup_command(self): caller = os.path.basename(sys.argv[0]) shortcut = re.match('^a(ping|traceroute|dig|sslcert|ntp|http)$', caller) available_commands = self._get_available_commands() if shortcut: self.command = "measure" sys.argv.insert(1, self._translate_shortcut(shortcut.group(1))) else: if len(sys.argv) < 2 or sys.argv[1] in ("-h", "--help"): raise RipeAtlasToolsException( "Usage: ripe-atlas <{}> [arguments]".format( "|".join(set(available_commands) - {"shibboleet"}) ) ) self.command = sys.argv.pop(1) @staticmethod def _translate_shortcut(shortcut): if shortcut == "dig": return "dns" return shortcut def main(self): self._setup_command() try: module = importlib.import_module( "ripe.atlas.tools.commands.{}".format(self.command)) # # If the imported module contains a `Factory` class, execute that # to get the `cmd` we're going to use. Otherwise, we expect there # to be a `Command` class in there. # if hasattr(module, "Factory"): cmd = module.Factory(*self.args, **self.kwargs).create() else: cmd = module.Command(*self.args, **self.kwargs) cmd.init_args() cmd.run() except ImportError: raise RipeAtlasToolsException("No such command.") if __name__ == '__main__': try: sys.exit(RipeAtlas().main()) except RipeAtlasToolsException as e: e.write() raise SystemExit() ripe-atlas-tools-1.2.2/setup.cfg000066400000000000000000000000261264541652500165440ustar00rootroot00000000000000[wheel] universal = 1 ripe-atlas-tools-1.2.2/setup.py000066400000000000000000000045541264541652500164470ustar00rootroot00000000000000import os from os.path import abspath, dirname, join from setuptools import setup __version__ = None exec(open("ripe/atlas/tools/version.py").read()) # Allow setup.py to be run from any path os.chdir(os.path.normpath(os.path.join(os.path.abspath(__file__), os.pardir))) # Get proper long description for package current_dir = dirname(abspath(__file__)) description = open(join(current_dir, "README.rst")).read() changes = open(join(current_dir, "CHANGES.rst")).read() long_description = '\n\n'.join([description, changes]) # Get the long description from README.md setup( name="ripe.atlas.tools", version=__version__, packages=["ripe", "ripe.atlas", "ripe.atlas.tools"], namespace_packages=["ripe", "ripe.atlas"], include_package_data=True, license="GPLv3", description="The official command line client for RIPE Atlas", long_description=long_description, url="https://github.com/RIPE-NCC/ripe-atlas-tools", download_url="https://github.com/RIPE-NCC/ripe-atlas-tools", author="The RIPE Atlas team", author_email="atlas@ripe.net", maintainer="The RIPE Atlas team", maintainer_email="atlas@ripe.net", install_requires=[ "python-dateutil", "requests>=2.7.0", "ripe.atlas.cousteau>=1.0.7", "ripe.atlas.sagan>=1.1.8", "tzlocal", "pyyaml", "pyOpenSSL>=0.13", ], tests_require=[ "nose", "coverage", "mock", ], extras_require={ "doc": ["sphinx", "sphinx_rtd_theme"], "fast": ["ujson"], }, test_suite="nose.collector", scripts=[ "scripts/aping", "scripts/atraceroute", "scripts/adig", "scripts/asslcert", "scripts/ahttp", "scripts/antp", "scripts/ripe-atlas", ], keywords=['RIPE', 'RIPE NCC', 'RIPE Atlas', 'Command Line'], classifiers=[ "License :: OSI Approved :: GNU General Public License v3 (GPLv3)", "Operating System :: POSIX", "Operating System :: Unix", "Programming Language :: Python", "Programming Language :: Python :: 2.7", "Programming Language :: Python :: 3.3", "Programming Language :: Python :: 3.4", "Programming Language :: Python :: Implementation :: CPython", "Programming Language :: Python :: Implementation :: PyPy", "Topic :: Internet :: WWW/HTTP", ], ) ripe-atlas-tools-1.2.2/tests/000077500000000000000000000000001264541652500160675ustar00rootroot00000000000000ripe-atlas-tools-1.2.2/tests/README000066400000000000000000000001501264541652500167430ustar00rootroot00000000000000How to run tests Make sure you have nose installed and run from the main directory: nosetests tests ripe-atlas-tools-1.2.2/tests/__init__.py000066400000000000000000000024131264541652500202000ustar00rootroot00000000000000# Copyright (c) 2015 RIPE NCC # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . from .aggregators import TestAggregators from .commands import ( TestProbesCommand, TestMeasureCommand, TestMeasurementsCommand, TestReportCommand ) from .helpers import TestArgumentTypeHelper from .renderers import ( TestPingRenderer, TestHttpRenderer, TestSSLConsistency, TestAggregatePing, TestRawRenderer, ) __all__ = [ TestAggregators, TestProbesCommand, TestMeasureCommand, TestMeasurementsCommand, TestReportCommand, TestArgumentTypeHelper, TestPingRenderer, TestHttpRenderer, TestSSLConsistency, TestAggregatePing, TestRawRenderer, ] ripe-atlas-tools-1.2.2/tests/aggregators/000077500000000000000000000000001264541652500203745ustar00rootroot00000000000000ripe-atlas-tools-1.2.2/tests/aggregators/__init__.py000066400000000000000000000013461264541652500225110ustar00rootroot00000000000000# Copyright (c) 2015 RIPE NCC # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . from .test_aggregators import TestAggregators __all__ = [TestAggregators] ripe-atlas-tools-1.2.2/tests/aggregators/test_aggregators.py000066400000000000000000000567521264541652500243310ustar00rootroot00000000000000# Copyright (c) 2015 RIPE NCC # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . import unittest from collections import namedtuple from ripe.atlas.tools.aggregators.base import ( aggregate, ValueKeyAggregator, RangeKeyAggregator ) class TestAggregators(unittest.TestCase): def setUp(self): self.results = [] self.probes = [] self.Result = namedtuple('Result', 'id probe rtt source prefix') self.Probe = namedtuple('Probe', 'id country asn status') probes = [ (1, "GR", 333, "Connected"), (2, "NL", 334, "Connected"), (3, "SE", 335, "Connected"), (4, "SE", 336, "DisConnected"), (5, "SE", 337, "DisConnected"), (6, "SE", 335, "DisConnected"), (7, "IN", 335, "Connected"), (8, "DE", 338, "Connected"), (9, "DK", 348, "Connected"), (10, "DE", 338, "NeverConnected"), (11, "DE", 340, "DisConnected"), ] results = [ (1, 3, "127.0.0.1", "192/8"), (2, 34.0, "127.0.0.1", "192/8"), (3, 35.0, "127.0.0.1", "193/8"), (4, 6, "127.0.0.1", "193/8"), (5, 17, "127.0.0.1", "192/8"), (6, 15, "127.0.0.1", "195/8"), (7, 35, "127.0.0.1", "192/8"), (8, 28, "127.0.0.1", "192/8"), (9, 48, "127.0.0.1", "195/8"), (10, 28, "127.0.0.1", "194/8"), (11, 40, "127.0.0.1", "193/8"), ] for index, result in enumerate(results): prb = self.Probe( id=probes[index][0], country=probes[index][1], asn=probes[index][2], status=probes[index][3] ) r = self.Result( id=result[0], probe=prb, rtt=result[1], source=result[2], prefix=result[3] ) self.results.append(r) self.probes.append(prb) def test_value_aggregation(self): """Test 2 tier aggregation with object attribute as key.""" keys = [ValueKeyAggregator(key='probe.country'), ValueKeyAggregator(key='rtt')] buckets = aggregate(self.results, keys) expected_output = { 'COUNTRY: DE': { 'RTT: 28': [self.Result(id=8, probe=self.Probe(id=8, country='DE', asn=338, status='Connected'), rtt=28, source='127.0.0.1', prefix='192/8'), self.Result(id=10, probe=self.Probe(id=10, country='DE', asn=338, status='NeverConnected'), rtt=28, source='127.0.0.1', prefix='194/8')], 'RTT: 40': [self.Result(id=11, probe=self.Probe(id=11, country='DE', asn=340, status='DisConnected'), rtt=40, source='127.0.0.1', prefix='193/8')] }, 'COUNTRY: DK': {'RTT: 48': [self.Result(id=9, probe=self.Probe(id=9, country='DK', asn=348, status='Connected'), rtt=48, source='127.0.0.1', prefix='195/8')]}, 'COUNTRY: GR': {'RTT: 3': [self.Result(id=1, probe=self.Probe(id=1, country='GR', asn=333, status='Connected'), rtt=3, source='127.0.0.1', prefix='192/8')]}, 'COUNTRY: IN': {'RTT: 35': [self.Result(id=7, probe=self.Probe(id=7, country='IN', asn=335, status='Connected'), rtt=35, source='127.0.0.1', prefix='192/8')]}, 'COUNTRY: NL': {'RTT: 34.0': [self.Result(id=2, probe=self.Probe(id=2, country='NL', asn=334, status='Connected'), rtt=34.0, source='127.0.0.1', prefix='192/8')]}, 'COUNTRY: SE': { 'RTT: 6': [self.Result(id=4, probe=self.Probe(id=4, country='SE', asn=336, status='DisConnected'), rtt=6, source='127.0.0.1', prefix='193/8')], 'RTT: 15': [self.Result(id=6, probe=self.Probe(id=6, country='SE', asn=335, status='DisConnected'), rtt=15, source='127.0.0.1', prefix='195/8')], 'RTT: 17': [self.Result(id=5, probe=self.Probe(id=5, country='SE', asn=337, status='DisConnected'), rtt=17, source='127.0.0.1', prefix='192/8')], 'RTT: 35.0': [self.Result(id=3, probe=self.Probe(id=3, country='SE', asn=335, status='Connected'), rtt=35.0, source='127.0.0.1', prefix='193/8')] } } self.assertEquals(buckets, expected_output) def test_value_aggregation1(self): """Test 1 tier aggregation""" keys = [ValueKeyAggregator(key='probe.country')] buckets = aggregate(self.results, keys) expected_output = { 'COUNTRY: DE': [self.Result(id=8, probe=self.Probe(id=8, country='DE', asn=338, status='Connected'), rtt=28, source='127.0.0.1', prefix='192/8'), self.Result(id=10, probe=self.Probe(id=10, country='DE', asn=338, status='NeverConnected'), rtt=28, source='127.0.0.1', prefix='194/8'), self.Result(id=11, probe=self.Probe(id=11, country='DE', asn=340, status='DisConnected'), rtt=40, source='127.0.0.1', prefix='193/8')], 'COUNTRY: DK': [self.Result(id=9, probe=self.Probe(id=9, country='DK', asn=348, status='Connected'), rtt=48, source='127.0.0.1', prefix='195/8')], 'COUNTRY: GR': [self.Result(id=1, probe=self.Probe(id=1, country='GR', asn=333, status='Connected'), rtt=3, source='127.0.0.1', prefix='192/8')], 'COUNTRY: IN': [self.Result(id=7, probe=self.Probe(id=7, country='IN', asn=335, status='Connected'), rtt=35, source='127.0.0.1', prefix='192/8')], 'COUNTRY: NL': [self.Result(id=2, probe=self.Probe(id=2, country='NL', asn=334, status='Connected'), rtt=34.0, source='127.0.0.1', prefix='192/8')], 'COUNTRY: SE': [self.Result(id=3, probe=self.Probe(id=3, country='SE', asn=335, status='Connected'), rtt=35.0, source='127.0.0.1', prefix='193/8'), self.Result(id=4, probe=self.Probe(id=4, country='SE', asn=336, status='DisConnected'), rtt=6, source='127.0.0.1', prefix='193/8'), self.Result(id=5, probe=self.Probe(id=5, country='SE', asn=337, status='DisConnected'), rtt=17, source='127.0.0.1', prefix='192/8'), self.Result(id=6, probe=self.Probe(id=6, country='SE', asn=335, status='DisConnected'), rtt=15, source='127.0.0.1', prefix='195/8')] } self.assertEquals(buckets, expected_output) def test_value_aggregation2(self): """Test 2 tier aggregation.""" keys = [ValueKeyAggregator(key='prefix'), ValueKeyAggregator(key='rtt')] buckets = aggregate(self.results, keys) expected_output = { 'PREFIX: 192/8': { 'RTT: 3': [self.Result(id=1, probe=self.Probe(id=1, country='GR', asn=333, status='Connected'), rtt=3, source='127.0.0.1', prefix='192/8')], 'RTT: 17': [self.Result(id=5, probe=self.Probe(id=5, country='SE', asn=337, status='DisConnected'), rtt=17, source='127.0.0.1', prefix='192/8')], 'RTT: 28': [self.Result(id=8, probe=self.Probe(id=8, country='DE', asn=338, status='Connected'), rtt=28, source='127.0.0.1', prefix='192/8')], 'RTT: 34.0': [self.Result(id=2, probe=self.Probe(id=2, country='NL', asn=334, status='Connected'), rtt=34.0, source='127.0.0.1', prefix='192/8')], 'RTT: 35': [self.Result(id=7, probe=self.Probe(id=7, country='IN', asn=335, status='Connected'), rtt=35, source='127.0.0.1', prefix='192/8')] }, 'PREFIX: 193/8': { 'RTT: 6': [self.Result(id=4, probe=self.Probe(id=4, country='SE', asn=336, status='DisConnected'), rtt=6, source='127.0.0.1', prefix='193/8')], 'RTT: 35.0': [self.Result(id=3, probe=self.Probe(id=3, country='SE', asn=335, status='Connected'), rtt=35.0, source='127.0.0.1', prefix='193/8')], 'RTT: 40': [self.Result(id=11, probe=self.Probe(id=11, country='DE', asn=340, status='DisConnected'), rtt=40, source='127.0.0.1', prefix='193/8')] }, 'PREFIX: 194/8': { 'RTT: 28': [self.Result(id=10, probe=self.Probe(id=10, country='DE', asn=338, status='NeverConnected'), rtt=28, source='127.0.0.1', prefix='194/8')] }, 'PREFIX: 195/8': { 'RTT: 15': [self.Result(id=6, probe=self.Probe(id=6, country='SE', asn=335, status='DisConnected'), rtt=15, source='127.0.0.1', prefix='195/8')], 'RTT: 48': [self.Result(id=9, probe=self.Probe(id=9, country='DK', asn=348, status='Connected'), rtt=48, source='127.0.0.1', prefix='195/8')] } } self.assertEquals(buckets, expected_output) def test_value_aggregation3(self): """Test 3 tier aggregation.""" keys = [ValueKeyAggregator(key='prefix'), ValueKeyAggregator(key='rtt'), ValueKeyAggregator(key='source')] buckets = aggregate(self.results, keys) expected_output = { 'PREFIX: 192/8': { 'RTT: 3': { 'SOURCE: 127.0.0.1': [self.Result(id=1, probe=self.Probe(id=1, country='GR', asn=333, status='Connected'), rtt=3, source='127.0.0.1', prefix='192/8')] }, 'RTT: 17': { 'SOURCE: 127.0.0.1': [self.Result(id=5, probe=self.Probe(id=5, country='SE', asn=337, status='DisConnected'), rtt=17, source='127.0.0.1', prefix='192/8')] }, 'RTT: 28': { 'SOURCE: 127.0.0.1': [self.Result(id=8, probe=self.Probe(id=8, country='DE', asn=338, status='Connected'), rtt=28, source='127.0.0.1', prefix='192/8')] }, 'RTT: 34.0': { 'SOURCE: 127.0.0.1': [self.Result(id=2, probe=self.Probe(id=2, country='NL', asn=334, status='Connected'), rtt=34.0, source='127.0.0.1', prefix='192/8')] }, 'RTT: 35': { 'SOURCE: 127.0.0.1': [self.Result(id=7, probe=self.Probe(id=7, country='IN', asn=335, status='Connected'), rtt=35, source='127.0.0.1', prefix='192/8')] } }, 'PREFIX: 193/8': { 'RTT: 6': { 'SOURCE: 127.0.0.1': [self.Result(id=4, probe=self.Probe(id=4, country='SE', asn=336, status='DisConnected'), rtt=6, source='127.0.0.1', prefix='193/8')] }, 'RTT: 35.0': { 'SOURCE: 127.0.0.1': [self.Result(id=3, probe=self.Probe(id=3, country='SE', asn=335, status='Connected'), rtt=35.0, source='127.0.0.1', prefix='193/8')] }, 'RTT: 40': { 'SOURCE: 127.0.0.1': [self.Result(id=11, probe=self.Probe(id=11, country='DE', asn=340, status='DisConnected'), rtt=40, source='127.0.0.1', prefix='193/8')] } }, 'PREFIX: 194/8': { 'RTT: 28': { 'SOURCE: 127.0.0.1': [self.Result(id=10, probe=self.Probe(id=10, country='DE', asn=338, status='NeverConnected'), rtt=28, source='127.0.0.1', prefix='194/8')] } }, 'PREFIX: 195/8': { 'RTT: 15': { 'SOURCE: 127.0.0.1': [self.Result(id=6, probe=self.Probe(id=6, country='SE', asn=335, status='DisConnected'), rtt=15, source='127.0.0.1', prefix='195/8')] }, 'RTT: 48': { 'SOURCE: 127.0.0.1': [self.Result(id=9, probe=self.Probe(id=9, country='DK', asn=348, status='Connected'), rtt=48, source='127.0.0.1', prefix='195/8')] } } } self.assertEquals(buckets, expected_output) def test_value_aggregation4(self): """Test 2 tier aggregation with probes.""" keys = [ValueKeyAggregator(key='status'), ValueKeyAggregator(key='asn')] buckets = aggregate(self.probes, keys) expected_output = { 'STATUS: Connected': { 'ASN: 338': [self.Probe(id=8, country='DE', asn=338, status='Connected')], 'ASN: 348': [self.Probe(id=9, country='DK', asn=348, status='Connected')], 'ASN: 333': [self.Probe(id=1, country='GR', asn=333, status='Connected')], 'ASN: 334': [self.Probe(id=2, country='NL', asn=334, status='Connected')], 'ASN: 335': [self.Probe(id=3, country='SE', asn=335, status='Connected'), self.Probe(id=7, country='IN', asn=335, status='Connected')] }, 'STATUS: DisConnected': { 'ASN: 336': [self.Probe(id=4, country='SE', asn=336, status='DisConnected')], 'ASN: 337': [self.Probe(id=5, country='SE', asn=337, status='DisConnected')], 'ASN: 340': [self.Probe(id=11, country='DE', asn=340, status='DisConnected')], 'ASN: 335': [self.Probe(id=6, country='SE', asn=335, status='DisConnected')] }, 'STATUS: NeverConnected': { 'ASN: 338': [self.Probe(id=10, country='DE', asn=338, status='NeverConnected')] } } self.assertEquals(buckets, expected_output) def test_range_aggregation(self): """Test 1 tier range aggregation""" keys = [RangeKeyAggregator(ranges=[10, 20, 30, 40, 50], key='rtt')] buckets = aggregate(self.results, keys) expected_output = { 'RTT: 10-20': [self.Result(id=5, probe=self.Probe(id=5, country='SE', asn=337, status='DisConnected'), rtt=17, source='127.0.0.1', prefix='192/8'), self.Result(id=6, probe=self.Probe(id=6, country='SE', asn=335, status='DisConnected'), rtt=15, source='127.0.0.1', prefix='195/8')], 'RTT: 20-30': [self.Result(id=8, probe=self.Probe(id=8, country='DE', asn=338, status='Connected'), rtt=28, source='127.0.0.1', prefix='192/8'), self.Result(id=10, probe=self.Probe(id=10, country='DE', asn=338, status='NeverConnected'), rtt=28, source='127.0.0.1', prefix='194/8')], 'RTT: 30-40': [self.Result(id=2, probe=self.Probe(id=2, country='NL', asn=334, status='Connected'), rtt=34.0, source='127.0.0.1', prefix='192/8'), self.Result(id=3, probe=self.Probe(id=3, country='SE', asn=335, status='Connected'), rtt=35.0, source='127.0.0.1', prefix='193/8'), self.Result(id=7, probe=self.Probe(id=7, country='IN', asn=335, status='Connected'), rtt=35, source='127.0.0.1', prefix='192/8'), self.Result(id=11, probe=self.Probe(id=11, country='DE', asn=340, status='DisConnected'), rtt=40, source='127.0.0.1', prefix='193/8')], 'RTT: 40-50': [self.Result(id=9, probe=self.Probe(id=9, country='DK', asn=348, status='Connected'), rtt=48, source='127.0.0.1', prefix='195/8')], 'RTT: < 10': [self.Result(id=1, probe=self.Probe(id=1, country='GR', asn=333, status='Connected'), rtt=3, source='127.0.0.1', prefix='192/8'), self.Result(id=4, probe=self.Probe(id=4, country='SE', asn=336, status='DisConnected'), rtt=6, source='127.0.0.1', prefix='193/8')] } self.maxDiff = None import pprint pp = pprint.PrettyPrinter() pp.pprint(buckets) self.assertEquals(buckets, expected_output) def test_range_aggregation1(self): """Test 1 tier range aggregation (1)""" keys = [RangeKeyAggregator(ranges=[10, 20, 30, 35, 40, 50, 60], key='rtt')] buckets = aggregate(self.results, keys) expected_output = { 'RTT: 10-20': [self.Result(id=5, probe=self.Probe(id=5, country='SE', asn=337, status='DisConnected'), rtt=17, source='127.0.0.1', prefix='192/8'), self.Result(id=6, probe=self.Probe(id=6, country='SE', asn=335, status='DisConnected'), rtt=15, source='127.0.0.1', prefix='195/8')], 'RTT: 20-30': [self.Result(id=8, probe=self.Probe(id=8, country='DE', asn=338, status='Connected'), rtt=28, source='127.0.0.1', prefix='192/8'), self.Result(id=10, probe=self.Probe(id=10, country='DE', asn=338, status='NeverConnected'), rtt=28, source='127.0.0.1', prefix='194/8')], 'RTT: 30-35': [self.Result(id=2, probe=self.Probe(id=2, country='NL', asn=334, status='Connected'), rtt=34.0, source='127.0.0.1', prefix='192/8'), self.Result(id=3, probe=self.Probe(id=3, country='SE', asn=335, status='Connected'), rtt=35.0, source='127.0.0.1', prefix='193/8'), self.Result(id=7, probe=self.Probe(id=7, country='IN', asn=335, status='Connected'), rtt=35, source='127.0.0.1', prefix='192/8')], 'RTT: 35-40': [self.Result(id=11, probe=self.Probe(id=11, country='DE', asn=340, status='DisConnected'), rtt=40, source='127.0.0.1', prefix='193/8')], 'RTT: 40-50': [self.Result(id=9, probe=self.Probe(id=9, country='DK', asn=348, status='Connected'), rtt=48, source='127.0.0.1', prefix='195/8')], 'RTT: < 10': [self.Result(id=1, probe=self.Probe(id=1, country='GR', asn=333, status='Connected'), rtt=3, source='127.0.0.1', prefix='192/8'), self.Result(id=4, probe=self.Probe(id=4, country='SE', asn=336, status='DisConnected'), rtt=6, source='127.0.0.1', prefix='193/8')] } self.assertEquals(buckets, expected_output) def test_range_aggregation2(self): """Test 1 tier range aggregation (2)""" keys = [RangeKeyAggregator(ranges=[1, 5, 10, 20, 30, 35], key='rtt')] buckets = aggregate(self.results, keys) expected_output = { 'RTT: 1-5': [self.Result(id=1, probe=self.Probe(id=1, country='GR', asn=333, status='Connected'), rtt=3, source='127.0.0.1', prefix='192/8')], 'RTT: 10-20': [self.Result(id=5, probe=self.Probe(id=5, country='SE', asn=337, status='DisConnected'), rtt=17, source='127.0.0.1', prefix='192/8'), self.Result(id=6, probe=self.Probe(id=6, country='SE', asn=335, status='DisConnected'), rtt=15, source='127.0.0.1', prefix='195/8')], 'RTT: 20-30': [self.Result(id=8, probe=self.Probe(id=8, country='DE', asn=338, status='Connected'), rtt=28, source='127.0.0.1', prefix='192/8'), self.Result(id=10, probe=self.Probe(id=10, country='DE', asn=338, status='NeverConnected'), rtt=28, source='127.0.0.1', prefix='194/8')], 'RTT: 30-35': [self.Result(id=2, probe=self.Probe(id=2, country='NL', asn=334, status='Connected'), rtt=34.0, source='127.0.0.1', prefix='192/8'), self.Result(id=3, probe=self.Probe(id=3, country='SE', asn=335, status='Connected'), rtt=35.0, source='127.0.0.1', prefix='193/8'), self.Result(id=7, probe=self.Probe(id=7, country='IN', asn=335, status='Connected'), rtt=35, source='127.0.0.1', prefix='192/8')], 'RTT: 5-10': [self.Result(id=4, probe=self.Probe(id=4, country='SE', asn=336, status='DisConnected'), rtt=6, source='127.0.0.1', prefix='193/8')], 'RTT: > 35': [self.Result(id=9, probe=self.Probe(id=9, country='DK', asn=348, status='Connected'), rtt=48, source='127.0.0.1', prefix='195/8'), self.Result(id=11, probe=self.Probe(id=11, country='DE', asn=340, status='DisConnected'), rtt=40, source='127.0.0.1', prefix='193/8')]} self.assertEquals(buckets, expected_output) def test_mixed_aggregation(self): """Test value and range aggregation together""" keys = [RangeKeyAggregator(ranges=[1, 5, 10, 20, 30, 35], key='rtt'), ValueKeyAggregator(key='probe.country')] buckets = aggregate(self.results, keys) expected_output = { 'RTT: 1-5': { 'COUNTRY: GR': [self.Result(id=1, probe=self.Probe(id=1, country='GR', asn=333, status='Connected'), rtt=3, source='127.0.0.1', prefix='192/8')]}, 'RTT: 10-20': { 'COUNTRY: SE': [self.Result(id=5, probe=self.Probe(id=5, country='SE', asn=337, status='DisConnected'), rtt=17, source='127.0.0.1', prefix='192/8'), self.Result(id=6, probe=self.Probe(id=6, country='SE', asn=335, status='DisConnected'), rtt=15, source='127.0.0.1', prefix='195/8')] }, 'RTT: 20-30': { 'COUNTRY: DE': [self.Result(id=8, probe=self.Probe(id=8, country='DE', asn=338, status='Connected'), rtt=28, source='127.0.0.1', prefix='192/8'), self.Result(id=10, probe=self.Probe(id=10, country='DE', asn=338, status='NeverConnected'), rtt=28, source='127.0.0.1', prefix='194/8')] }, 'RTT: 30-35': { 'COUNTRY: IN': [self.Result(id=7, probe=self.Probe(id=7, country='IN', asn=335, status='Connected'), rtt=35, source='127.0.0.1', prefix='192/8')], 'COUNTRY: NL': [self.Result(id=2, probe=self.Probe(id=2, country='NL', asn=334, status='Connected'), rtt=34.0, source='127.0.0.1', prefix='192/8')], 'COUNTRY: SE': [self.Result(id=3, probe=self.Probe(id=3, country='SE', asn=335, status='Connected'), rtt=35.0, source='127.0.0.1', prefix='193/8')]}, 'RTT: 5-10': {'COUNTRY: SE': [self.Result(id=4, probe=self.Probe(id=4, country='SE', asn=336, status='DisConnected'), rtt=6, source='127.0.0.1', prefix='193/8')]}, 'RTT: > 35': { 'COUNTRY: DE': [self.Result(id=11, probe=self.Probe(id=11, country='DE', asn=340, status='DisConnected'), rtt=40, source='127.0.0.1', prefix='193/8')], 'COUNTRY: DK': [self.Result(id=9, probe=self.Probe(id=9, country='DK', asn=348, status='Connected'), rtt=48, source='127.0.0.1', prefix='195/8')] } } self.assertEquals(buckets, expected_output) def test_mixed_aggregation1(self): """Test value and range aggregation together (1)""" keys = [RangeKeyAggregator(ranges=[10, 200], key='rtt'), ValueKeyAggregator(key='probe.country'), ValueKeyAggregator(key='probe.status')] buckets = aggregate(self.results, keys) expected_output = { 'RTT: 10-200': { 'COUNTRY: DE': { 'STATUS: Connected': [self.Result(id=8, probe=self.Probe(id=8, country='DE', asn=338, status='Connected'), rtt=28, source='127.0.0.1', prefix='192/8')], 'STATUS: DisConnected': [self.Result(id=11, probe=self.Probe(id=11, country='DE', asn=340, status='DisConnected'), rtt=40, source='127.0.0.1', prefix='193/8')], 'STATUS: NeverConnected': [self.Result(id=10, probe=self.Probe(id=10, country='DE', asn=338, status='NeverConnected'), rtt=28, source='127.0.0.1', prefix='194/8')] }, 'COUNTRY: DK': {'STATUS: Connected': [self.Result(id=9, probe=self.Probe(id=9, country='DK', asn=348, status='Connected'), rtt=48, source='127.0.0.1', prefix='195/8')]}, 'COUNTRY: IN': {'STATUS: Connected': [self.Result(id=7, probe=self.Probe(id=7, country='IN', asn=335, status='Connected'), rtt=35, source='127.0.0.1', prefix='192/8')]}, 'COUNTRY: NL': {'STATUS: Connected': [self.Result(id=2, probe=self.Probe(id=2, country='NL', asn=334, status='Connected'), rtt=34.0, source='127.0.0.1', prefix='192/8')]}, 'COUNTRY: SE': { 'STATUS: Connected': [self.Result(id=3, probe=self.Probe(id=3, country='SE', asn=335, status='Connected'), rtt=35.0, source='127.0.0.1', prefix='193/8')], 'STATUS: DisConnected': [self.Result(id=5, probe=self.Probe(id=5, country='SE', asn=337, status='DisConnected'), rtt=17, source='127.0.0.1', prefix='192/8'), self.Result(id=6, probe=self.Probe(id=6, country='SE', asn=335, status='DisConnected'), rtt=15, source='127.0.0.1', prefix='195/8')] } }, 'RTT: < 10': { 'COUNTRY: GR': {'STATUS: Connected': [self.Result(id=1, probe=self.Probe(id=1, country='GR', asn=333, status='Connected'), rtt=3, source='127.0.0.1', prefix='192/8')]}, 'COUNTRY: SE': {'STATUS: DisConnected': [self.Result(id=4, probe=self.Probe(id=4, country='SE', asn=336, status='DisConnected'), rtt=6, source='127.0.0.1', prefix='193/8')]} } } self.assertEquals(buckets, expected_output) ripe-atlas-tools-1.2.2/tests/base.py000066400000000000000000000026001264541652500173510ustar00rootroot00000000000000# Copyright (c) 2015 RIPE NCC # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . import sys from contextlib import contextmanager try: from cStringIO import StringIO except ImportError: # Python 3 from io import StringIO @contextmanager def capture_sys_output(): """ Wrap a block with this, and it'll capture standard out and standard error into handy variables: with capture_sys_output() as (stdout, stderr): self.cmd.run() More info: https://stackoverflow.com/questions/18651705/ """ capture_out, capture_err = StringIO(), StringIO() current_out, current_err = sys.stdout, sys.stderr try: sys.stdout, sys.stderr = capture_out, capture_err yield capture_out, capture_err finally: sys.stdout, sys.stderr = current_out, current_err ripe-atlas-tools-1.2.2/tests/commands/000077500000000000000000000000001264541652500176705ustar00rootroot00000000000000ripe-atlas-tools-1.2.2/tests/commands/__init__.py000066400000000000000000000016621264541652500220060ustar00rootroot00000000000000# Copyright (c) 2015 RIPE NCC # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . from .measure import TestMeasureCommand from .measurements import TestMeasurementsCommand from .probes import TestProbesCommand from .report import TestReportCommand __all__ = [ TestMeasureCommand, TestMeasurementsCommand, TestProbesCommand, TestReportCommand ] ripe-atlas-tools-1.2.2/tests/commands/measure.py000066400000000000000000000737601264541652500217200ustar00rootroot00000000000000# -*- coding: UTF-8 -*- # Copyright (c) 2015 RIPE NCC # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . import copy import unittest from random import randint # Python 3.4+ comes with mock in unittest try: from unittest import mock except ImportError: import mock from ripe.atlas.tools.commands.measure.base import Command from ripe.atlas.tools.commands.measure import ( PingMeasureCommand, TracerouteMeasureCommand, DnsMeasureCommand, SslcertMeasureCommand, HttpMeasureCommand, NtpMeasureCommand, ) from ripe.atlas.tools.exceptions import RipeAtlasToolsException from ripe.atlas.tools.settings import Configuration from ..base import capture_sys_output class TestMeasureCommand(unittest.TestCase): CONF = "ripe.atlas.tools.commands.measure.base.conf" KINDS = { "ping": PingMeasureCommand, "traceroute": TracerouteMeasureCommand, "dns": DnsMeasureCommand, "sslcert": SslcertMeasureCommand, "http": HttpMeasureCommand, "ntp": NtpMeasureCommand, } def setUp(self): self.maxDiff = None def test_no_arguments(self): with capture_sys_output(): with self.assertRaises(RipeAtlasToolsException) as e: Command().init_args([]) self.assertTrue( str(e.exception).startswith("Usage: ripe-atlas measure <")) def test_bad_type_argument(self): with capture_sys_output(): with self.assertRaises(RipeAtlasToolsException) as e: Command().init_args(["not-a-type"]) self.assertTrue( str(e.exception).startswith("Usage: ripe-atlas measure <")) @mock.patch(CONF, Configuration.DEFAULT) def test_dry_run(self): with capture_sys_output() as (stdout, stderr): cmd = PingMeasureCommand() cmd.init_args(["ping", "--target", "ripe.net", "--dry-run"]) cmd.run() expected = ( "\n" "Definitions:\n" "================================================================================\n" "target ripe.net\n" "packet_interval 1000\n" "description Ping measurement to ripe.net\n" "af 4\n" "packets 3\n" "size 48\n" "\n" "Sources:\n" "================================================================================\n" "requested 50\n" "type area\n" "value WW\n" "tags\n" " include system-ipv4-works\n" " exclude \n" "\n" ) self.assertEqual( set(stdout.getvalue().split("\n")), set(expected.split("\n")) ) with capture_sys_output() as (stdout, stderr): cmd = PingMeasureCommand() cmd.init_args([ "ping", "--target", "ripe.net", "--af", "6", "--include-tag", "alpha", "--include-tag", "bravo", "--include-tag", "charlie", "--exclude-tag", "delta", "--exclude-tag", "echo", "--exclude-tag", "foxtrot", "--dry-run" ]) cmd.run() expected = ( "\n" "Definitions:\n" "================================================================================\n" "target ripe.net\n" "packet_interval 1000\n" "description Ping measurement to ripe.net\n" "af 6\n" "packets 3\n" "size 48\n" "\n" "Sources:\n" "================================================================================\n" "requested 50\n" "type area\n" "value WW\n" "tags\n" " include alpha, bravo, charlie\n" " exclude delta, echo, foxtrot\n" "\n" ) self.assertEqual( set(stdout.getvalue().split("\n")), set(expected.split("\n")) ) def test_clean_target(self): cmd = PingMeasureCommand() with capture_sys_output(): cmd.init_args(["ping", "--target", "ripe.net"]) self.assertEqual(cmd.clean_target(), "ripe.net") cmd = DnsMeasureCommand() with capture_sys_output(): cmd.init_args(["dns", "--query-argument", "ripe.net"]) self.assertEqual(cmd.clean_target(), None) @mock.patch(CONF, Configuration.DEFAULT) def test_get_measurement_kwargs_ping(self): spec = Configuration.DEFAULT["specification"]["types"]["ping"] cmd = PingMeasureCommand() cmd.init_args([ "ping", "--target", "ripe.net" ]) self.assertEqual( cmd._get_measurement_kwargs(), { "af": Configuration.DEFAULT["specification"]["af"], "description": "Ping measurement to ripe.net", "target": "ripe.net", "packets": spec["packets"], "packet_interval": spec["packet-interval"], "size": spec["size"] } ) cmd = PingMeasureCommand() cmd.init_args([ "ping", "--target", "ripe.net", "--af", "6", "--description", "This is my description", "--packets", "7", "--packet-interval", "200", "--size", "24" ]) self.assertEqual( cmd._get_measurement_kwargs(), { "af": 6, "description": "This is my description", "target": "ripe.net", "packets": 7, "packet_interval": 200, "size": 24 } ) @mock.patch(CONF, Configuration.DEFAULT) def test_get_measurement_kwargs_traceroute(self): spec = Configuration.DEFAULT["specification"]["types"]["traceroute"] cmd = TracerouteMeasureCommand() cmd.init_args([ "traceroute", "--target", "ripe.net" ]) self.assertEqual( cmd._get_measurement_kwargs(), { "af": Configuration.DEFAULT["specification"]["af"], "description": "Traceroute measurement to ripe.net", "target": "ripe.net", "packets": spec["packets"], "size": spec["size"], "destination_option_size": spec["destination-option-size"], "hop_by_hop_option_size": spec["hop-by-hop-option-size"], "dont_fragment": spec["dont-fragment"], "first_hop": spec["first-hop"], "max_hops": spec["max-hops"], "paris": spec["paris"], "port": spec["port"], "protocol": spec["protocol"], "timeout": spec["timeout"] } ) cmd = TracerouteMeasureCommand() cmd.init_args([ "traceroute", "--af", "6", "--description", "This is my description", "--target", "ripe.net", "--packets", "7", "--size", "24", "--destination-option-size", "12", "--hop-by-hop-option-size", "13", "--dont-fragment", "--first-hop", "2", "--max-hops", "5", "--paris", "8", "--port", "123", "--protocol", "TCP", "--timeout", "1500" ]) self.assertEqual( cmd._get_measurement_kwargs(), { "af": 6, "description": "This is my description", "target": "ripe.net", "packets": 7, "size": 24, "destination_option_size": 12, "hop_by_hop_option_size": 13, "dont_fragment": True, "first_hop": 2, "max_hops": 5, "paris": 8, "port": 123, "protocol": "TCP", "timeout": 1500 } ) @mock.patch(CONF, Configuration.DEFAULT) def test_get_measurement_kwargs_dns(self): spec = Configuration.DEFAULT["specification"]["types"]["dns"] cmd = DnsMeasureCommand() cmd.init_args([ "dns", "--query-argument", "ripe.net" ]) self.assertEqual( cmd._get_measurement_kwargs(), { "af": Configuration.DEFAULT["specification"]["af"], "description": "DNS measurement for ripe.net", "query_class": spec["query-class"], "query_type": spec["query-type"], "query_argument": "ripe.net", "set_cd_bit": spec["set-cd-bit"], "set_do_bit": spec["set-do-bit"], "set_rd_bit": spec["set-rd-bit"], "set_nsid_bit": spec["set-nsid-bit"], "protocol": spec["protocol"], "retry": spec["retry"], "udp_payload_size": spec["udp-payload-size"], "use_probe_resolver": True, } ) cmd = DnsMeasureCommand() cmd.init_args([ "dns", "--af", "6", "--description", "This is my description", "--target", "non-existent-dns.ripe.net", "--query-class", "CHAOS", "--query-type", "SOA", "--query-argument", "some arbitrary string", "--set-cd-bit", "--set-do-bit", "--set-rd-bit", "--set-nsid-bit", "--protocol", "TCP", "--retry", "2", "--udp-payload-size", "5" ]) self.assertEqual( cmd._get_measurement_kwargs(), { "af": 6, "description": "This is my description", "target": "non-existent-dns.ripe.net", "query_class": "CHAOS", "query_type": "SOA", "query_argument": "some arbitrary string", "set_cd_bit": True, "set_do_bit": True, "set_rd_bit": True, "set_nsid_bit": True, "protocol": "TCP", "retry": 2, "udp_payload_size": 5, "use_probe_resolver": False } ) @mock.patch(CONF, Configuration.DEFAULT) def test_get_measurement_kwargs_sslcert(self): spec = Configuration.DEFAULT["specification"]["types"]["sslcert"] cmd = SslcertMeasureCommand() cmd.init_args([ "sslcert", "--target", "ripe.net" ]) self.assertEqual( cmd._get_measurement_kwargs(), { "af": Configuration.DEFAULT["specification"]["af"], "description": "Sslcert measurement to ripe.net", "target": "ripe.net", "port": spec["port"] } ) cmd = SslcertMeasureCommand() cmd.init_args([ "sslcert", "--target", "ripe.net", "--af", "6", "--description", "This is my description", "--port", "7" ]) self.assertEqual( cmd._get_measurement_kwargs(), { "af": 6, "description": "This is my description", "target": "ripe.net", "port": 7 } ) @mock.patch(CONF, Configuration.DEFAULT) def test_get_measurement_kwargs_http(self): spec = Configuration.DEFAULT["specification"]["types"]["http"] cmd = HttpMeasureCommand() cmd.init_args([ "http", "--target", "ripe.net" ]) self.assertEqual( cmd._get_measurement_kwargs(), { "af": Configuration.DEFAULT["specification"]["af"], "description": "Http measurement to ripe.net", "target": "ripe.net", "header_bytes": spec["header-bytes"], "version": spec["version"], "method": spec["method"], "port": spec["port"], "path": spec["path"], "query_string": spec["query-string"], "user_agent": spec["user-agent"], "max_bytes_read": spec["body-bytes"], } ) cmd = HttpMeasureCommand() cmd.init_args([ "http", "--target", "ripe.net", "--af", "6", "--description", "This is my description", "--header-bytes", "100", "--version", "99", "--method", "a-method", "--port", "7", "--path", "/path/to/something", "--query-string", "x=7", "--user-agent", "This is my custom user agent", "--body-bytes", "200", "--timing-verbosity", "2" ]) self.assertEqual( cmd._get_measurement_kwargs(), { "af": 6, "description": "This is my description", "target": "ripe.net", "header_bytes": 100, "version": "99", "method": "a-method", "port": 7, "path": "/path/to/something", "query_string": "x=7", "user_agent": "This is my custom user agent", "max_bytes_read": 200, "extended_timing": True, "more_extended_timing": True } ) @mock.patch(CONF, Configuration.DEFAULT) def test_get_measurement_kwargs_ntp(self): spec = Configuration.DEFAULT["specification"]["types"]["ntp"] cmd = NtpMeasureCommand() cmd.init_args([ "ntp", "--target", "ripe.net" ]) self.assertEqual( cmd._get_measurement_kwargs(), { "af": Configuration.DEFAULT["specification"]["af"], "description": "Ntp measurement to ripe.net", "target": "ripe.net", "packets": spec["packets"], "timeout": spec["timeout"] } ) cmd = NtpMeasureCommand() cmd.init_args([ "ntp", "--target", "ripe.net", "--af", "6", "--description", "This is my description", "--packets", "6", "--timeout", "9000" ]) self.assertEqual( cmd._get_measurement_kwargs(), { "af": 6, "description": "This is my description", "target": "ripe.net", "packets": 6, "timeout": 9000 } ) @mock.patch(CONF, Configuration.DEFAULT) def test_get_source_kwargs(self): spec = Configuration.DEFAULT["specification"] for kind, klass in self.KINDS.items(): cmd = klass() args = [kind, "--target", "example.com"] if kind == "dns": args = ["dns", "--query-argument", "example.com"] cmd.init_args(list(args)) tags = spec["tags"]["ipv{}".format(cmd._get_af())] includes = tags[kind]["include"] + tags["all"]["include"] excludes = tags[kind]["exclude"] + tags["all"]["exclude"] self.assertEqual(cmd._get_source_kwargs(), { "requested": spec["source"]["requested"], "type": spec["source"]["type"], "value": spec["source"]["value"], "tags": {"include": includes, "exclude": excludes} }) cmd = klass() cmd.init_args(args + [ "--probes", "10", "--from-country", "ca" ]) self.assertEqual(cmd._get_source_kwargs(), { "requested": 10, "type": "country", "value": "CA", "tags": {"include": includes, "exclude": excludes} }) for area in ("WW", "West", "North-Central", "South-Central", "North-East", "South-East"): cmd = klass() cmd.init_args(args + [ "--probes", "10", "--from-area", area ]) self.assertEqual(cmd._get_source_kwargs(), { "requested": 10, "type": "area", "value": area, "tags": {"include": includes, "exclude": excludes} }) cmd = klass() cmd.init_args(args + [ "--probes", "10", "--from-prefix", "1.2.3.0/22" ]) self.assertEqual(cmd._get_source_kwargs(), { "requested": 10, "type": "prefix", "value": "1.2.3.0/22", "tags": {"include": includes, "exclude": excludes} }) cmd = klass() cmd.init_args(args + [ "--probes", "10", "--from-asn", "3333" ]) self.assertEqual(cmd._get_source_kwargs(), { "requested": 10, "type": "asn", "value": 3333, "tags": {"include": includes, "exclude": excludes} }) # Try 20 permutations of probe lists for __ in range(0, 20): requested = randint(1, 500) selection = [str(randint(1, 5000)) for _ in range(0, 100)] cmd = klass() cmd.init_args(args + [ "--probes", str(requested), "--from-probes", ",".join(selection) ]) self.assertEqual(cmd._get_source_kwargs(), { "requested": requested, "type": "probes", "value": ",".join(selection), "tags": {"include": includes, "exclude": excludes} }) cmd = klass() cmd.init_args(args + [ "--probes", "10", "--from-measurement", "1001" ]) self.assertEqual(cmd._get_source_kwargs(), { "requested": 10, "type": "msm", "value": 1001, "tags": {"include": includes, "exclude": excludes} }) cmd = klass() cmd.init_args(args + [ "--include-tag", "tag-to-include", "--exclude-tag", "tag-to-exclude" ]) self.assertEqual(cmd._get_source_kwargs(), { "requested": spec["source"]["requested"], "type": spec["source"]["type"], "value": spec["source"]["value"], "tags": { "include": ["tag-to-include"], "exclude": ["tag-to-exclude"] } }) def test_get_af(self): conf = copy.deepcopy(Configuration.DEFAULT) conf["specification"]["af"] = 100 for kind, klass in self.KINDS.items(): with mock.patch(self.CONF, conf): for af in (4, 6): cmd = klass() cmd.init_args([kind, "--af", str(af)]) self.assertEqual(cmd._get_af(), af) cmd = klass() cmd.init_args([kind, "--target", "1.2.3.4"]) self.assertEqual(cmd._get_af(), 4) cmd = klass() cmd.init_args([kind, "--target", "1:2:3:4:5:6:7:8"]) self.assertEqual(cmd._get_af(), 6) cmd = klass() cmd.init_args([kind, "--target", "1.2.3.4.5"]) self.assertEqual( cmd._get_af(), conf["specification"]["af"] ) def test_handle_api_error(self): cmd = Command() message = ( "There was a problem communicating with the RIPE Atlas " "infrastructure. The message given was:\n\n {}" ) with self.assertRaises(RipeAtlasToolsException) as e: cmd._handle_api_error("This is a plain text error") self.assertEqual( str(e.exception), message.format("This is a plain text error") ) with self.assertRaises(RipeAtlasToolsException) as e: cmd._handle_api_error({"detail": "This is a formatted error"}) self.assertEqual( str(e.exception), message.format("This is a formatted error") ) def test_add_arguments(self): with capture_sys_output() as (stdout, stderr): with self.assertRaises(SystemExit): PingMeasureCommand().init_args([ "ping", "--renderer", "not-a-renderer" ]) self.assertTrue(stderr.getvalue().split("\n")[-2].startswith( "ripe-atlas measure: error: argument --renderer: invalid " "choice: 'not-a-renderer' (choose from" )) with capture_sys_output() as (stdout, stderr): with self.assertRaises(SystemExit): PingMeasureCommand().init_args(["ping", "--af", "5"]) self.assertEqual( stderr.getvalue().split("\n")[-2], "ripe-atlas measure: error: argument --af: invalid choice: 5 " "(choose from 4, 6)" ) with capture_sys_output() as (stdout, stderr): with self.assertRaises(SystemExit): PingMeasureCommand().init_args([ "ping", "--target", "not a target" ]) self.assertEqual( stderr.getvalue().split("\n")[-2], "ripe-atlas measure: error: argument --target: " "\"not a target\" does not appear to be an IP address or host " "name" ) with capture_sys_output() as (stdout, stderr): with self.assertRaises(SystemExit): PingMeasureCommand().init_args(["ping", "--from-area", "not an area"]) self.assertTrue(stderr.getvalue().split("\n")[-2].startswith( "ripe-atlas measure: error: argument --from-area: invalid " "choice:")) with capture_sys_output() as (stdout, stderr): with self.assertRaises(SystemExit): PingMeasureCommand().init_args(["ping", "--from-probes", "0,50000"]) self.assertEqual( stderr.getvalue().split("\n")[-2], "ripe-atlas measure: error: argument --from-probes: 0 " "is lower than the minimum permitted value of 1." ) for clude in ("in", "ex"): with capture_sys_output() as (stdout, stderr): for tag in ("NotATag", "tag!", "not a tag", "νοτ α ταγ"): with self.assertRaises(SystemExit): PingMeasureCommand().init_args([ "ping", "--{}clude-tag".format(clude), tag ]) self.assertEqual( stderr.getvalue().split("\n")[-2], 'ripe-atlas measure: error: argument --{}clude-tag: ' '"{}" does not appear to be valid.'.format(clude, tag) ) with capture_sys_output() as (stdout, stderr): with self.assertRaises(SystemExit): TracerouteMeasureCommand().init_args(["traceroute", "--protocol", "invalid"]) self.assertEqual( stderr.getvalue().split("\n")[-2], "ripe-atlas measure: error: argument --protocol: invalid " "choice: 'invalid' (choose from 'ICMP', 'UDP', 'TCP')" ) with capture_sys_output() as (stdout, stderr): with self.assertRaises(SystemExit): DnsMeasureCommand().init_args(["dns", "--protocol", "invalid"]) self.assertEqual( stderr.getvalue().split("\n")[-2], "ripe-atlas measure: error: argument --protocol: invalid " "choice: 'invalid' (choose from 'UDP', 'TCP')" ) with capture_sys_output() as (stdout, stderr): with self.assertRaises(SystemExit): HttpMeasureCommand().init_args( ["http", "--timing-verbosity", "3"]) self.assertEqual( stderr.getvalue().split("\n")[-2], "ripe-atlas measure: error: argument --timing-verbosity: " "invalid choice: 3 (choose from 0, 1, 2)" ) min_options = { "from-measurement": ((PingMeasureCommand,), 1), "probes": ((PingMeasureCommand,), 1), "packets": ( ( PingMeasureCommand, TracerouteMeasureCommand, NtpMeasureCommand ), 1 ), "size": ((PingMeasureCommand, TracerouteMeasureCommand), 1), "packet-interval": ((PingMeasureCommand,), 1), "timeout": ((TracerouteMeasureCommand, NtpMeasureCommand,), 1), "destination-option-size": ((TracerouteMeasureCommand,), 1), "hop-by-hop-option-size": ((TracerouteMeasureCommand,), 1), "retry": ((DnsMeasureCommand,), 1), "udp-payload-size": ((DnsMeasureCommand,), 1), } for option, (klasses, minimum) in min_options.items(): for klass in klasses: with capture_sys_output() as (stdout, stderr): test_value = minimum - 1 with self.assertRaises(SystemExit): klass().init_args([ klass.__name__.replace("MeasureCommand", "").lower(), "--{}".format(option), str(test_value) ]) self.assertEqual( stderr.getvalue().split("\n")[-2], "ripe-atlas measure: error: argument --{}: " "The integer must be greater than {}.".format( option, minimum ) ) min_max_options = { "from-asn": ( (PingMeasureCommand,), (0, 2 ** 32 - 2 + 1) ), "paris": ( (TracerouteMeasureCommand,), (-1, 65) ), "first-hop": ( (TracerouteMeasureCommand,), (0, 256) ), "max-hops": ( (TracerouteMeasureCommand,), (0, 256) ), "port": ( ( TracerouteMeasureCommand, SslcertMeasureCommand, HttpMeasureCommand ), (0, 2 ** 16 + 1) ), "header-bytes": ((HttpMeasureCommand,), (-1, 2049)), "body-bytes": ((HttpMeasureCommand,), (0, 1020049)), } for option, (klasses, extremes) in min_max_options.items(): for klass in klasses: for val in extremes: with capture_sys_output() as (stdout, stderr): with self.assertRaises(SystemExit): klass().init_args([ klass.__name__.replace("MeasureCommand", "").lower(), "--{}".format(option), str(val) ]) self.assertEqual( stderr.getvalue().split("\n")[-2], "ripe-atlas measure: error: argument --{}: The " "integer must be between {} and {}.".format( option, extremes[0] + 1, extremes[1] - 1 ) ) @mock.patch(CONF, Configuration.DEFAULT) def test_account_for_selected_probes(self): spec = Configuration.DEFAULT["specification"] cmd = PingMeasureCommand() cmd.init_args(["ping", "--target", "ripe.net"]) cmd._account_for_selected_probes(), self.assertEqual(cmd.arguments.probes, spec["source"]["requested"]) cmd = PingMeasureCommand() cmd.init_args(["ping", "--target", "ripe.net", "--probes", "7"]) cmd._account_for_selected_probes(), self.assertEqual(cmd.arguments.probes, 7) cmd = PingMeasureCommand() cmd.init_args(["ping", "--target", "ripe.net", "--from-probes", "1,2"]) cmd._account_for_selected_probes(), self.assertEqual(cmd.arguments.probes, 2) cmd = PingMeasureCommand() cmd.init_args([ "ping", "--target", "ripe.net", "--from-probes", "1,2", "--probes", "7" ]) with capture_sys_output() as (stdout, stderr): with self.assertRaises(RipeAtlasToolsException): cmd._account_for_selected_probes(), ripe-atlas-tools-1.2.2/tests/commands/measurements.py000066400000000000000000000171531264541652500227610ustar00rootroot00000000000000# Copyright (c) 2015 RIPE NCC # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . import collections import datetime import unittest # Python 3 comes with mock in unittest try: from unittest import mock except ImportError: import mock from ripe.atlas.tools.commands.measurements import Command from ..base import capture_sys_output class FakeGen(object): """ A rip-off of the code used for testing probes, but a little prettier. """ Measurement = collections.namedtuple("Measurement", ( "id", "type", "status", "status_id", "meta_data", "destination_name", "description" )) def __init__(self): self.data = [ self.Measurement( id=1, type="ping", status="Ongoing", status_id=2, meta_data={"status": {"name": "Ongoing", "id": 2}}, destination_name="Name 1", description="Description 1", ), self.Measurement( id=2, type="ping", status="Ongoing", status_id=2, meta_data={"status": {"name": "Ongoing", "id": 2}}, destination_name="Name 2", description="Description 2", ), self.Measurement( id=3, type="ping", status="Ongoing", status_id=2, meta_data={"status": {"name": "Ongoing", "id": 2}}, destination_name="Name 3", description="Description 3", ), self.Measurement( id=4, type="ping", status="Ongoing", status_id=2, meta_data={"status": {"name": "Ongoing", "id": 2}}, destination_name="Name 4", description="Description 4", ), self.Measurement( id=5, type="ping", status="Ongoing", status_id=2, meta_data={"status": {"name": "Ongoing", "id": 2}}, destination_name="Name 5", description="Description 5", ), ] self.total_count = 5 def __iter__(self): return self # Python 3 compatibility def __next__(self): return self.next() def next(self): if not self.data: raise StopIteration() else: return self.data.pop(0) class TestMeasurementsCommand(unittest.TestCase): @mock.patch("ripe.atlas.tools.commands.measurements.MeasurementRequest") def test_with_empty_args(self, mock_request): mock_request.return_value = FakeGen() cmd = Command() with capture_sys_output() as (stdout, stderr): cmd.init_args([]) cmd.run() expected_content = ( "\n" "Id Type Description Status\n" "===================================================================================\n" "1 ping Description 1 Ongoing\n" "2 ping Description 2 Ongoing\n" "3 ping Description 3 Ongoing\n" "4 ping Description 4 Ongoing\n" "5 ping Description 5 Ongoing\n" "===================================================================================\n" " Showing 5 of 5 total measurements\n" "\n" ) self.assertEqual( set(stdout.getvalue().split("\n")), set(expected_content.split("\n")) ) self.assertEqual( cmd.arguments.field, ("id", "type", "description", "status")) @mock.patch("ripe.atlas.tools.commands.measurements.MeasurementRequest") def test_get_line_items(self, mock_request): mock_request.return_value = FakeGen() cmd = Command() cmd.init_args([]) cmd.run() self.assertEqual( cmd._get_line_items(FakeGen.Measurement( id=1, type="ping", status="Ongoing", status_id=2, meta_data={"status": {"name": "Ongoing", "id": 2}}, destination_name="Name 1", description="Description 1", )), [1, "ping", "Description 1", "Ongoing"] ) cmd = Command() cmd.init_args([ "--field", "id", "--field", "status" ]) self.assertEqual( cmd._get_line_items(FakeGen.Measurement( id=1, type="ping", status="Ongoing", status_id=2, meta_data={"status": {"name": "Ongoing", "id": 2}}, destination_name="Name 1", description="Description 1", )), [1, "Ongoing"] ) cmd = Command() cmd.init_args([ "--field", "url", ]) self.assertEqual( cmd._get_line_items(FakeGen.Measurement( id=1, type="ping", status="Ongoing", status_id=2, meta_data={"status": {"name": "Ongoing", "id": 2}}, destination_name="Name 1", description="Description 1", )), ["https://atlas.ripe.net/measurements/1/"] ) def test_get_filters(self): cmd = Command() cmd.init_args([ "--search", "the force is strong with this one", "--status", "ongoing", "--af", "6", "--type", "ping", "--started-before", "2015-01-01", "--started-after", "2014-01-01", "--stopped-before", "2015-01-01", "--stopped-after", "2014-01-01", ]) self.assertEqual(cmd._get_filters(), { "search": "the force is strong with this one", "status__in": (2,), "af": 6, "type": "ping", "start_time__lt": datetime.datetime(2015, 1, 1), "start_time__gt": datetime.datetime(2014, 1, 1), "stop_time__lt": datetime.datetime(2015, 1, 1), "stop_time__gt": datetime.datetime(2014, 1, 1), }) def test_get_colour_from_status(self): cmd = Command() self.assertEqual(cmd._get_colour_from_status(0), "blue") self.assertEqual(cmd._get_colour_from_status(1), "blue") self.assertEqual(cmd._get_colour_from_status(2), "green") self.assertEqual(cmd._get_colour_from_status(4), "yellow") self.assertEqual(cmd._get_colour_from_status(5), "red") self.assertEqual(cmd._get_colour_from_status(6), "red") self.assertEqual(cmd._get_colour_from_status(7), "red") self.assertEqual(cmd._get_colour_from_status("XXX"), "white") def test_fail_arguments(self): expected_failures = ( ("--status", "not a status"), ("--not-an-option",), ("--af", "5"), ("--type", "not a type"), ("--field", "not a field"), ) for failure in expected_failures: with capture_sys_output(): with self.assertRaises(SystemExit): Command().init_args(failure) ripe-atlas-tools-1.2.2/tests/commands/probes.py000066400000000000000000000507451264541652500215470ustar00rootroot00000000000000# Copyright (c) 2015 RIPE NCC # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . import mock import unittest import requests from ripe.atlas.tools.commands.probes import Command from ripe.atlas.tools.exceptions import RipeAtlasToolsException from ripe.atlas.cousteau import Probe from ripe.atlas.tools.aggregators import ValueKeyAggregator from ..base import capture_sys_output class FakeGen(object): def __init__(self,): self.probes = [ Probe(id=1, meta_data={ "country_code": "GR", "asn_v4": 3333, "prefix_v4": "193.0/22"}), Probe(id=2, meta_data={ "country_code": "DE", "asn_v4": 3333, "prefix_v4": "193.0/22"}), Probe(id=3, meta_data={ "country_code": "DE", "asn_v4": 3332, "prefix_v4": "193.0/22"}), Probe(id=4, meta_data={ "country_code": "NL", "asn_v4": 3333, "prefix_v4": "193.0/22"}), Probe(id=5, meta_data={ "country_code": "GR", "asn_v4": 3333, "prefix_v4": "193.0/22"}), ] self.total_count = 4 def __iter__(self): return self # Python 3 compatibility def __next__(self): return self.next() def next(self): if not self.probes: raise StopIteration() else: return self.probes.pop(0) class TestProbesCommand(unittest.TestCase): def setUp(self): self.maxDiff = None def test_with_empty_args(self): """User passes no args, should fail with RipeAtlasToolsException""" with self.assertRaises(RipeAtlasToolsException): cmd = Command() cmd.init_args([]) cmd.run() def test_with_random_args(self): """User passes random args, should fail with SystemExit""" with capture_sys_output(): with self.assertRaises(SystemExit): cmd = Command() cmd.init_args(["blaaaaaaa"]) cmd.run() def test_arg_with_no_value(self): """User passed not boolean arg but no value""" with capture_sys_output(): with self.assertRaises(SystemExit): cmd = Command() cmd.init_args(["--asn"]) cmd.run() def test_arg_with_wrong_type(self): """User passed arg with wrong type. e.g string for asn""" with capture_sys_output(): with self.assertRaises(SystemExit): cmd = Command() cmd.init_args(["--asn", "blaaaaa"]) cmd.run() with self.assertRaises(SystemExit): cmd = Command() cmd.init_args(["--asnv4", "blaaaaa"]) cmd.run() with self.assertRaises(SystemExit): cmd = Command() cmd.init_args(["--asnv6", "blaaaaa"]) cmd.run() with self.assertRaises(SystemExit): cmd = Command() cmd.init_args(["--limit", "blaaaaa"]) cmd.run() with self.assertRaises(SystemExit): cmd = Command() cmd.init_args(["--radius", "blaaaaa"]) cmd.run() def test_location_google_breaks(self): """User passed location arg but google api gave error""" caught_exceptions = [ requests.ConnectionError, requests.HTTPError, requests.Timeout] with mock.patch('requests.get') as mock_get: for exception in caught_exceptions: mock_get.side_effect = exception with capture_sys_output(): with self.assertRaises(RipeAtlasToolsException): cmd = Command() cmd.init_args(["--location", "blaaaa"]) cmd.run() mock_get.side_effect = Exception() with self.assertRaises(Exception): cmd = Command() cmd.init_args(["--location", "blaaaa"]) cmd.run() def test_location_google_wrong_output(self): """User passed location arg but google api gave not expected format""" with mock.patch('requests.get') as mock_get: mock_get.return_value = requests.Response() with mock.patch('requests.Response.json') as mock_json: mock_json.return_value = {"blaaa": "bla"} with self.assertRaises(RipeAtlasToolsException): cmd = Command() cmd.init_args(["--location", "blaaaa"]) cmd.run() def test_location_arg(self): """User passed location arg""" with mock.patch('requests.get') as mock_get: mock_get.return_value = requests.Response() with mock.patch('requests.Response.json') as mock_json: mock_json.return_value = {"results": [ {"geometry": {"location": {"lat": 1, "lng": 2}}}]} cmd = Command() cmd.init_args(["--location", "blaaaa"]) self.assertEquals(cmd.build_request_args(), { "latitude": '1', "longitude": '2'}) def test_location_arg_with_radius(self): """User passed location arg""" with mock.patch('requests.get') as mock_get: mock_get.return_value = requests.Response() with mock.patch('requests.Response.json') as mock_json: mock_json.return_value = {"results": [ {"geometry": {"location": {"lat": 1, "lng": 2}}} ]} cmd = Command() cmd.init_args(["--location", "blaaaa", "--radius", "4"]) self.assertEquals( cmd.build_request_args(), {"radius": "1,2:4"} ) def test_asn_args(self): """User passed asn arg together with asnv4 or asnv6""" with self.assertRaises(RipeAtlasToolsException): cmd = Command() cmd.init_args(["--asn", "3333", "--asnv4", "3333"]) cmd.run() with self.assertRaises(RipeAtlasToolsException): cmd = Command() cmd.init_args(["--asn", "3333", "--asnv6", "3333"]) cmd.run() def test_prefix_args(self): """User passed prefix arg together with prefixv4 or prefixv6""" with self.assertRaises(RipeAtlasToolsException): cmd = Command() cmd.init_args([ "--prefix", "193.0.0.0/21", "--prefixv4", "193.0.0.0/21" ]) cmd.run() with self.assertRaises(RipeAtlasToolsException): cmd = Command() cmd.init_args([ "--prefix", "2001:67c:2e8::/48", "--prefixv6", "2001:67c:2e8::/48" ]) cmd.run() def test_all_args(self): """User passed all arguments""" cmd = Command() cmd.init_args(["--all"]) self.assertEquals(cmd.build_request_args(), {}) def test_center_arg_wrong_value(self): """User passed center arg with wrong value""" with self.assertRaises(RipeAtlasToolsException): cmd = Command() cmd.init_args(["--center", "blaaaa"]) cmd.run() def test_center_arg(self): """User passed center arg""" cmd = Command() cmd.init_args(["--center", "1,2"]) self.assertEquals( cmd.build_request_args(), {"latitude": "1", "longitude": "2"} ) def test_center_arg_with_radius(self): """User passed center and radius arg""" cmd = Command() cmd.init_args(["--center", "1,2", "--radius", "4"]) self.assertEquals(cmd.build_request_args(), {"radius": "1,2:4"}) def test_country_arg(self): """User passed country code arg""" cmd = Command() cmd.init_args(["--country", "GR"]) self.assertEquals(cmd.build_request_args(), {"country_code": "GR"}) def test_country_arg_with_radius(self): """User passed country code arg together with radius""" cmd = Command() cmd.init_args(["--country", "GR", "--radius", "4"]) self.assertEquals(cmd.build_request_args(), {"country_code": "GR"}) def test_sane_args1(self): """User passed several arguments (1)""" cmd = Command() cmd.init_args([ "--center", "1,2", "--radius", "4", "--asnv4", "3333", "--prefix", "193.0.0.0/21" ]) self.assertEquals( cmd.build_request_args(), {'asn_v4': 3333, 'prefix': '193.0.0.0/21', 'radius': '1,2:4'} ) def test_sane_args2(self): """User passed several arguments (2)""" cmd = Command() cmd.init_args([ "--location", "Amsterdam", "--asn", "3333", "--prefixv4", "193.0.0.0/21" ]) path = 'ripe.atlas.tools.commands.probes.Command.location2degrees' with mock.patch(path) as mock_get: mock_get.return_value = (1, 2) self.assertEquals(cmd.build_request_args(), { 'asn': 3333, 'prefix_v4': '193.0.0.0/21', 'latitude': 1, 'longitude': 2 }) def test_sane_args3(self): """User passed several arguments (3)""" cmd = Command() cmd.init_args([ "--center", "1,2", "--asnv6", "3333", "--prefixv6", "2001:67c:2e8::/48" ]) self.assertEquals(cmd.build_request_args(), { 'asn_v6': 3333, 'prefix_v6': '2001:67c:2e8::/48', 'latitude': '1', 'longitude': '2' }) def test_render_ids_only(self): """User passed ids_only arg, testing rendiring""" cmd = Command() cmd.init_args([ "--ids-only", "--country", "GR" ]) with capture_sys_output() as (stdout, stderr): path = 'ripe.atlas.tools.commands.probes.ProbeRequest' with mock.patch(path) as mock_get: mock_get.return_value = FakeGen() cmd.run() self.assertEquals(stdout.getvalue(), "1\n2\n3\n4\n5\n") def test_render_ids_only_with_limit(self): """User passed ids_only arg together with limit, testing rendering""" cmd = Command() cmd.init_args([ "--ids-only", "--country", "GR", "--limit", "2" ]) with capture_sys_output() as (stdout, stderr): path = 'ripe.atlas.tools.commands.probes.ProbeRequest' with mock.patch(path) as mock_get: mock_get.return_value = FakeGen() cmd.run() self.assertEquals(stdout.getvalue(), "1\n2\n") def test_render_ids_only_with_aggr(self): """ User passed ids_only arg together with aggregate, testing rendering """ cmd = Command() cmd.init_args([ "--ids-only", "--country", "GR", "--aggregate-by", "country" ]) with capture_sys_output() as (stdout, stderr): path = 'ripe.atlas.tools.commands.probes.ProbeRequest' with mock.patch(path) as mock_get: mock_get.return_value = FakeGen() cmd.run() self.assertEquals(stdout.getvalue(), "1\n2\n3\n4\n5\n") def test_get_aggregators(self): """User passed --aggregate-by args""" cmd = Command() cmd.init_args([ "--aggregate-by", "asn_v4", "--aggregate-by", "country", "--aggregate-by", "prefix_v4" ]) expected_output = [ ValueKeyAggregator(key="asn_v4"), ValueKeyAggregator(key="country_code"), ValueKeyAggregator(key="prefix_v4") ] cmd.set_aggregators() for index, v in enumerate(cmd.aggregators): self.assertTrue(isinstance(v, ValueKeyAggregator)) self.assertEquals( v.aggregation_keys, expected_output[index].aggregation_keys ) def test_render_without_aggregation(self): """Tests rendering of results without aggregation""" cmd = Command() cmd.init_args([ "--country", "GR" ]) with capture_sys_output() as (stdout, stderr): path = 'ripe.atlas.tools.commands.probes.ProbeRequest' with mock.patch(path) as mock_get: mock_get.return_value = FakeGen() cmd.run() expected_output = ( "\n" "Filters:\n" " Country: GR\n" "\n" "ID Asn_v4 Asn_v6 Country Status \n" "===========================================\n" "1 3333 gr None \n" "2 3333 de None \n" "3 3332 de None \n" "4 3333 nl None \n" "5 3333 gr None \n" "===========================================\n" " Showing 4 of 4 total probes\n" "\n" ) self.assertEquals(stdout.getvalue(), expected_output) def test_render_without_aggregation_with_limit(self): """Tests rendering of results without aggregation but with limit""" cmd = Command() cmd.init_args([ "--country", "GR", "--limit", "2" ]) with capture_sys_output() as (stdout, stderr): path = 'ripe.atlas.tools.commands.probes.ProbeRequest' with mock.patch(path) as mock_get: mock_get.return_value = FakeGen() cmd.run() expected_output = ( "\n" "Filters:\n" " Country: GR\n" "\n" "ID Asn_v4 Asn_v6 Country Status \n" "===========================================\n" "1 3333 gr None \n" "2 3333 de None \n" "===========================================\n" " Showing 2 of 4 total probes\n" "\n" ) self.assertEquals(stdout.getvalue(), expected_output) def test_render_with_aggregation(self): """Tests rendering of results with aggregation""" cmd = Command() cmd.init_args([ "--country", "GR", "--aggregate-by", "country", "--aggregate-by", "asn_v4", "--aggregate-by", "prefix_v4" ]) with capture_sys_output() as (stdout, stderr): path = 'ripe.atlas.tools.commands.probes.ProbeRequest' with mock.patch(path) as mock_get: mock_get.return_value = FakeGen() cmd.run() expected_blob = ( "\n" "Filters:\n" " Country: GR\n" "\n" " ID Asn_v4 Asn_v6 Country Status \n" "==============================================\n" "Country: DE\n" " ASN_V4: 3332\n" " PREFIX_V4: 193.0/22\n" " 3 3332 de None \n" " ASN_V4: 3333\n" " PREFIX_V4: 193.0/22\n" " 2 3333 de None \n" "\n" "Country: GR\n" " ASN_V4: 3333\n" " PREFIX_V4: 193.0/22\n" " 1 3333 gr None \n" " 5 3333 gr None \n" "\n" "Country: NL\n" " ASN_V4: 3333\n" " PREFIX_V4: 193.0/22\n" " 4 3333 nl None \n" "==============================================\n" " Showing 4 of 4 total probes\n" "\n" ) expected_set = set(expected_blob.split("\n")) returned_set = set(stdout.getvalue().split("\n")) self.assertEquals(returned_set, expected_set) def test_render_with_aggregation_with_limit(self): """Tests rendering of results with aggregation with limit""" cmd = Command() cmd.init_args([ "--country", "GR", "--aggregate-by", "country", "--aggregate-by", "asn_v4", "--aggregate-by", "prefix_v4", "--limit", "1" ]) with capture_sys_output() as (stdout, stderr): path = 'ripe.atlas.tools.commands.probes.ProbeRequest' with mock.patch(path) as mock_get: mock_get.return_value = FakeGen() cmd.run() expected_output = ( "\n" "Filters:\n" " Country: GR\n" "\n" " ID Asn_v4 Asn_v6 Country Status \n" "==============================================\n" "Country: GR\n" " ASN_V4: 3333\n" " PREFIX_V4: 193.0/22\n" " 1 3333 gr None \n" "==============================================\n" " Showing 1 of 4 total probes\n" "\n" ) expected_set = set(expected_output.split("\n")) returned_set = set(stdout.getvalue().split("\n")) self.assertEquals(returned_set, expected_set) def test_render_with_aggregation_with_max_per_aggr(self): """ Tests rendering of results with aggregation with max per aggr option """ cmd = Command() cmd.init_args([ "--country", "GR", "--aggregate-by", "country", "--aggregate-by", "asn_v4", "--aggregate-by", "prefix_v4", "--max-per-aggregation", "1" ]) with capture_sys_output() as (stdout, stderr): path = 'ripe.atlas.tools.commands.probes.ProbeRequest' with mock.patch(path) as mock_get: mock_get.return_value = FakeGen() cmd.run() expected_output = ( "\n" "Filters:\n " "Country: GR\n" "\n" " ID Asn_v4 Asn_v6 Country Status \n" "==============================================\n" "Country: DE\n" " ASN_V4: 3332\n" " PREFIX_V4: 193.0/22\n" " 3 3332 de None \n" " ASN_V4: 3333\n" " PREFIX_V4: 193.0/22\n" " 2 3333 de None \n" "\n" "Country: GR\n" " ASN_V4: 3333\n" " PREFIX_V4: 193.0/22\n" " 1 3333 gr None \n" "\n" "Country: NL\n" " ASN_V4: 3333\n" " PREFIX_V4: 193.0/22\n" " 4 3333 nl None \n" "==============================================\n" " Showing 4 of 4 total probes\n" "\n" ) expected_set = set(expected_output.split("\n")) returned_set = set(stdout.getvalue().split("\n")) self.assertEquals(returned_set, expected_set) ripe-atlas-tools-1.2.2/tests/commands/report.py000066400000000000000000000466501264541652500215700ustar00rootroot00000000000000# coding=utf-8 # Copyright (c) 2015 RIPE NCC # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . import mock import unittest from ripe.atlas.cousteau import Probe from ripe.atlas.tools.commands.report import Command from ripe.atlas.tools.exceptions import RipeAtlasToolsException from ripe.atlas.tools.renderers import Renderer from ripe.atlas.tools.version import __version__ from ..base import capture_sys_output class TestReportCommand(unittest.TestCase): mocked_results = [ {'af': 4, 'prb_id': 1216, 'result': [{'rtt': 27.429}, {'rtt': 25.672}, {'rtt': 25.681}], 'ttl': 54, 'avg': 26.2606666667, 'size': 20, 'from': '109.190.83.40', ' proto': 'ICMP', 'timestamp': 1445025400, 'dup': 0, 'type': 'ping', 'sent': 3, 'msm_id': 1000192, 'fw': 4700, 'max': 27.429, 'step': 360, 'src_addr': '192.168.103.132', 'rcvd': 3, 'msm_name': 'Ping', 'lts': 377, 'dst_name': 'hsi.cablecom.ch', 'min': 25.672, 'dst_addr': '62.2.16.24'}, {'af': 4, 'prb_id': 165, 'result': [{'rtt': 87.824658}, {'rtt': 87.611154}, {'rtt': 90.99957}], 'ttl': 48, 'avg': 88.811794, 'size': 20, 'from': '194.85.27.7', 'proto': 'ICMP', 'timestamp': 1445025590, 'dup': 0, 'type': 'ping', 'sent': 3, 'msm_id': 1000192, 'fw': 4720, 'max': 90.99957, 'step': 360, 'src_addr': '192.168.3.8', 'rcvd': 3, 'msm_name': 'Ping', ' lts': 87, 'dst_name': 'hsi.cablecom.ch', 'min': 87.611154, 'dst_addr': '62.2.16.24'}, {'af': 4, 'prb_id': 202, 'result': [{'rtt': 40.02356}, {'rtt': 40.399112}, {'rtt': 39.29012}], 'ttl': 52, 'avg': 39.904264, 'size': 20, 'from': '178.190.51.206', 'proto': 'ICMP', 'timestamp': 1445015502, 'dup': 0, 'type': 'ping', 'sent': 3, 'msm_id': 1000192, 'fw': 4720, 'max': 40.399112, 'step': 360, 'src_addr': '10.0.0.2', 'rcvd': 3, 'msm_name': 'Ping', 'lts': 502, 'dst_name': 'hsi.cablecom.ch', 'min': 39.29012, 'dst_addr': '62.2.16.24'}, {'af': 4, 'prb_id': 2225, 'result': [{'rtt': 10.858}, {'rtt': 12.632}, {'rtt': 20.53}, {'dup': 1, 'rtt': 32.775}, {'dup': 1, 'rtt': 47.509}, {'dup': 1, 'rtt': 62.745}, {'dup': 1, 'rtt': 78.54}, {'dup': 1, 'rtt': 93.272}, {'dup': 1, 'rtt': 109.738}], 'ttl': 56, 'avg': 14.6733333333, 'size': 20, 'from': '46.126.90.165', 'proto': 'ICMP', 'timestamp': 1445025616, 'dup': 6, 'type': 'ping', 'sent': 3, 'msm_id': 1000192, 'fw': 4700, 'max': 20.53, 'step': 360, 'src_addr': '192.168.111.103', 'rcvd': 3, 'msm_name': 'Ping', 'lts': 309, 'dst_name': 'hsi.cablecom.ch', 'min': 10.858, 'dst_addr': '62.2.16.24'}, {'af': 4, 'prb_id': 270, 'result': [{'rtt': 28.527366}, {'rtt': 26.585862}, {'rtt': 26.393094}], 'ttl': 51, 'avg': 27.168774, 'size': 20, 'from': '188.192.110.111', 'proto': 'ICMP', 'timestamp': 1445025513, 'dup': 0, 'type': 'ping', 'sent': 3, 'msm_id': 1000192, 'fw': 4720, 'max': 28.527366, 'step': 360, 'src_addr': '192.168.178.21', 'rcvd': 3, 'msm_name': 'Ping', 'lts': 182, 'dst_name': 'hsi.cablecom.ch', 'min': 26.393094, 'group_id': 1000192, 'dst_addr': '62.2.16.24'}, {'af': 4, 'prb_id': 579, 'result': [{'rtt': 23.201285}, {'rtt': 22.980868}, {'rtt': 22.863364}], 'ttl': 51, 'avg': 23.0151723333, 'size': 20, 'from': '195.88.195.170', 'proto': 'ICMP', 'timestamp': 1445025521, 'dup': 0, 'type': 'ping', 'sent': 3, 'msm_id': 1000192, 'fw': 4720, 'max': 23.201285, 'step': 360, 'src_addr': '10.69.8.150', 'rcvd': 3, 'msm_name': 'Ping', 'lts': 56, 'dst_name': 'hsi.cablecom.ch', 'min': 22.863364, 'group_id': 1000192, 'dst_addr': '62.2.16.24'}, {'af': 4, 'prb_id': 677, 'result': [{'rtt': 40.71476}, {'rtt': 40.258568}, {'rtt': 40.316936}], 'ttl': 54, 'avg': 40.430088, 'size': 20, 'from': '78.128.9.202', 'proto': 'ICMP', 'timestamp': 1445025298, 'dup': 0, 'type': 'ping', 'sent': 3, 'msm_id': 1000192, 'fw': 4720, 'max': 40.71476, 'step': 360, 'src_addr': '10.100.0.25', 'rcvd': 3, 'msm_name': 'Ping', 'lts': 97, 'dst_name': 'hsi.cablecom.ch', 'min': 40.258568, 'group_id': 1000192, 'dst_addr': '62.2.16.24'}, {'af': 4, 'prb_id': 879, 'result': [{'rtt': 34.319623}, {'rtt': 34.445575}, {'rtt': 34.376455}], 'ttl': 53, 'avg': 34.380551, 'size': 20, 'from': '94.254.125.2', 'proto': 'ICMP', 'timestamp': 1445025223, 'dup': 0, 'type': 'ping', 'sent': 3, 'msm_id': 1000192, 'fw': 4720, 'max': 34.445575, 'step': 360, 'src_addr': '192.168.8.130', 'rcvd': 3, 'msm_name': 'Ping', 'lts': 189, 'dst_name': 'hsi.cablecom.ch', 'min': 34.319623, 'dst_addr': '62.2.16.24'}, {'af': 4, 'prb_id': 945, 'result': [{'rtt': 61.665036}, {'rtt': 23.833349}, {'rtt': 23.268868}], 'ttl': 56, 'avg': 36.255751, 'size': 20, 'from': '92.111.237.94', 'proto': 'ICMP', 'timestamp': 1445025494, 'dup': 0, 'type': 'ping', 'sent': 3, 'msm_id': 1000192, 'fw': 4720, 'max': 61.665036, 'step': 360, 'src_addr': '92.111.237.94', 'rcvd': 3, 'msm_name': 'Ping', 'lts': 746, 'dst_name': 'hsi.cablecom.ch', 'min': 23.268868, 'dst_addr': '62.2.16.24'} ] def setUp(self): self.cmd = Command() def test_with_empty_args(self): """User passes no args, should fail with SystemExit""" with capture_sys_output(): with self.assertRaises(SystemExit): self.cmd.init_args([]) self.cmd.run() def test_with_random_args(self): """User passes random args, should fail with SystemExit""" with capture_sys_output(): with self.assertRaises(SystemExit): self.cmd.init_args(["blaaaaaaa"]) self.cmd.run() def test_arg_with_no_value(self): """User passed not boolean arg but no value""" with capture_sys_output(): with self.assertRaises(SystemExit): self.cmd.init_args(["--probes"]) self.cmd.run() def test_arg_with_wrong_type(self): """User passed arg with wrong type.""" with capture_sys_output(): with self.assertRaises(SystemExit): self.cmd.init_args(["--probes", "blaaaaa"]) self.cmd.run() def test_arg_renderer_with_wrong_choice(self): """User passed arg renderer with unavailable type.""" with capture_sys_output(): with self.assertRaises(SystemExit): self.cmd.init_args(["--renderer", "blaaaaa"]) self.cmd.run() def test_arg_renderer_with_valid_choice(self): """User passed arg renderer with valid type.""" # Mock AtlasRequest to fail run fast and test args validity. path = 'ripe.atlas.cousteau.AtlasRequest.get' with mock.patch(path) as mock_get: mock_get.return_value = False, {} for choice in Renderer.get_available(): with self.assertRaises(RipeAtlasToolsException): cmd = Command() cmd.init_args(["--renderer", choice, "1"]) cmd.run() def test_arg_aggregate_with_valid_choice(self): """User passed arg aggregate with valid type.""" # Mock AtlasRequest to fail run fast and test args validity. path = 'ripe.atlas.cousteau.AtlasRequest.get' with mock.patch(path) as mock_get: mock_get.return_value = False, {} for choice in Command.AGGREGATORS.keys(): with self.assertRaises(RipeAtlasToolsException): cmd = Command() cmd.init_args(["--aggregate-by", choice, "1"]) cmd.run() def test_arg_aggregate_with_wrong_choice(self): """User passed arg aggregate with unavailable type.""" with capture_sys_output(): with self.assertRaises(SystemExit): self.cmd.init_args(["--aggregate-by", "blaaaaa"]) self.cmd.run() def test_arg_no_msm_id(self): """User passed no measurement id.""" with capture_sys_output(): with self.assertRaises(SystemExit): self.cmd.init_args(["--aggregate-by", "country"]) self.cmd.run() def test_arg_no_valid_msm_id(self): """User passed non valid type of measurement id.""" with capture_sys_output(): with self.assertRaises(SystemExit): self.cmd.init_args(["blaaa"]) self.cmd.run() def test_measurement_failure(self): """Testcase where given measurement id doesn't exist.""" path = 'ripe.atlas.cousteau.AtlasRequest.get' with mock.patch(path) as mock_get: mock_get.return_value = False, {} with self.assertRaises(RipeAtlasToolsException): self.cmd.init_args(["--aggregate-by", "country", "1"]) self.cmd.run() def test_no_results(self): """Testcase where given measurement id doesn't have any results.""" path = 'ripe.atlas.cousteau.AtlasRequest.get' with mock.patch(path) as mock_get: mock_get.side_effect = [ (True, {"creation_time": 1, "start_time": 1, "type": {"name": "ping"}}), (True, {}) ] with self.assertRaises(RipeAtlasToolsException): self.cmd.init_args(["--aggregate-by", "country", "1"]) self.cmd.run() def test_no_renderer_found(self): """Testcase where renderer canoot be founbd from measurement type.""" path = 'ripe.atlas.cousteau.AtlasRequest.get' with mock.patch(path) as mock_get: mock_get.side_effect = [ (True, {"creation_time": 1, "start_time": 1, "type": {"name": "shit"}}), (True, {}) ] with self.assertRaises(RipeAtlasToolsException): self.cmd.init_args(["--aggregate-by", "country", "1"]) self.cmd.run() def test_valid_case_no_aggr(self): """Test case where we have result no aggregation.""" expected_output = ( "20 bytes from probe #1216 109.190.83.40 to hsi.cablecom.ch (62.2.16.24): ttl=54 times:27.429, 25.672, 25.681, \n" "20 bytes from probe #165 194.85.27.7 to hsi.cablecom.ch (62.2.16.24): ttl=48 times:87.825, 87.611, 91.0, \n" "20 bytes from probe #202 178.190.51.206 to hsi.cablecom.ch (62.2.16.24): ttl=52 times:40.024, 40.399, 39.29, \n" "20 bytes from probe #2225 46.126.90.165 to hsi.cablecom.ch (62.2.16.24): ttl=56 times:10.858, 12.632, 20.53, 32.775, 47.509, 62.745, 78.54, 93.272, 109.738,\n" "20 bytes from probe #270 188.192.110.111 to hsi.cablecom.ch (62.2.16.24): ttl=51 times:28.527, 26.586, 26.393, \n" "20 bytes from probe #579 195.88.195.170 to hsi.cablecom.ch (62.2.16.24): ttl=51 times:23.201, 22.981, 22.863, \n" "20 bytes from probe #677 78.128.9.202 to hsi.cablecom.ch (62.2.16.24): ttl=54 times:40.715, 40.259, 40.317, \n" "20 bytes from probe #879 94.254.125.2 to hsi.cablecom.ch (62.2.16.24): ttl=53 times:34.32, 34.446, 34.376, \n" "20 bytes from probe #945 92.111.237.94 to hsi.cablecom.ch (62.2.16.24): ttl=56 times:61.665, 23.833, 23.269, \n" ) probes = [ Probe(id=202, meta_data={ "country_code": "GR", "asn_v4": 3333, "asn_v6": "4444"}), Probe(id=677, meta_data={ "country_code": "DE", "asn_v4": 3333, "asn_v6": "4444"}), Probe(id=2225, meta_data={ "country_code": "DE", "asn_v4": 3332, "asn_v6": "4444"}), Probe(id=165, meta_data={ "country_code": "NL", "asn_v4": 3333, "asn_v6": "4444"}), Probe(id=1216, meta_data={ "country_code": "GR", "asn_v4": 3333, "asn_v6": "4444"}), Probe(id=270, meta_data={ "country_code": "GR", "asn_v4": 3333, "asn_v6": "4444"}), Probe(id=579, meta_data={ "country_code": "GR", "asn_v4": 3333, "asn_v6": "4444"}), Probe(id=945, meta_data={ "country_code": "GR", "asn_v4": 3333, "asn_v6": "4444"}), Probe(id=879, meta_data={ "country_code": "GR", "asn_v4": 3333, "asn_v6": "4444"}), ] with capture_sys_output() as (stdout, stderr): path = 'ripe.atlas.cousteau.AtlasRequest.get' with mock.patch(path) as mock_get: mock_get.side_effect = [ (True, {"creation_time": 1, "start_time": 1, "type": {"name": "ping"}, "description": ""}), (True, self.mocked_results) ] mpath = 'ripe.atlas.tools.helpers.rendering.Probe.get_many' with mock.patch(mpath) as mock_get_many: mock_get_many.return_value = probes self.cmd.init_args(["1"]) self.cmd.run() self.assertEquals(stdout.getvalue(), expected_output) def test_valid_case_with_aggr(self): """Test case where we have result with aggregation.""" expected_output = ( "RTT_MEDIAN: 40-50\n" " 20 bytes from probe #202 178.190.51.206 to hsi.cablecom.ch (62.2.16.24): ttl=52 times:40.024, 40.399, 39.29, \n" " 20 bytes from probe #677 78.128.9.202 to hsi.cablecom.ch (62.2.16.24): ttl=54 times:40.715, 40.259, 40.317, \n" "RTT_MEDIAN: 10-20\n" " 20 bytes from probe #2225 46.126.90.165 to hsi.cablecom.ch (62.2.16.24): ttl=56 times:10.858, 12.632, 20.53, 32.775, 47.509, 62.745, 78.54, 93.272, 109.738,\n" "RTT_MEDIAN: 50-100\n" " 20 bytes from probe #165 194.85.27.7 to hsi.cablecom.ch (62.2.16.24): ttl=48 times:87.825, 87.611, 91.0, \n" "RTT_MEDIAN: 20-30\n" " 20 bytes from probe #1216 109.190.83.40 to hsi.cablecom.ch (62.2.16.24): ttl=54 times:27.429, 25.672, 25.681, \n" " 20 bytes from probe #270 188.192.110.111 to hsi.cablecom.ch (62.2.16.24): ttl=51 times:28.527, 26.586, 26.393, \n" " 20 bytes from probe #579 195.88.195.170 to hsi.cablecom.ch (62.2.16.24): ttl=51 times:23.201, 22.981, 22.863, \n" " 20 bytes from probe #945 92.111.237.94 to hsi.cablecom.ch (62.2.16.24): ttl=56 times:61.665, 23.833, 23.269, \n" "RTT_MEDIAN: 30-40\n" " 20 bytes from probe #879 94.254.125.2 to hsi.cablecom.ch (62.2.16.24): ttl=53 times:34.32, 34.446, 34.376, \n" ) probes = [ Probe(id=202, meta_data={ "country_code": "GR", "asn_v4": 3333, "asn_v6": "4444"}), Probe(id=677, meta_data={ "country_code": "DE", "asn_v4": 3333, "asn_v6": "4444"}), Probe(id=2225, meta_data={ "country_code": "DE", "asn_v4": 3332, "asn_v6": "4444"}), Probe(id=165, meta_data={ "country_code": "NL", "asn_v4": 3333, "asn_v6": "4444"}), Probe(id=1216, meta_data={ "country_code": "GR", "asn_v4": 3333, "asn_v6": "4444"}), Probe(id=270, meta_data={ "country_code": "GR", "asn_v4": 3333, "asn_v6": "4444"}), Probe(id=579, meta_data={ "country_code": "GR", "asn_v4": 3333, "asn_v6": "4444"}), Probe(id=945, meta_data={ "country_code": "GR", "asn_v4": 3333, "asn_v6": "4444"}), Probe(id=879, meta_data={ "country_code": "GR", "asn_v4": 3333, "asn_v6": "4444"}), ] with capture_sys_output() as (stdout, stderr): path = 'ripe.atlas.cousteau.AtlasRequest.get' with mock.patch(path) as mock_get: mock_get.side_effect = [ (True, {"creation_time": 1, "start_time": 1, "type": {"name": "ping"}, "description": ""}), (True, self.mocked_results) ] mpath = 'ripe.atlas.tools.helpers.rendering.Probe.get_many' with mock.patch(mpath) as mock_get_many: mock_get_many.return_value = probes self.cmd.init_args(["--aggregate-by", "rtt-median", "1"]) self.cmd.run() expected_set = set(expected_output.split("\n")) returned_set = set(stdout.getvalue().split("\n")) self.assertEquals(returned_set, expected_set) def test_asns_filter(self): """Test case where user specified probe asns filters..""" expected_output = ( "20 bytes from probe #165 194.85.27.7 to hsi.cablecom.ch (62.2.16.24): ttl=48 times:87.825, 87.611, 91.0, \n" "20 bytes from probe #945 92.111.237.94 to hsi.cablecom.ch (62.2.16.24): ttl=56 times:61.665, 23.833, 23.269, \n" ) probes = [ Probe(id=202, meta_data={ "country_code": "GR", "asn_v4": 3337, "asn_v6": "4445"}), Probe(id=677, meta_data={ "country_code": "DE", "asn_v4": 3333, "asn_v6": "4444"}), Probe(id=2225, meta_data={ "country_code": "DE", "asn_v4": 3332, "asn_v6": "4444"}), Probe(id=165, meta_data={ "country_code": "NL", "asn_v4": 3334, "asn_v6": "4444"}), Probe(id=1216, meta_data={ "country_code": "GR", "asn_v4": 3335, "asn_v6": "4444"}), Probe(id=270, meta_data={ "country_code": "GR", "asn_v4": 3340, "asn_v6": "4444"}), Probe(id=579, meta_data={ "country_code": "GR", "asn_v4": 3336, "asn_v6": "4444"}), Probe(id=945, meta_data={ "country_code": "GR", "asn_v4": 3334, "asn_v6": "4444"}), Probe(id=879, meta_data={ "country_code": "GR", "asn_v4": 3333, "asn_v6": "4444"}), ] with capture_sys_output() as (stdout, stderr): path = 'ripe.atlas.cousteau.AtlasRequest.get' with mock.patch(path) as mock_get: mock_get.side_effect = [ (True, {"creation_time": 1, "start_time": 1, "type": {"name": "ping"}, "description": ""}), (True, self.mocked_results) ] mpath = 'ripe.atlas.tools.helpers.rendering.Probe.get_many' with mock.patch(mpath) as mock_get_many: mock_get_many.return_value = probes self.cmd.init_args(["1", "--probe-asns", "3334"]) self.cmd.run() self.assertEquals(stdout.getvalue(), expected_output) def test_user_agent(self): standard = "RIPE Atlas Tools (Magellan) {}".format(__version__) tests = { standard: standard, "Some custom agent": "Some custom agent", "Some custom agent\nwith a second line": "Some custom agent", "x" * 3000: "x" * 128, "Πράκτορας χρήστη": "Πράκτορας χρήστη", "이것은 테스트 요원": "이것은 테스트 요원", } self.assertEqual(self.cmd.user_agent, standard) for in_string, out_string in tests.items(): path = "ripe.atlas.tools.commands.base.open" content = mock.mock_open(read_data=in_string) with mock.patch(path, content): self.assertEqual(Command().user_agent, out_string) ripe-atlas-tools-1.2.2/tests/docs.py000066400000000000000000000026111264541652500173710ustar00rootroot00000000000000# Copyright (c) 2015 RIPE NCC # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . import unittest from sphinx.application import Sphinx class DocTest(unittest.TestCase): SOURCE_DIR = "docs" CONFIG_DIR = "docs" OUTPUT_DIR = "docs/build" DOCTREE_DIR = "docs/build/doctrees" def test_html_documentation(self): Sphinx( self.SOURCE_DIR, self.CONFIG_DIR, self.OUTPUT_DIR, self.DOCTREE_DIR, buildername="html", warningiserror=True, ).build(force_all=True) def test_text_documentation(self): Sphinx( self.SOURCE_DIR, self.CONFIG_DIR, self.OUTPUT_DIR, self.DOCTREE_DIR, buildername="text", warningiserror=True, ).build(force_all=True) ripe-atlas-tools-1.2.2/tests/helpers/000077500000000000000000000000001264541652500175315ustar00rootroot00000000000000ripe-atlas-tools-1.2.2/tests/helpers/__init__.py000066400000000000000000000013561264541652500216470ustar00rootroot00000000000000# Copyright (c) 2015 RIPE NCC # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . from .validators import TestArgumentTypeHelper __all__ = [TestArgumentTypeHelper] ripe-atlas-tools-1.2.2/tests/helpers/sanitisers.py000066400000000000000000000026341264541652500222740ustar00rootroot00000000000000# Copyright (c) 2015 RIPE NCC # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . import unittest from ripe.atlas.tools.helpers.sanitisers import sanitise class TestSanitisersHelper(unittest.TestCase): def test_sanitise(self): self.assertEqual("clean", sanitise("clean")) for i in list(range(0, 32)) + [127]: self.assertEqual("unclean", sanitise("unclean" + chr(i))) self.assertEqual(None, sanitise(None)) self.assertEqual(7, sanitise(7)) def test_sanitise_with_newline_exception(self): self.assertEqual( "unc\nlean", sanitise("unc\nlean", strip_newlines=False)) for i in set(list(range(0, 32)) + [127]).difference({10}): self.assertEqual( "unc\nlean", sanitise("unc\nlean" + chr(i), strip_newlines=False) ) ripe-atlas-tools-1.2.2/tests/helpers/validators.py000066400000000000000000000116361264541652500222620ustar00rootroot00000000000000# Copyright (c) 2015 RIPE NCC # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . import argparse import datetime import os import sys import unittest try: from cStringIO import StringIO except ImportError: # Python 3 from io import StringIO from ripe.atlas.tools.helpers.validators import ArgumentType class TestArgumentTypeHelper(unittest.TestCase): def test_path(self): self.assertEqual("/tmp", ArgumentType.path("/tmp")) with self.assertRaises(argparse.ArgumentTypeError): ArgumentType.path("/not/a/real/place") def test_country_code(self): self.assertEqual("CA", ArgumentType.country_code("CA")) self.assertEqual("CA", ArgumentType.country_code("ca")) self.assertEqual("CA", ArgumentType.country_code("Ca")) self.assertEqual("CA", ArgumentType.country_code("cA")) for value in ("CAN", "Canada", "can", "This isn't even a country"): with self.assertRaises(argparse.ArgumentTypeError): ArgumentType.country_code(value) def test_comma_separated_integers(self): self.assertEqual( [1, 2, 3], ArgumentType.comma_separated_integers()("1,2,3")) self.assertEqual( [1, 2, 3], ArgumentType.comma_separated_integers()("1, 2, 3")) self.assertEqual([1], ArgumentType.comma_separated_integers()("1")) with self.assertRaises(argparse.ArgumentTypeError): ArgumentType.comma_separated_integers()("1,2,3,pizza!") with self.assertRaises(argparse.ArgumentTypeError): ArgumentType.comma_separated_integers(minimum=5)("4,5,6,7") with self.assertRaises(argparse.ArgumentTypeError): ArgumentType.comma_separated_integers(maximum=5)("1,2,3,4,6") def test_datetime(self): d = datetime.datetime(2015, 12, 1) self.assertEqual(d, ArgumentType.datetime("2015-12-1")) self.assertEqual(d, ArgumentType.datetime("2015-12-1T00")) self.assertEqual(d, ArgumentType.datetime("2015-12-1T00:00")) self.assertEqual(d, ArgumentType.datetime("2015-12-1T00:00:00")) self.assertEqual(d, ArgumentType.datetime("2015-12-1")) self.assertEqual(d, ArgumentType.datetime("2015-12-1 00")) self.assertEqual(d, ArgumentType.datetime("2015-12-1 00:00")) self.assertEqual(d, ArgumentType.datetime("2015-12-1 00:00:00")) with self.assertRaises(argparse.ArgumentTypeError): ArgumentType.datetime("yesterday") with self.assertRaises(argparse.ArgumentTypeError): ArgumentType.datetime("Definitely not a date, or even a time") def test_integer_range(self): self.assertEqual(1, ArgumentType.integer_range(1, 10)("1")) self.assertEqual(10, ArgumentType.integer_range(1, 10)("10")) self.assertEqual(1, ArgumentType.integer_range(-1, 1)("1")) self.assertEqual(-1, ArgumentType.integer_range(-1, 1)("-1")) for value in ("0", "11", "-1"): with self.assertRaises(argparse.ArgumentTypeError): ArgumentType.integer_range(1, 10)(value) def test_ip_or_domain(self): passable_hosts = ( "localhost", "ripe.net", "www.ripe.net", "1.2.3.4", "2001:67c:2e8:22::c100:68b" ) for host in passable_hosts: self.assertEqual(host, ArgumentType.ip_or_domain(host)) with self.assertRaises(argparse.ArgumentTypeError): ArgumentType.ip_or_domain("Definitely not a host") def test_comma_separated_integers_or_file(self): with self.assertRaises(argparse.ArgumentTypeError): ArgumentType.comma_separated_integers_or_file("/dev/null/fail") with self.assertRaises(argparse.ArgumentTypeError): ArgumentType.comma_separated_integers_or_file("not,a,number") with self.assertRaises(argparse.ArgumentTypeError): ArgumentType.comma_separated_integers_or_file("1, 2, 3") old = sys.stdin sys.stdin = StringIO("1\n2\n") self.assertEqual( ArgumentType.comma_separated_integers_or_file("-"), [1, 2] ) sys.stdin = old in_file = "/tmp/__test_file__" with open(in_file, "w") as f: f.write("1\n2\n3\n") self.assertEqual( ArgumentType.comma_separated_integers_or_file(in_file), [1, 2, 3] ) os.unlink(in_file) ripe-atlas-tools-1.2.2/tests/renderers/000077500000000000000000000000001264541652500200605ustar00rootroot00000000000000ripe-atlas-tools-1.2.2/tests/renderers/__init__.py000066400000000000000000000017111264541652500221710ustar00rootroot00000000000000# Copyright (c) 2015 RIPE NCC # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . from .ping import TestPingRenderer from .http import TestHttpRenderer from .aggregate_ping import TestAggregatePing from .ssl_consistency import TestSSLConsistency from .raw import TestRawRenderer __all__ = [ TestPingRenderer, TestHttpRenderer, TestAggregatePing, TestSSLConsistency ] ripe-atlas-tools-1.2.2/tests/renderers/aggregate_ping.py000066400000000000000000000173351264541652500234060ustar00rootroot00000000000000# Copyright (c) 2015 RIPE NCC # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . import unittest from collections import namedtuple from ripe.atlas.tools.renderers.aggregate_ping import Renderer from ripe.atlas.sagan import Result from ..base import capture_sys_output class TestAggregatePing(unittest.TestCase): @classmethod def setUpClass(cls): cls.results = [ {u'af': 4, u'prb_id': 11421, u'result': [{u'rtt': 42.342895}, {u'rtt': 42.220215}, {u'rtt': 42.40614}], u'ttl': 53, u'avg': 42.3230833333, u'size': 48, u'from': u'178.11.85.39', u'proto': u'ICMP', u'timestamp': 1446146495, u'dup': 0, u'type': u'ping', u'sent': 3, u'msm_id': 2882184, u'fw': 4720, u'max': 42.40614, u'step': 240, u'src_addr': u'192.168.2.101', u'rcvd': 3, u'msm_name': u'Ping', u'lts': 3, u'dst_name': u'194.88.241.228', u'min': 42.220215, u'group_id': 2882184, u'dst_addr': u'194.88.241.228'}, {u'af': 4, u'prb_id': 11779, u'result': [{u'rtt': 76.61127}, {u'rtt': 76.38997}, {u'rtt': 76.47354}], u'ttl': 51, u'avg': 76.4915933333, u'size': 48, u'from': u'79.106.99.242', u'proto': u'ICMP', u'timestamp': 1446146494, u'dup': 0, u'type': u'ping', u'sent': 3, u'msm_id': 2882184, u'fw': 4720, u'max': 76.61127, u'step': 240, u'src_addr': u'192.168.1.110', u'rcvd': 3, u'msm_name': u'Ping', u'lts': 127, u'dst_name': u'194.88.241.228', u'min': 76.38997, u'group_id': 2882184, u'dst_addr': u'194.88.241.228'}, {u'af': 4, u'prb_id': 17854, u'result': [{u'rtt': 154.118}, {u'rtt': 154.197}, {u'rtt': 154.845}], u'ttl': 55, u'avg': 154.3866666667, u'size': 48, u'from': u'134.202.20.3', u'proto': u'ICMP', u'timestamp': 1446146493, u'dup': 0, u'type': u'ping', u'sent': 3, u'msm_id': 2882184, u'fw': 4700, u'max': 154.845, u'step': 240, u'src_addr': u'134.202.20.3', u'rcvd': 3, u'msm_name': u'Ping', u'lts': 1, u'dst_name': u'194.88.241.228', u'min': 154.118, u'group_id': 2882184, u'dst_addr': u'194.88.241.228'}, {u'af': 4, u'prb_id': 3183, u'result': [{u'rtt': 42.263816}, {u'rtt': 42.196233}, {u'rtt': 42.342921}], u'ttl': 56, u'avg': 42.2676566667, u'size': 48, u'from': u'212.122.42.107', u'proto': u'ICMP', u'timestamp': 1446146494, u'dup': 0, u'type': u'ping', u'sent': 3, u'msm_id': 2882184, u'fw': 4720, u'max': 42.342921, u'step': 240, u'src_addr': u'192.168.189.1', u'rcvd': 3, u'msm_name': u'Ping', u'lts': 3, u'dst_name': u'194.88.241.228', u'min': 42.196233, u'group_id': 2882184, u'dst_addr': u'194.88.241.228'}, {u'af': 4, u'prb_id': 3207, u'result': [{u'rtt': 218.077484}, {u'rtt': 36.921608}, {u'rtt': 38.99444}], u'ttl': 49, u'avg': 97.997844, u'size': 48, u'from': u'134.3.245.244', u'proto': u'ICMP', u'timestamp': 1446146494, u'dup': 0, u'type': u'ping', u'sent': 3, u'msm_id': 2882184, u'fw': 4720, u'max': 218.077484, u'step': 240, u'src_addr': u'10.30.0.4', u'rcvd': 3, u'msm_name': u'Ping', u'lts': 70, u'dst_name': u'194.88.241.228', u'min': 36.921608, u'group_id': 2882184, u'dst_addr': u'194.88.241.228'} ] cls.sagans = [ Result.get( result, on_error=Result.ACTION_IGNORE, on_warning=Result.ACTION_IGNORE ) for result in cls.results ] def test_additional(self): """Tests whole functionality of additional unit.""" expected_output = ( "-- 194.88.241.228 ping statistics ---\n" "15 packets transmitted, 15 received, 0.0% loss\n" "rtt min/med/avg/max = 36.921608/42.406/82.693/218.077484 ms\n\n" ) self.assertEquals(Renderer().additional(self.sagans), expected_output) def test_collect_stats(self): """Tests collect stats function.""" renderer = Renderer() renderer.collect_stats(self.sagans) self.assertEquals( renderer.rtts, [ 42.343, 42.22, 42.406, 76.611, 76.39, 76.474, 154.118, 154.197, 154.845, 42.264, 42.196, 42.343, 218.077, 36.922, 38.994 ] ) self.assertEquals(renderer.target, "194.88.241.228") self.assertEquals(renderer.sent_packets, 15) self.assertEquals(renderer.received_packets, 15) self.assertEquals( renderer.rtts_min, [42.220215, 76.38997, 154.118, 42.196233, 36.921608] ) self.assertEquals( renderer.rtts_max, [42.40614, 76.61127, 154.845, 42.342921, 218.077484] ) def test_collect_min_max_rtts(self): """Test use cases for collecting min max rtts.""" renderer = Renderer() renderer.collect_min_max_rtts("min", 3) self.assertEquals(renderer.rtts_min, [3]) renderer.collect_min_max_rtts("min", None) self.assertEquals(renderer.rtts_min, [3, 0]) renderer.collect_min_max_rtts("max", 3) self.assertEquals(renderer.rtts_max, [3]) renderer.collect_min_max_rtts("max", None) self.assertEquals(renderer.rtts_max, [3, 0]) def test_collect_packets_rtt(self): """Test use cases for collecting rtts.""" Packet = namedtuple("Packet", "rtt") packets = [Packet(rtt=2), Packet(rtt=3.2), Packet(rtt=5.0)] renderer = Renderer() renderer.collect_packets_rtt(packets) self.assertEquals(renderer.rtts, [2, 3.2, 5.0]) packets = [Packet(rtt=None), Packet(rtt=3.2), Packet(rtt=5.0)] renderer = Renderer() renderer.collect_packets_rtt(packets) self.assertEquals(renderer.rtts, [0, 3.2, 5.0]) def test_set_target(self): """Tests setting the target.""" Sagan = namedtuple("Sagan", "destination_name") renderer = Renderer() sagan = Sagan(destination_name="1") renderer.set_target(sagan) self.assertEquals(renderer.target, "1") sagan = Sagan(destination_name="2") renderer.set_target(sagan) self.assertEquals(renderer.target, "1") def test_calculate_loss(self): """Test use cases for calculating loss.""" renderer = Renderer() renderer.sent_packets = 10 renderer.received_packets = 9 self.assertEquals(renderer.calculate_loss(), 9.999999999999998) renderer.sent_packets = 0 self.assertEquals(renderer.calculate_loss(), 0) renderer.received_packets = 0 renderer.sent_packets = 10 self.assertEquals(renderer.calculate_loss(), 100) renderer.received_packets = 5 self.assertEquals(renderer.calculate_loss(), 50) def test_mean(self): """Test use cases for calculating mean.""" renderer = Renderer() renderer.rtts = [0, 2.0, 5.0, 20] self.assertEquals(renderer.mean(), 6.75) renderer.rtts = [0, 2.0, 7.5, 5.0, 20] self.assertEquals(renderer.mean(), 6.9) renderer.rtts = [0, 2.0, 7.5, 5.0, 20, 50] self.assertEquals(renderer.mean(), 14.083) def test_median(self): """Test use cases for calculating median.""" renderer = Renderer() renderer.rtts = [0, 2.0, 5.0, 20] self.assertEquals(renderer.median(), 3.5) renderer.rtts = [0, 2.0, 7.5, 5.0, 20] self.assertEquals(renderer.median(), 5) renderer.rtts = [0, 2.0, 7.5, 5.0, 20, 50] self.assertEquals(renderer.median(), 6.25) ripe-atlas-tools-1.2.2/tests/renderers/http.py000066400000000000000000000054701264541652500214170ustar00rootroot00000000000000# Copyright (c) 2015 RIPE NCC # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . import unittest from ripe.atlas.sagan import Result from ripe.atlas.tools.renderers.http import Renderer class TestHttpRenderer(unittest.TestCase): def __init__(self, *args, **kwargs): unittest.TestCase.__init__(self, *args, **kwargs) self.basic = Result.get('{"lts":64,"from":"217.13.64.36","msm_id":2841267,"fw":4720,"timestamp":1450185727,"uri":"http://at-vie-as1120.anchors.atlas.ripe.net:80/4096","prb_id":1,"result":[{"rt":45.953289,"src_addr":"217.13.64.36","hsize":131,"af":4,"bsize":1668618,"res":200,"method":"GET","ver":"1.1","dst_addr":"193.171.255.2"}],"group_id":2841267,"type":"http","msm_name":"HTTPGet"}') self.multiple = Result.get('{"lts":64,"from":"217.13.64.36","msm_id":2841267,"fw":4720,"timestamp":1450185727,"uri":"http://at-vie-as1120.anchors.atlas.ripe.net:80/4096","prb_id":1,"result":[{"rt":45.953289,"src_addr":"217.13.64.36","hsize":131,"af":4,"bsize":1668618,"res":200,"method":"GET","ver":"1.1","dst_addr":"193.171.255.2"},{"rt":45.953289,"src_addr":"217.13.64.36","hsize":131,"af":4,"bsize":1668618,"res":200,"method":"GET","ver":"1.1","dst_addr":"193.171.255.2"}],"group_id":2841267,"type":"http","msm_name":"HTTPGet"}') def test_basic(self): expected = ( '#Version: 1.0\n' '#Date: 2015-12-15 13:22:07\n' '#Fields: cs-method cs-uri c-ip s-ip sc-status time-taken http-version header-bytes body-bytes\n' 'GET http://at-vie-as1120.anchors.atlas.ripe.net:80/4096 217.13.64.36 193.171.255.2 200 45.953289 1.1 131 1668618\n\n' ) self.assertEqual(Renderer().on_result(self.basic), expected) def test_multiple(self): expected = ( '#Version: 1.0\n' '#Date: 2015-12-15 13:22:07\n' '#Fields: cs-method cs-uri c-ip s-ip sc-status time-taken http-version header-bytes body-bytes\n' 'GET http://at-vie-as1120.anchors.atlas.ripe.net:80/4096 217.13.64.36 193.171.255.2 200 45.953289 1.1 131 1668618\n' 'GET http://at-vie-as1120.anchors.atlas.ripe.net:80/4096 217.13.64.36 193.171.255.2 200 45.953289 1.1 131 1668618\n\n' ) self.assertEqual(Renderer().on_result(self.multiple), expected) ripe-atlas-tools-1.2.2/tests/renderers/ping.py000066400000000000000000000037551264541652500214010ustar00rootroot00000000000000# Copyright (c) 2015 RIPE NCC # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . import unittest from ripe.atlas.sagan import Result from ripe.atlas.tools.renderers.ping import Renderer class TestPingRenderer(unittest.TestCase): def __init__(self, *args, **kwargs): unittest.TestCase.__init__(self, *args, **kwargs) self.basic = Result.get('{"af":4,"prb_id":1,"result":[{"rtt":10.001},{"rtt":10.002},{"rtt":10.003}],"ttl":20,"avg":10.002,"size":20,"from":"1.2.3.4","proto":"ICMP","timestamp":1440000000,"dup":0,"type":"ping","sent":3,"msm_id":1000001,"fw":4700,"max":10.003,"step":360,"src_addr":"2.3.4.5","rcvd":3,"msm_name":"Ping","lts":40,"dst_name":"my.name.ca","min":10.001,"dst_addr":"3.4.5.6"}') self.no_packets = Result.get('{"af":4,"prb_id":2,"result":[],"ttl":20,"avg":10.002,"size":20,"from":"1.2.3.4","proto":"ICMP","timestamp":1440000000,"dup":0,"type":"ping","sent":3,"msm_id":1000001,"fw":4700,"max":null,"step":360,"src_addr":"2.3.4.5","rcvd":0,"msm_name":"Ping","lts":40,"dst_name":"my.name.ca","min":null,"dst_addr":"3.4.5.6"}') def test_basic(self): self.assertEqual( Renderer().on_result(self.basic), "20 bytes from probe #1 1.2.3.4 to my.name.ca (3.4.5.6): ttl=20 times:10.001, 10.002, 10.003, \n" ) def test_no_packets(self): self.assertEqual( Renderer().on_result(self.no_packets), "No packets found\n") ripe-atlas-tools-1.2.2/tests/renderers/raw.py000066400000000000000000000163331264541652500212310ustar00rootroot00000000000000# Copyright (c) 2015 RIPE NCC # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . import mock import json import unittest from ripe.atlas.cousteau import Probe from ripe.atlas.tools.commands.report import Command from ..base import capture_sys_output class TestRawRenderer(unittest.TestCase): mocked_results = [ {'af': 4, 'prb_id': 1216, 'result': [{'rtt': 27.429}, {'rtt': 25.672}, {'rtt': 25.681}], 'ttl': 54, 'avg': 26.2606666667, 'size': 20, 'from': '109.190.83.40', ' proto': 'ICMP', 'timestamp': 1445025400, 'dup': 0, 'type': 'ping', 'sent': 3, 'msm_id': 1000192, 'fw': 4700, 'max': 27.429, 'step': 360, 'src_addr': '192.168.103.132', 'rcvd': 3, 'msm_name': 'Ping', 'lts': 377, 'dst_name': 'hsi.cablecom.ch', 'min': 25.672, 'dst_addr': '62.2.16.24'}, {'af': 4, 'prb_id': 165, 'result': [{'rtt': 87.824658}, {'rtt': 87.611154}, {'rtt': 90.99957}], 'ttl': 48, 'avg': 88.811794, 'size': 20, 'from': '194.85.27.7', 'proto': 'ICMP', 'timestamp': 1445025590, 'dup': 0, 'type': 'ping', 'sent': 3, 'msm_id': 1000192, 'fw': 4720, 'max': 90.99957, 'step': 360, 'src_addr': '192.168.3.8', 'rcvd': 3, 'msm_name': 'Ping', ' lts': 87, 'dst_name': 'hsi.cablecom.ch', 'min': 87.611154, 'dst_addr': '62.2.16.24'}, {'af': 4, 'prb_id': 202, 'result': [{'rtt': 40.02356}, {'rtt': 40.399112}, {'rtt': 39.29012}], 'ttl': 52, 'avg': 39.904264, 'size': 20, 'from': '178.190.51.206', 'proto': 'ICMP', 'timestamp': 1445015502, 'dup': 0, 'type': 'ping', 'sent': 3, 'msm_id': 1000192, 'fw': 4720, 'max': 40.399112, 'step': 360, 'src_addr': '10.0.0.2', 'rcvd': 3, 'msm_name': 'Ping', 'lts': 502, 'dst_name': 'hsi.cablecom.ch', 'min': 39.29012, 'dst_addr': '62.2.16.24'}, {'af': 4, 'prb_id': 2225, 'result': [{'rtt': 10.858}, {'rtt': 12.632}, {'rtt': 20.53}, {'dup': 1, 'rtt': 32.775}, {'dup': 1, 'rtt': 47.509}, {'dup': 1, 'rtt': 62.745}, {'dup': 1, 'rtt': 78.54}, {'dup': 1, 'rtt': 93.272}, {'dup': 1, 'rtt': 109.738}], 'ttl': 56, 'avg': 14.6733333333, 'size': 20, 'from': '46.126.90.165', 'proto': 'ICMP', 'timestamp': 1445025616, 'dup': 6, 'type': 'ping', 'sent': 3, 'msm_id': 1000192, 'fw': 4700, 'max': 20.53, 'step': 360, 'src_addr': '192.168.111.103', 'rcvd': 3, 'msm_name': 'Ping', 'lts': 309, 'dst_name': 'hsi.cablecom.ch', 'min': 10.858, 'dst_addr': '62.2.16.24'}, {'af': 4, 'prb_id': 270, 'result': [{'rtt': 28.527366}, {'rtt': 26.585862}, {'rtt': 26.393094}], 'ttl': 51, 'avg': 27.168774, 'size': 20, 'from': '188.192.110.111', 'proto': 'ICMP', 'timestamp': 1445025513, 'dup': 0, 'type': 'ping', 'sent': 3, 'msm_id': 1000192, 'fw': 4720, 'max': 28.527366, 'step': 360, 'src_addr': '192.168.178.21', 'rcvd': 3, 'msm_name': 'Ping', 'lts': 182, 'dst_name': 'hsi.cablecom.ch', 'min': 26.393094, 'group_id': 1000192, 'dst_addr': '62.2.16.24'}, {'af': 4, 'prb_id': 579, 'result': [{'rtt': 23.201285}, {'rtt': 22.980868}, {'rtt': 22.863364}], 'ttl': 51, 'avg': 23.0151723333, 'size': 20, 'from': '195.88.195.170', 'proto': 'ICMP', 'timestamp': 1445025521, 'dup': 0, 'type': 'ping', 'sent': 3, 'msm_id': 1000192, 'fw': 4720, 'max': 23.201285, 'step': 360, 'src_addr': '10.69.8.150', 'rcvd': 3, 'msm_name': 'Ping', 'lts': 56, 'dst_name': 'hsi.cablecom.ch', 'min': 22.863364, 'group_id': 1000192, 'dst_addr': '62.2.16.24'}, {'af': 4, 'prb_id': 677, 'result': [{'rtt': 40.71476}, {'rtt': 40.258568}, {'rtt': 40.316936}], 'ttl': 54, 'avg': 40.430088, 'size': 20, 'from': '78.128.9.202', 'proto': 'ICMP', 'timestamp': 1445025298, 'dup': 0, 'type': 'ping', 'sent': 3, 'msm_id': 1000192, 'fw': 4720, 'max': 40.71476, 'step': 360, 'src_addr': '10.100.0.25', 'rcvd': 3, 'msm_name': 'Ping', 'lts': 97, 'dst_name': 'hsi.cablecom.ch', 'min': 40.258568, 'group_id': 1000192, 'dst_addr': '62.2.16.24'}, {'af': 4, 'prb_id': 879, 'result': [{'rtt': 34.319623}, {'rtt': 34.445575}, {'rtt': 34.376455}], 'ttl': 53, 'avg': 34.380551, 'size': 20, 'from': '94.254.125.2', 'proto': 'ICMP', 'timestamp': 1445025223, 'dup': 0, 'type': 'ping', 'sent': 3, 'msm_id': 1000192, 'fw': 4720, 'max': 34.445575, 'step': 360, 'src_addr': '192.168.8.130', 'rcvd': 3, 'msm_name': 'Ping', 'lts': 189, 'dst_name': 'hsi.cablecom.ch', 'min': 34.319623, 'dst_addr': '62.2.16.24'}, {'af': 4, 'prb_id': 945, 'result': [{'rtt': 61.665036}, {'rtt': 23.833349}, {'rtt': 23.268868}], 'ttl': 56, 'avg': 36.255751, 'size': 20, 'from': '92.111.237.94', 'proto': 'ICMP', 'timestamp': 1445025494, 'dup': 0, 'type': 'ping', 'sent': 3, 'msm_id': 1000192, 'fw': 4720, 'max': 61.665036, 'step': 360, 'src_addr': '92.111.237.94', 'rcvd': 3, 'msm_name': 'Ping', 'lts': 746, 'dst_name': 'hsi.cablecom.ch', 'min': 23.268868, 'dst_addr': '62.2.16.24'} ] def setUp(self): self.cmd = Command() def test_raw_renderer(self): """Test case where user specified report with raw rendering.""" json_results = [] for result in self.mocked_results: json_results.append(json.dumps(result, separators=(",", ":"))) expected_output = "\n".join(json_results) + "\n" probes = [ Probe(id=202, meta_data={ "country_code": "GR", "asn_v4": 3337, "asn_v6": "4445"}), Probe(id=677, meta_data={ "country_code": "DE", "asn_v4": 3333, "asn_v6": "4444"}), Probe(id=2225, meta_data={ "country_code": "DE", "asn_v4": 3332, "asn_v6": "4444"}), Probe(id=165, meta_data={ "country_code": "NL", "asn_v4": 3334, "asn_v6": "4444"}), Probe(id=1216, meta_data={ "country_code": "GR", "asn_v4": 3335, "asn_v6": "4444"}), Probe(id=270, meta_data={ "country_code": "GR", "asn_v4": 3340, "asn_v6": "4444"}), Probe(id=579, meta_data={ "country_code": "GR", "asn_v4": 3336, "asn_v6": "4444"}), Probe(id=945, meta_data={ "country_code": "GR", "asn_v4": 3334, "asn_v6": "4444"}), Probe(id=879, meta_data={ "country_code": "GR", "asn_v4": 3333, "asn_v6": "4444"}), ] with capture_sys_output() as (stdout, stderr): path = 'ripe.atlas.cousteau.AtlasRequest.get' with mock.patch(path) as mock_get: mock_get.side_effect = [ (True, {"creation_time": 1, "start_time": 1, "type": {"name": "ping"}, "description": ""}), (True, self.mocked_results) ] mpath = 'ripe.atlas.tools.helpers.rendering.Probe.get_many' with mock.patch(mpath) as mock_get_many: mock_get_many.return_value = probes self.cmd.init_args(["1", "--renderer", "raw"]) self.cmd.run() self.assertEquals(stdout.getvalue(), expected_output) ripe-atlas-tools-1.2.2/tests/renderers/ssl_consistency.py000066400000000000000000002232051264541652500236600ustar00rootroot00000000000000# Copyright (c) 2015 RIPE NCC # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . import mock import unittest from ripe.atlas.cousteau import Probe as CProbe from ripe.atlas.tools.helpers.rendering import SaganSet from ripe.atlas.tools.commands.report import Command from ripe.atlas.tools.renderers.ssl_consistency import Renderer from ..base import capture_sys_output class TestSSLConsistency(unittest.TestCase): @classmethod def setUpClass(cls): cls.results = [ {u'rt': 737.834, u'msm_id': 1443369, u'from': u'210.6.135.152', u'dst_name': u'torproject.org', u'af': 4, u'timestamp': 1392727294, u'fw': 4570, u'cert': [u'-----BEGIN CERTIFICATE-----\nMIIGujCCBaKgAwIBAgIQBt6X5R3DncJkjaxy3UEB/DANBgkqhkiG9w0BAQsFADBm\nMQswCQYDVQQGEwJVUzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMRkwFwYDVQQLExB3\nd3cuZGlnaWNlcnQuY29tMSUwIwYDVQQDExxEaWdpQ2VydCBIaWdoIEFzc3VyYW5j\nZSBDQS0zMB4XDTEzMDEyOTAwMDAwMFoXDTE2MDUwMzEyMDAwMFowcjELMAkGA1UE\nBhMCVVMxFjAUBgNVBAgTDU1hc3NhY2h1c2V0dHMxEDAOBgNVBAcTB1dhbHBvbGUx\nHjAcBgNVBAoTFVRoZSBUb3IgUHJvamVjdCwgSW5jLjEZMBcGA1UEAwwQKi50b3Jw\ncm9qZWN0Lm9yZzCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAN1oOe8B\n0kT0l6MXsIOWiBiXqLfGCk8nIeX+GXc0TNez14HBWPOzLMmA6Kfj3h9kJ0hLCzlS\nGui3xsT1ca5ZXONP/2beDkIoxwF+7/MCS8gOu4Cyua0CjR0ce6YWemKYVKxoqJvY\nH/S2UnzMHaBI/bhJ+QK5kMYg/JXoMx9IMIJnjl9clFt3TE34UR5/NZTsytXAtCjI\n5qMSpzKRE31RREGv1kxwTqJq/g5UFJWzZEwISDEhTeFTVOru0qjbEAqaip4hQH9D\nITjDOFw7Upgdab4TN4gLwDaZuo+Qcz+CQR6vCSlP2KziQAH9nlU+qT81eYVv+NOf\njogvdu/Atl/q+z0CAwEAAaOCA1YwggNSMB8GA1UdIwQYMBaAFFDqc4nbKfsQj57l\nASDU3nmZSIP3MB0GA1UdDgQWBBSx87Iq0fmAeNURYjYpnSG8riduZjArBgNVHREE\nJDAighAqLnRvcnByb2plY3Qub3Jngg50b3Jwcm9qZWN0Lm9yZzAOBgNVHQ8BAf8E\nBAMCBaAwHQYDVR0lBBYwFAYIKwYBBQUHAwEGCCsGAQUFBwMCMGEGA1UdHwRaMFgw\nKqAooCaGJGh0dHA6Ly9jcmwzLmRpZ2ljZXJ0LmNvbS9jYTMtZzE4LmNybDAqoCig\nJoYkaHR0cDovL2NybDQuZGlnaWNlcnQuY29tL2NhMy1nMTguY3JsMIIBxAYDVR0g\nBIIBuzCCAbcwggGzBglghkgBhv1sAQEwggGkMDoGCCsGAQUFBwIBFi5odHRwOi8v\nd3d3LmRpZ2ljZXJ0LmNvbS9zc2wtY3BzLXJlcG9zaXRvcnkuaHRtMIIBZAYIKwYB\nBQUHAgIwggFWHoIBUgBBAG4AeQAgAHUAcwBlACAAbwBmACAAdABoAGkAcwAgAEMA\nZQByAHQAaQBmAGkAYwBhAHQAZQAgAGMAbwBuAHMAdABpAHQAdQB0AGUAcwAgAGEA\nYwBjAGUAcAB0AGEAbgBjAGUAIABvAGYAIAB0AGgAZQAgAEQAaQBnAGkAQwBlAHIA\ndAAgAEMAUAAvAEMAUABTACAAYQBuAGQAIAB0AGgAZQAgAFIAZQBsAHkAaQBuAGcA\nIABQAGEAcgB0AHkAIABBAGcAcgBlAGUAbQBlAG4AdAAgAHcAaABpAGMAaAAgAGwA\naQBtAGkAdAAgAGwAaQBhAGIAaQBsAGkAdAB5ACAAYQBuAGQAIABhAHIAZQAgAGkA\nbgBjAG8AcgBwAG8AcgBhAHQAZQBkACAAaABlAHIAZQBpAG4AIABiAHkAIAByAGUA\nZgBlAHIAZQBuAGMAZQAuMHsGCCsGAQUFBwEBBG8wbTAkBggrBgEFBQcwAYYYaHR0\ncDovL29jc3AuZGlnaWNlcnQuY29tMEUGCCsGAQUFBzAChjlodHRwOi8vY2FjZXJ0\ncy5kaWdpY2VydC5jb20vRGlnaUNlcnRIaWdoQXNzdXJhbmNlQ0EtMy5jcnQwDAYD\nVR0TAQH/BAIwADANBgkqhkiG9w0BAQsFAAOCAQEAFfAsIxhBxzSVi5a9FpEp9JGc\n0wL5/4BVFv0lKYjHkRVoBdvN3gnAfGt2YXrAJZb7OCVwW3KFdSaTwm8T10eCVSXX\nASTrp6DWs6mHxw9HGIkVF9YESq6x5/ZGHDTovuRMCeHuIwn+nBL21z1WDqwozwcQ\nAxNXeRXJvXO4bOj301+26as9cOWjonGzkW9uc3WTWp89+YOpRo6RQ59Yc3UJlxjW\nHZR3Oqp/GM1jo2NPHnFeMpnFtVj+uuQBtNj7D7jiWhGtNxFIePizOBs8k+ao9lWO\nE2UHK5iM17YISRhBPNwi4YL+nf+jo5untE6WgvFYhEH2pwmCSKrIYBdGatbxfw==\n-----END CERTIFICATE-----', u'-----BEGIN CERTIFICATE-----\nMIIGWDCCBUCgAwIBAgIQCl8RTQNbF5EX0u/UA4w/OzANBgkqhkiG9w0BAQUFADBs\nMQswCQYDVQQGEwJVUzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMRkwFwYDVQQLExB3\nd3cuZGlnaWNlcnQuY29tMSswKQYDVQQDEyJEaWdpQ2VydCBIaWdoIEFzc3VyYW5j\nZSBFViBSb290IENBMB4XDTA4MDQwMjEyMDAwMFoXDTIyMDQwMzAwMDAwMFowZjEL\nMAkGA1UEBhMCVVMxFTATBgNVBAoTDERpZ2lDZXJ0IEluYzEZMBcGA1UECxMQd3d3\nLmRpZ2ljZXJ0LmNvbTElMCMGA1UEAxMcRGlnaUNlcnQgSGlnaCBBc3N1cmFuY2Ug\nQ0EtMzCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAL9hCikQH17+NDdR\nCPge+yLtYb4LDXBMUGMmdRW5QYiXtvCgFbsIYOBC6AUpEIc2iihlqO8xB3RtNpcv\nKEZmBMcqeSZ6mdWOw21PoF6tvD2Rwll7XjZswFPPAAgyPhBkWBATaccM7pxCUQD5\nBUTuJM56H+2MEb0SqPMV9Bx6MWkBG6fmXcCabH4JnudSREoQOiPkm7YDr6ictFuf\n1EutkozOtREqqjcYjbTCuNhcBoz4/yO9NV7UfD5+gw6RlgWYw7If48hl66l7XaAs\nzPw82W3tzPpLQ4zJ1LilYRyyQLYoEt+5+F/+07LJ7z20Hkt8HEyZNp496+ynaF4d\n32duXvsCAwEAAaOCAvowggL2MA4GA1UdDwEB/wQEAwIBhjCCAcYGA1UdIASCAb0w\nggG5MIIBtQYLYIZIAYb9bAEDAAIwggGkMDoGCCsGAQUFBwIBFi5odHRwOi8vd3d3\nLmRpZ2ljZXJ0LmNvbS9zc2wtY3BzLXJlcG9zaXRvcnkuaHRtMIIBZAYIKwYBBQUH\nAgIwggFWHoIBUgBBAG4AeQAgAHUAcwBlACAAbwBmACAAdABoAGkAcwAgAEMAZQBy\nAHQAaQBmAGkAYwBhAHQAZQAgAGMAbwBuAHMAdABpAHQAdQB0AGUAcwAgAGEAYwBj\nAGUAcAB0AGEAbgBjAGUAIABvAGYAIAB0AGgAZQAgAEQAaQBnAGkAQwBlAHIAdAAg\nAEMAUAAvAEMAUABTACAAYQBuAGQAIAB0AGgAZQAgAFIAZQBsAHkAaQBuAGcAIABQ\nAGEAcgB0AHkAIABBAGcAcgBlAGUAbQBlAG4AdAAgAHcAaABpAGMAaAAgAGwAaQBt\nAGkAdAAgAGwAaQBhAGIAaQBsAGkAdAB5ACAAYQBuAGQAIABhAHIAZQAgAGkAbgBj\nAG8AcgBwAG8AcgBhAHQAZQBkACAAaABlAHIAZQBpAG4AIABiAHkAIAByAGUAZgBl\nAHIAZQBuAGMAZQAuMBIGA1UdEwEB/wQIMAYBAf8CAQAwNAYIKwYBBQUHAQEEKDAm\nMCQGCCsGAQUFBzABhhhodHRwOi8vb2NzcC5kaWdpY2VydC5jb20wgY8GA1UdHwSB\nhzCBhDBAoD6gPIY6aHR0cDovL2NybDMuZGlnaWNlcnQuY29tL0RpZ2lDZXJ0SGln\naEFzc3VyYW5jZUVWUm9vdENBLmNybDBAoD6gPIY6aHR0cDovL2NybDQuZGlnaWNl\ncnQuY29tL0RpZ2lDZXJ0SGlnaEFzc3VyYW5jZUVWUm9vdENBLmNybDAfBgNVHSME\nGDAWgBSxPsNpA/i/RwHUmCYaCALvY2QrwzAdBgNVHQ4EFgQUUOpzidsp+xCPnuUB\nINTeeZlIg/cwDQYJKoZIhvcNAQEFBQADggEBAB7ipUiebNtTOA/vphoqrOIDQ+2a\nvD6OdRvw/S4iWawTwGHi5/rpmc2HCXVUKL9GYNy+USyS8xuRfDEIcOI3ucFbqL2j\nCwD7GhX9A61YasXHJJlIR0YxHpLvtF9ONMeQvzHB+LGEhtCcAarfilYGzjrpDq6X\ndF3XcZpCdF/ejUN83ulV7WkAywXgemFhM9EZTfkI7qA5xSU1tyvED7Ld8aW3DiTE\nJiiNeXf1L/BXunwH1OH8zVowV36GEEfdMR/X/KLCvzB8XSSq6PmuX2p0ws5rs0bY\nIb4p1I5eFdZCSucyb6Sxa1GDWL4/bcf72gMhy2oWGU4K8K2Eyl2Us1p292E=\n-----END CERTIFICATE-----'], u'method': u'SSL', u'prb_id': 1003, u'dst_port': u'443', u'dst_addr': u'86.59.30.40', u'ttc': 355.078, u'src_addr': u'192.168.1.182', u'group_id': 1443369, u'type': u'sslcert', u'msm_name': u'SSLCert', u'ver': u'3.0'}, {u'rt': 496.488, u'msm_id': 1443369, u'from': u'218.254.74.122', u'dst_name': u'torproject.org', u'af': 4, u'timestamp': 1392727295, u'fw': 4570, u'cert': [u'-----BEGIN CERTIFICATE-----\nMIIGujCCBaKgAwIBAgIQBt6X5R3DncJkjaxy3UEB/DANBgkqhkiG9w0BAQsFADBm\nMQswCQYDVQQGEwJVUzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMRkwFwYDVQQLExB3\nd3cuZGlnaWNlcnQuY29tMSUwIwYDVQQDExxEaWdpQ2VydCBIaWdoIEFzc3VyYW5j\nZSBDQS0zMB4XDTEzMDEyOTAwMDAwMFoXDTE2MDUwMzEyMDAwMFowcjELMAkGA1UE\nBhMCVVMxFjAUBgNVBAgTDU1hc3NhY2h1c2V0dHMxEDAOBgNVBAcTB1dhbHBvbGUx\nHjAcBgNVBAoTFVRoZSBUb3IgUHJvamVjdCwgSW5jLjEZMBcGA1UEAwwQKi50b3Jw\ncm9qZWN0Lm9yZzCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAN1oOe8B\n0kT0l6MXsIOWiBiXqLfGCk8nIeX+GXc0TNez14HBWPOzLMmA6Kfj3h9kJ0hLCzlS\nGui3xsT1ca5ZXONP/2beDkIoxwF+7/MCS8gOu4Cyua0CjR0ce6YWemKYVKxoqJvY\nH/S2UnzMHaBI/bhJ+QK5kMYg/JXoMx9IMIJnjl9clFt3TE34UR5/NZTsytXAtCjI\n5qMSpzKRE31RREGv1kxwTqJq/g5UFJWzZEwISDEhTeFTVOru0qjbEAqaip4hQH9D\nITjDOFw7Upgdab4TN4gLwDaZuo+Qcz+CQR6vCSlP2KziQAH9nlU+qT81eYVv+NOf\njogvdu/Atl/q+z0CAwEAAaOCA1YwggNSMB8GA1UdIwQYMBaAFFDqc4nbKfsQj57l\nASDU3nmZSIP3MB0GA1UdDgQWBBSx87Iq0fmAeNURYjYpnSG8riduZjArBgNVHREE\nJDAighAqLnRvcnByb2plY3Qub3Jngg50b3Jwcm9qZWN0Lm9yZzAOBgNVHQ8BAf8E\nBAMCBaAwHQYDVR0lBBYwFAYIKwYBBQUHAwEGCCsGAQUFBwMCMGEGA1UdHwRaMFgw\nKqAooCaGJGh0dHA6Ly9jcmwzLmRpZ2ljZXJ0LmNvbS9jYTMtZzE4LmNybDAqoCig\nJoYkaHR0cDovL2NybDQuZGlnaWNlcnQuY29tL2NhMy1nMTguY3JsMIIBxAYDVR0g\nBIIBuzCCAbcwggGzBglghkgBhv1sAQEwggGkMDoGCCsGAQUFBwIBFi5odHRwOi8v\nd3d3LmRpZ2ljZXJ0LmNvbS9zc2wtY3BzLXJlcG9zaXRvcnkuaHRtMIIBZAYIKwYB\nBQUHAgIwggFWHoIBUgBBAG4AeQAgAHUAcwBlACAAbwBmACAAdABoAGkAcwAgAEMA\nZQByAHQAaQBmAGkAYwBhAHQAZQAgAGMAbwBuAHMAdABpAHQAdQB0AGUAcwAgAGEA\nYwBjAGUAcAB0AGEAbgBjAGUAIABvAGYAIAB0AGgAZQAgAEQAaQBnAGkAQwBlAHIA\ndAAgAEMAUAAvAEMAUABTACAAYQBuAGQAIAB0AGgAZQAgAFIAZQBsAHkAaQBuAGcA\nIABQAGEAcgB0AHkAIABBAGcAcgBlAGUAbQBlAG4AdAAgAHcAaABpAGMAaAAgAGwA\naQBtAGkAdAAgAGwAaQBhAGIAaQBsAGkAdAB5ACAAYQBuAGQAIABhAHIAZQAgAGkA\nbgBjAG8AcgBwAG8AcgBhAHQAZQBkACAAaABlAHIAZQBpAG4AIABiAHkAIAByAGUA\nZgBlAHIAZQBuAGMAZQAuMHsGCCsGAQUFBwEBBG8wbTAkBggrBgEFBQcwAYYYaHR0\ncDovL29jc3AuZGlnaWNlcnQuY29tMEUGCCsGAQUFBzAChjlodHRwOi8vY2FjZXJ0\ncy5kaWdpY2VydC5jb20vRGlnaUNlcnRIaWdoQXNzdXJhbmNlQ0EtMy5jcnQwDAYD\nVR0TAQH/BAIwADANBgkqhkiG9w0BAQsFAAOCAQEAFfAsIxhBxzSVi5a9FpEp9JGc\n0wL5/4BVFv0lKYjHkRVoBdvN3gnAfGt2YXrAJZb7OCVwW3KFdSaTwm8T10eCVSXX\nASTrp6DWs6mHxw9HGIkVF9YESq6x5/ZGHDTovuRMCeHuIwn+nBL21z1WDqwozwcQ\nAxNXeRXJvXO4bOj301+26as9cOWjonGzkW9uc3WTWp89+YOpRo6RQ59Yc3UJlxjW\nHZR3Oqp/GM1jo2NPHnFeMpnFtVj+uuQBtNj7D7jiWhGtNxFIePizOBs8k+ao9lWO\nE2UHK5iM17YISRhBPNwi4YL+nf+jo5untE6WgvFYhEH2pwmCSKrIYBdGatbxfw==\n-----END CERTIFICATE-----', u'-----BEGIN CERTIFICATE-----\nMIIGWDCCBUCgAwIBAgIQCl8RTQNbF5EX0u/UA4w/OzANBgkqhkiG9w0BAQUFADBs\nMQswCQYDVQQGEwJVUzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMRkwFwYDVQQLExB3\nd3cuZGlnaWNlcnQuY29tMSswKQYDVQQDEyJEaWdpQ2VydCBIaWdoIEFzc3VyYW5j\nZSBFViBSb290IENBMB4XDTA4MDQwMjEyMDAwMFoXDTIyMDQwMzAwMDAwMFowZjEL\nMAkGA1UEBhMCVVMxFTATBgNVBAoTDERpZ2lDZXJ0IEluYzEZMBcGA1UECxMQd3d3\nLmRpZ2ljZXJ0LmNvbTElMCMGA1UEAxMcRGlnaUNlcnQgSGlnaCBBc3N1cmFuY2Ug\nQ0EtMzCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAL9hCikQH17+NDdR\nCPge+yLtYb4LDXBMUGMmdRW5QYiXtvCgFbsIYOBC6AUpEIc2iihlqO8xB3RtNpcv\nKEZmBMcqeSZ6mdWOw21PoF6tvD2Rwll7XjZswFPPAAgyPhBkWBATaccM7pxCUQD5\nBUTuJM56H+2MEb0SqPMV9Bx6MWkBG6fmXcCabH4JnudSREoQOiPkm7YDr6ictFuf\n1EutkozOtREqqjcYjbTCuNhcBoz4/yO9NV7UfD5+gw6RlgWYw7If48hl66l7XaAs\nzPw82W3tzPpLQ4zJ1LilYRyyQLYoEt+5+F/+07LJ7z20Hkt8HEyZNp496+ynaF4d\n32duXvsCAwEAAaOCAvowggL2MA4GA1UdDwEB/wQEAwIBhjCCAcYGA1UdIASCAb0w\nggG5MIIBtQYLYIZIAYb9bAEDAAIwggGkMDoGCCsGAQUFBwIBFi5odHRwOi8vd3d3\nLmRpZ2ljZXJ0LmNvbS9zc2wtY3BzLXJlcG9zaXRvcnkuaHRtMIIBZAYIKwYBBQUH\nAgIwggFWHoIBUgBBAG4AeQAgAHUAcwBlACAAbwBmACAAdABoAGkAcwAgAEMAZQBy\nAHQAaQBmAGkAYwBhAHQAZQAgAGMAbwBuAHMAdABpAHQAdQB0AGUAcwAgAGEAYwBj\nAGUAcAB0AGEAbgBjAGUAIABvAGYAIAB0AGgAZQAgAEQAaQBnAGkAQwBlAHIAdAAg\nAEMAUAAvAEMAUABTACAAYQBuAGQAIAB0AGgAZQAgAFIAZQBsAHkAaQBuAGcAIABQ\nAGEAcgB0AHkAIABBAGcAcgBlAGUAbQBlAG4AdAAgAHcAaABpAGMAaAAgAGwAaQBt\nAGkAdAAgAGwAaQBhAGIAaQBsAGkAdAB5ACAAYQBuAGQAIABhAHIAZQAgAGkAbgBj\nAG8AcgBwAG8AcgBhAHQAZQBkACAAaABlAHIAZQBpAG4AIABiAHkAIAByAGUAZgBl\nAHIAZQBuAGMAZQAuMBIGA1UdEwEB/wQIMAYBAf8CAQAwNAYIKwYBBQUHAQEEKDAm\nMCQGCCsGAQUFBzABhhhodHRwOi8vb2NzcC5kaWdpY2VydC5jb20wgY8GA1UdHwSB\nhzCBhDBAoD6gPIY6aHR0cDovL2NybDMuZGlnaWNlcnQuY29tL0RpZ2lDZXJ0SGln\naEFzc3VyYW5jZUVWUm9vdENBLmNybDBAoD6gPIY6aHR0cDovL2NybDQuZGlnaWNl\ncnQuY29tL0RpZ2lDZXJ0SGlnaEFzc3VyYW5jZUVWUm9vdENBLmNybDAfBgNVHSME\nGDAWgBSxPsNpA/i/RwHUmCYaCALvY2QrwzAdBgNVHQ4EFgQUUOpzidsp+xCPnuUB\nINTeeZlIg/cwDQYJKoZIhvcNAQEFBQADggEBAB7ipUiebNtTOA/vphoqrOIDQ+2a\nvD6OdRvw/S4iWawTwGHi5/rpmc2HCXVUKL9GYNy+USyS8xuRfDEIcOI3ucFbqL2j\nCwD7GhX9A61YasXHJJlIR0YxHpLvtF9ONMeQvzHB+LGEhtCcAarfilYGzjrpDq6X\ndF3XcZpCdF/ejUN83ulV7WkAywXgemFhM9EZTfkI7qA5xSU1tyvED7Ld8aW3DiTE\nJiiNeXf1L/BXunwH1OH8zVowV36GEEfdMR/X/KLCvzB8XSSq6PmuX2p0ws5rs0bY\nIb4p1I5eFdZCSucyb6Sxa1GDWL4/bcf72gMhy2oWGU4K8K2Eyl2Us1p292E=\n-----END CERTIFICATE-----'], u'method': u'SSL', u'prb_id': 1004, u'dst_port': u'443', u'dst_addr': u'38.229.72.14', u'ttc': 239.37, u'src_addr': u'192.168.1.100', u'group_id': 1443369, u'type': u'sslcert', u'msm_name': u'SSLCert', u'ver': u'3.0'}, {u'from': u'183.62.4.188', u'dst_name': u'torproject.org', u'err': u'connect: timeout', u'fw': 4600, u'timestamp': 1392727294, u'af': 4, u'dst_addr': u'216.234.179.13', u'prb_id': 1008, u'dst_port': u'443', u'group_id': 1443369, u'type': u'sslcert', u'msm_name': u'SSLCert', u'msm_id': 1443369}, {u'rt': 658.671, u'msm_id': 1443369, u'from': u'113.255.77.244', u'dst_name': u'torproject.org', u'af': 4, u'timestamp': 1392727293, u'fw': 4570, u'cert': [u'-----BEGIN CERTIFICATE-----\nMIIGujCCBaKgAwIBAgIQBt6X5R3DncJkjaxy3UEB/DANBgkqhkiG9w0BAQsFADBm\nMQswCQYDVQQGEwJVUzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMRkwFwYDVQQLExB3\nd3cuZGlnaWNlcnQuY29tMSUwIwYDVQQDExxEaWdpQ2VydCBIaWdoIEFzc3VyYW5j\nZSBDQS0zMB4XDTEzMDEyOTAwMDAwMFoXDTE2MDUwMzEyMDAwMFowcjELMAkGA1UE\nBhMCVVMxFjAUBgNVBAgTDU1hc3NhY2h1c2V0dHMxEDAOBgNVBAcTB1dhbHBvbGUx\nHjAcBgNVBAoTFVRoZSBUb3IgUHJvamVjdCwgSW5jLjEZMBcGA1UEAwwQKi50b3Jw\ncm9qZWN0Lm9yZzCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAN1oOe8B\n0kT0l6MXsIOWiBiXqLfGCk8nIeX+GXc0TNez14HBWPOzLMmA6Kfj3h9kJ0hLCzlS\nGui3xsT1ca5ZXONP/2beDkIoxwF+7/MCS8gOu4Cyua0CjR0ce6YWemKYVKxoqJvY\nH/S2UnzMHaBI/bhJ+QK5kMYg/JXoMx9IMIJnjl9clFt3TE34UR5/NZTsytXAtCjI\n5qMSpzKRE31RREGv1kxwTqJq/g5UFJWzZEwISDEhTeFTVOru0qjbEAqaip4hQH9D\nITjDOFw7Upgdab4TN4gLwDaZuo+Qcz+CQR6vCSlP2KziQAH9nlU+qT81eYVv+NOf\njogvdu/Atl/q+z0CAwEAAaOCA1YwggNSMB8GA1UdIwQYMBaAFFDqc4nbKfsQj57l\nASDU3nmZSIP3MB0GA1UdDgQWBBSx87Iq0fmAeNURYjYpnSG8riduZjArBgNVHREE\nJDAighAqLnRvcnByb2plY3Qub3Jngg50b3Jwcm9qZWN0Lm9yZzAOBgNVHQ8BAf8E\nBAMCBaAwHQYDVR0lBBYwFAYIKwYBBQUHAwEGCCsGAQUFBwMCMGEGA1UdHwRaMFgw\nKqAooCaGJGh0dHA6Ly9jcmwzLmRpZ2ljZXJ0LmNvbS9jYTMtZzE4LmNybDAqoCig\nJoYkaHR0cDovL2NybDQuZGlnaWNlcnQuY29tL2NhMy1nMTguY3JsMIIBxAYDVR0g\nBIIBuzCCAbcwggGzBglghkgBhv1sAQEwggGkMDoGCCsGAQUFBwIBFi5odHRwOi8v\nd3d3LmRpZ2ljZXJ0LmNvbS9zc2wtY3BzLXJlcG9zaXRvcnkuaHRtMIIBZAYIKwYB\nBQUHAgIwggFWHoIBUgBBAG4AeQAgAHUAcwBlACAAbwBmACAAdABoAGkAcwAgAEMA\nZQByAHQAaQBmAGkAYwBhAHQAZQAgAGMAbwBuAHMAdABpAHQAdQB0AGUAcwAgAGEA\nYwBjAGUAcAB0AGEAbgBjAGUAIABvAGYAIAB0AGgAZQAgAEQAaQBnAGkAQwBlAHIA\ndAAgAEMAUAAvAEMAUABTACAAYQBuAGQAIAB0AGgAZQAgAFIAZQBsAHkAaQBuAGcA\nIABQAGEAcgB0AHkAIABBAGcAcgBlAGUAbQBlAG4AdAAgAHcAaABpAGMAaAAgAGwA\naQBtAGkAdAAgAGwAaQBhAGIAaQBsAGkAdAB5ACAAYQBuAGQAIABhAHIAZQAgAGkA\nbgBjAG8AcgBwAG8AcgBhAHQAZQBkACAAaABlAHIAZQBpAG4AIABiAHkAIAByAGUA\nZgBlAHIAZQBuAGMAZQAuMHsGCCsGAQUFBwEBBG8wbTAkBggrBgEFBQcwAYYYaHR0\ncDovL29jc3AuZGlnaWNlcnQuY29tMEUGCCsGAQUFBzAChjlodHRwOi8vY2FjZXJ0\ncy5kaWdpY2VydC5jb20vRGlnaUNlcnRIaWdoQXNzdXJhbmNlQ0EtMy5jcnQwDAYD\nVR0TAQH/BAIwADANBgkqhkiG9w0BAQsFAAOCAQEAFfAsIxhBxzSVi5a9FpEp9JGc\n0wL5/4BVFv0lKYjHkRVoBdvN3gnAfGt2YXrAJZb7OCVwW3KFdSaTwm8T10eCVSXX\nASTrp6DWs6mHxw9HGIkVF9YESq6x5/ZGHDTovuRMCeHuIwn+nBL21z1WDqwozwcQ\nAxNXeRXJvXO4bOj301+26as9cOWjonGzkW9uc3WTWp89+YOpRo6RQ59Yc3UJlxjW\nHZR3Oqp/GM1jo2NPHnFeMpnFtVj+uuQBtNj7D7jiWhGtNxFIePizOBs8k+ao9lWO\nE2UHK5iM17YISRhBPNwi4YL+nf+jo5untE6WgvFYhEH2pwmCSKrIYBdGatbxfw==\n-----END CERTIFICATE-----', u'-----BEGIN CERTIFICATE-----\nMIIGWDCCBUCgAwIBAgIQCl8RTQNbF5EX0u/UA4w/OzANBgkqhkiG9w0BAQUFADBs\nMQswCQYDVQQGEwJVUzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMRkwFwYDVQQLExB3\nd3cuZGlnaWNlcnQuY29tMSswKQYDVQQDEyJEaWdpQ2VydCBIaWdoIEFzc3VyYW5j\nZSBFViBSb290IENBMB4XDTA4MDQwMjEyMDAwMFoXDTIyMDQwMzAwMDAwMFowZjEL\nMAkGA1UEBhMCVVMxFTATBgNVBAoTDERpZ2lDZXJ0IEluYzEZMBcGA1UECxMQd3d3\nLmRpZ2ljZXJ0LmNvbTElMCMGA1UEAxMcRGlnaUNlcnQgSGlnaCBBc3N1cmFuY2Ug\nQ0EtMzCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAL9hCikQH17+NDdR\nCPge+yLtYb4LDXBMUGMmdRW5QYiXtvCgFbsIYOBC6AUpEIc2iihlqO8xB3RtNpcv\nKEZmBMcqeSZ6mdWOw21PoF6tvD2Rwll7XjZswFPPAAgyPhBkWBATaccM7pxCUQD5\nBUTuJM56H+2MEb0SqPMV9Bx6MWkBG6fmXcCabH4JnudSREoQOiPkm7YDr6ictFuf\n1EutkozOtREqqjcYjbTCuNhcBoz4/yO9NV7UfD5+gw6RlgWYw7If48hl66l7XaAs\nzPw82W3tzPpLQ4zJ1LilYRyyQLYoEt+5+F/+07LJ7z20Hkt8HEyZNp496+ynaF4d\n32duXvsCAwEAAaOCAvowggL2MA4GA1UdDwEB/wQEAwIBhjCCAcYGA1UdIASCAb0w\nggG5MIIBtQYLYIZIAYb9bAEDAAIwggGkMDoGCCsGAQUFBwIBFi5odHRwOi8vd3d3\nLmRpZ2ljZXJ0LmNvbS9zc2wtY3BzLXJlcG9zaXRvcnkuaHRtMIIBZAYIKwYBBQUH\nAgIwggFWHoIBUgBBAG4AeQAgAHUAcwBlACAAbwBmACAAdABoAGkAcwAgAEMAZQBy\nAHQAaQBmAGkAYwBhAHQAZQAgAGMAbwBuAHMAdABpAHQAdQB0AGUAcwAgAGEAYwBj\nAGUAcAB0AGEAbgBjAGUAIABvAGYAIAB0AGgAZQAgAEQAaQBnAGkAQwBlAHIAdAAg\nAEMAUAAvAEMAUABTACAAYQBuAGQAIAB0AGgAZQAgAFIAZQBsAHkAaQBuAGcAIABQ\nAGEAcgB0AHkAIABBAGcAcgBlAGUAbQBlAG4AdAAgAHcAaABpAGMAaAAgAGwAaQBt\nAGkAdAAgAGwAaQBhAGIAaQBsAGkAdAB5ACAAYQBuAGQAIABhAHIAZQAgAGkAbgBj\nAG8AcgBwAG8AcgBhAHQAZQBkACAAaABlAHIAZQBpAG4AIABiAHkAIAByAGUAZgBl\nAHIAZQBuAGMAZQAuMBIGA1UdEwEB/wQIMAYBAf8CAQAwNAYIKwYBBQUHAQEEKDAm\nMCQGCCsGAQUFBzABhhhodHRwOi8vb2NzcC5kaWdpY2VydC5jb20wgY8GA1UdHwSB\nhzCBhDBAoD6gPIY6aHR0cDovL2NybDMuZGlnaWNlcnQuY29tL0RpZ2lDZXJ0SGln\naEFzc3VyYW5jZUVWUm9vdENBLmNybDBAoD6gPIY6aHR0cDovL2NybDQuZGlnaWNl\ncnQuY29tL0RpZ2lDZXJ0SGlnaEFzc3VyYW5jZUVWUm9vdENBLmNybDAfBgNVHSME\nGDAWgBSxPsNpA/i/RwHUmCYaCALvY2QrwzAdBgNVHQ4EFgQUUOpzidsp+xCPnuUB\nINTeeZlIg/cwDQYJKoZIhvcNAQEFBQADggEBAB7ipUiebNtTOA/vphoqrOIDQ+2a\nvD6OdRvw/S4iWawTwGHi5/rpmc2HCXVUKL9GYNy+USyS8xuRfDEIcOI3ucFbqL2j\nCwD7GhX9A61YasXHJJlIR0YxHpLvtF9ONMeQvzHB+LGEhtCcAarfilYGzjrpDq6X\ndF3XcZpCdF/ejUN83ulV7WkAywXgemFhM9EZTfkI7qA5xSU1tyvED7Ld8aW3DiTE\nJiiNeXf1L/BXunwH1OH8zVowV36GEEfdMR/X/KLCvzB8XSSq6PmuX2p0ws5rs0bY\nIb4p1I5eFdZCSucyb6Sxa1GDWL4/bcf72gMhy2oWGU4K8K2Eyl2Us1p292E=\n-----END CERTIFICATE-----'], u'method': u'SSL', u'prb_id': 1033, u'dst_port': u'443', u'dst_addr': u'82.195.75.101', u'ttc': 324.6, u'src_addr': u'192.168.0.183', u'group_id': 1443369, u'type': u'sslcert', u'msm_name': u'SSLCert', u'ver': u'3.0'}, {u'rt': 518.082, u'msm_id': 1443369, u'from': u'203.218.168.183', u'dst_name': u'torproject.org', u'af': 4, u'timestamp': 1392727295, u'fw': 4600, u'cert': [u'-----BEGIN CERTIFICATE-----\nMIIGujCCBaKgAwIBAgIQBt6X5R3DncJkjaxy3UEB/DANBgkqhkiG9w0BAQsFADBm\nMQswCQYDVQQGEwJVUzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMRkwFwYDVQQLExB3\nd3cuZGlnaWNlcnQuY29tMSUwIwYDVQQDExxEaWdpQ2VydCBIaWdoIEFzc3VyYW5j\nZSBDQS0zMB4XDTEzMDEyOTAwMDAwMFoXDTE2MDUwMzEyMDAwMFowcjELMAkGA1UE\nBhMCVVMxFjAUBgNVBAgTDU1hc3NhY2h1c2V0dHMxEDAOBgNVBAcTB1dhbHBvbGUx\nHjAcBgNVBAoTFVRoZSBUb3IgUHJvamVjdCwgSW5jLjEZMBcGA1UEAwwQKi50b3Jw\ncm9qZWN0Lm9yZzCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAN1oOe8B\n0kT0l6MXsIOWiBiXqLfGCk8nIeX+GXc0TNez14HBWPOzLMmA6Kfj3h9kJ0hLCzlS\nGui3xsT1ca5ZXONP/2beDkIoxwF+7/MCS8gOu4Cyua0CjR0ce6YWemKYVKxoqJvY\nH/S2UnzMHaBI/bhJ+QK5kMYg/JXoMx9IMIJnjl9clFt3TE34UR5/NZTsytXAtCjI\n5qMSpzKRE31RREGv1kxwTqJq/g5UFJWzZEwISDEhTeFTVOru0qjbEAqaip4hQH9D\nITjDOFw7Upgdab4TN4gLwDaZuo+Qcz+CQR6vCSlP2KziQAH9nlU+qT81eYVv+NOf\njogvdu/Atl/q+z0CAwEAAaOCA1YwggNSMB8GA1UdIwQYMBaAFFDqc4nbKfsQj57l\nASDU3nmZSIP3MB0GA1UdDgQWBBSx87Iq0fmAeNURYjYpnSG8riduZjArBgNVHREE\nJDAighAqLnRvcnByb2plY3Qub3Jngg50b3Jwcm9qZWN0Lm9yZzAOBgNVHQ8BAf8E\nBAMCBaAwHQYDVR0lBBYwFAYIKwYBBQUHAwEGCCsGAQUFBwMCMGEGA1UdHwRaMFgw\nKqAooCaGJGh0dHA6Ly9jcmwzLmRpZ2ljZXJ0LmNvbS9jYTMtZzE4LmNybDAqoCig\nJoYkaHR0cDovL2NybDQuZGlnaWNlcnQuY29tL2NhMy1nMTguY3JsMIIBxAYDVR0g\nBIIBuzCCAbcwggGzBglghkgBhv1sAQEwggGkMDoGCCsGAQUFBwIBFi5odHRwOi8v\nd3d3LmRpZ2ljZXJ0LmNvbS9zc2wtY3BzLXJlcG9zaXRvcnkuaHRtMIIBZAYIKwYB\nBQUHAgIwggFWHoIBUgBBAG4AeQAgAHUAcwBlACAAbwBmACAAdABoAGkAcwAgAEMA\nZQByAHQAaQBmAGkAYwBhAHQAZQAgAGMAbwBuAHMAdABpAHQAdQB0AGUAcwAgAGEA\nYwBjAGUAcAB0AGEAbgBjAGUAIABvAGYAIAB0AGgAZQAgAEQAaQBnAGkAQwBlAHIA\ndAAgAEMAUAAvAEMAUABTACAAYQBuAGQAIAB0AGgAZQAgAFIAZQBsAHkAaQBuAGcA\nIABQAGEAcgB0AHkAIABBAGcAcgBlAGUAbQBlAG4AdAAgAHcAaABpAGMAaAAgAGwA\naQBtAGkAdAAgAGwAaQBhAGIAaQBsAGkAdAB5ACAAYQBuAGQAIABhAHIAZQAgAGkA\nbgBjAG8AcgBwAG8AcgBhAHQAZQBkACAAaABlAHIAZQBpAG4AIABiAHkAIAByAGUA\nZgBlAHIAZQBuAGMAZQAuMHsGCCsGAQUFBwEBBG8wbTAkBggrBgEFBQcwAYYYaHR0\ncDovL29jc3AuZGlnaWNlcnQuY29tMEUGCCsGAQUFBzAChjlodHRwOi8vY2FjZXJ0\ncy5kaWdpY2VydC5jb20vRGlnaUNlcnRIaWdoQXNzdXJhbmNlQ0EtMy5jcnQwDAYD\nVR0TAQH/BAIwADANBgkqhkiG9w0BAQsFAAOCAQEAFfAsIxhBxzSVi5a9FpEp9JGc\n0wL5/4BVFv0lKYjHkRVoBdvN3gnAfGt2YXrAJZb7OCVwW3KFdSaTwm8T10eCVSXX\nASTrp6DWs6mHxw9HGIkVF9YESq6x5/ZGHDTovuRMCeHuIwn+nBL21z1WDqwozwcQ\nAxNXeRXJvXO4bOj301+26as9cOWjonGzkW9uc3WTWp89+YOpRo6RQ59Yc3UJlxjW\nHZR3Oqp/GM1jo2NPHnFeMpnFtVj+uuQBtNj7D7jiWhGtNxFIePizOBs8k+ao9lWO\nE2UHK5iM17YISRhBPNwi4YL+nf+jo5untE6WgvFYhEH2pwmCSKrIYBdGatbxfw==\n-----END CERTIFICATE-----', u'-----BEGIN CERTIFICATE-----\nMIIGWDCCBUCgAwIBAgIQCl8RTQNbF5EX0u/UA4w/OzANBgkqhkiG9w0BAQUFADBs\nMQswCQYDVQQGEwJVUzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMRkwFwYDVQQLExB3\nd3cuZGlnaWNlcnQuY29tMSswKQYDVQQDEyJEaWdpQ2VydCBIaWdoIEFzc3VyYW5j\nZSBFViBSb290IENBMB4XDTA4MDQwMjEyMDAwMFoXDTIyMDQwMzAwMDAwMFowZjEL\nMAkGA1UEBhMCVVMxFTATBgNVBAoTDERpZ2lDZXJ0IEluYzEZMBcGA1UECxMQd3d3\nLmRpZ2ljZXJ0LmNvbTElMCMGA1UEAxMcRGlnaUNlcnQgSGlnaCBBc3N1cmFuY2Ug\nQ0EtMzCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAL9hCikQH17+NDdR\nCPge+yLtYb4LDXBMUGMmdRW5QYiXtvCgFbsIYOBC6AUpEIc2iihlqO8xB3RtNpcv\nKEZmBMcqeSZ6mdWOw21PoF6tvD2Rwll7XjZswFPPAAgyPhBkWBATaccM7pxCUQD5\nBUTuJM56H+2MEb0SqPMV9Bx6MWkBG6fmXcCabH4JnudSREoQOiPkm7YDr6ictFuf\n1EutkozOtREqqjcYjbTCuNhcBoz4/yO9NV7UfD5+gw6RlgWYw7If48hl66l7XaAs\nzPw82W3tzPpLQ4zJ1LilYRyyQLYoEt+5+F/+07LJ7z20Hkt8HEyZNp496+ynaF4d\n32duXvsCAwEAAaOCAvowggL2MA4GA1UdDwEB/wQEAwIBhjCCAcYGA1UdIASCAb0w\nggG5MIIBtQYLYIZIAYb9bAEDAAIwggGkMDoGCCsGAQUFBwIBFi5odHRwOi8vd3d3\nLmRpZ2ljZXJ0LmNvbS9zc2wtY3BzLXJlcG9zaXRvcnkuaHRtMIIBZAYIKwYBBQUH\nAgIwggFWHoIBUgBBAG4AeQAgAHUAcwBlACAAbwBmACAAdABoAGkAcwAgAEMAZQBy\nAHQAaQBmAGkAYwBhAHQAZQAgAGMAbwBuAHMAdABpAHQAdQB0AGUAcwAgAGEAYwBj\nAGUAcAB0AGEAbgBjAGUAIABvAGYAIAB0AGgAZQAgAEQAaQBnAGkAQwBlAHIAdAAg\nAEMAUAAvAEMAUABTACAAYQBuAGQAIAB0AGgAZQAgAFIAZQBsAHkAaQBuAGcAIABQ\nAGEAcgB0AHkAIABBAGcAcgBlAGUAbQBlAG4AdAAgAHcAaABpAGMAaAAgAGwAaQBt\nAGkAdAAgAGwAaQBhAGIAaQBsAGkAdAB5ACAAYQBuAGQAIABhAHIAZQAgAGkAbgBj\nAG8AcgBwAG8AcgBhAHQAZQBkACAAaABlAHIAZQBpAG4AIABiAHkAIAByAGUAZgBl\nAHIAZQBuAGMAZQAuMBIGA1UdEwEB/wQIMAYBAf8CAQAwNAYIKwYBBQUHAQEEKDAm\nMCQGCCsGAQUFBzABhhhodHRwOi8vb2NzcC5kaWdpY2VydC5jb20wgY8GA1UdHwSB\nhzCBhDBAoD6gPIY6aHR0cDovL2NybDMuZGlnaWNlcnQuY29tL0RpZ2lDZXJ0SGln\naEFzc3VyYW5jZUVWUm9vdENBLmNybDBAoD6gPIY6aHR0cDovL2NybDQuZGlnaWNl\ncnQuY29tL0RpZ2lDZXJ0SGlnaEFzc3VyYW5jZUVWUm9vdENBLmNybDAfBgNVHSME\nGDAWgBSxPsNpA/i/RwHUmCYaCALvY2QrwzAdBgNVHQ4EFgQUUOpzidsp+xCPnuUB\nINTeeZlIg/cwDQYJKoZIhvcNAQEFBQADggEBAB7ipUiebNtTOA/vphoqrOIDQ+2a\nvD6OdRvw/S4iWawTwGHi5/rpmc2HCXVUKL9GYNy+USyS8xuRfDEIcOI3ucFbqL2j\nCwD7GhX9A61YasXHJJlIR0YxHpLvtF9ONMeQvzHB+LGEhtCcAarfilYGzjrpDq6X\ndF3XcZpCdF/ejUN83ulV7WkAywXgemFhM9EZTfkI7qA5xSU1tyvED7Ld8aW3DiTE\nJiiNeXf1L/BXunwH1OH8zVowV36GEEfdMR/X/KLCvzB8XSSq6PmuX2p0ws5rs0bY\nIb4p1I5eFdZCSucyb6Sxa1GDWL4/bcf72gMhy2oWGU4K8K2Eyl2Us1p292E=\n-----END CERTIFICATE-----'], u'method': u'SSL', u'prb_id': 1038, u'dst_port': u'443', u'dst_addr': u'38.229.72.14', u'ttc': 244.924, u'src_addr': u'192.168.95.248', u'group_id': 1443369, u'type': u'sslcert', u'msm_name': u'SSLCert', u'ver': u'3.0'}, {u'rt': 615.64, u'msm_id': 1443369, u'from': u'103.244.157.214', u'dst_name': u'torproject.org', u'af': 4, u'timestamp': 1392727295, u'fw': 4600, u'cert': [u'-----BEGIN CERTIFICATE-----\nMIIGujCCBaKgAwIBAgIQBt6X5R3DncJkjaxy3UEB/DANBgkqhkiG9w0BAQsFADBm\nMQswCQYDVQQGEwJVUzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMRkwFwYDVQQLExB3\nd3cuZGlnaWNlcnQuY29tMSUwIwYDVQQDExxEaWdpQ2VydCBIaWdoIEFzc3VyYW5j\nZSBDQS0zMB4XDTEzMDEyOTAwMDAwMFoXDTE2MDUwMzEyMDAwMFowcjELMAkGA1UE\nBhMCVVMxFjAUBgNVBAgTDU1hc3NhY2h1c2V0dHMxEDAOBgNVBAcTB1dhbHBvbGUx\nHjAcBgNVBAoTFVRoZSBUb3IgUHJvamVjdCwgSW5jLjEZMBcGA1UEAwwQKi50b3Jw\ncm9qZWN0Lm9yZzCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAN1oOe8B\n0kT0l6MXsIOWiBiXqLfGCk8nIeX+GXc0TNez14HBWPOzLMmA6Kfj3h9kJ0hLCzlS\nGui3xsT1ca5ZXONP/2beDkIoxwF+7/MCS8gOu4Cyua0CjR0ce6YWemKYVKxoqJvY\nH/S2UnzMHaBI/bhJ+QK5kMYg/JXoMx9IMIJnjl9clFt3TE34UR5/NZTsytXAtCjI\n5qMSpzKRE31RREGv1kxwTqJq/g5UFJWzZEwISDEhTeFTVOru0qjbEAqaip4hQH9D\nITjDOFw7Upgdab4TN4gLwDaZuo+Qcz+CQR6vCSlP2KziQAH9nlU+qT81eYVv+NOf\njogvdu/Atl/q+z0CAwEAAaOCA1YwggNSMB8GA1UdIwQYMBaAFFDqc4nbKfsQj57l\nASDU3nmZSIP3MB0GA1UdDgQWBBSx87Iq0fmAeNURYjYpnSG8riduZjArBgNVHREE\nJDAighAqLnRvcnByb2plY3Qub3Jngg50b3Jwcm9qZWN0Lm9yZzAOBgNVHQ8BAf8E\nBAMCBaAwHQYDVR0lBBYwFAYIKwYBBQUHAwEGCCsGAQUFBwMCMGEGA1UdHwRaMFgw\nKqAooCaGJGh0dHA6Ly9jcmwzLmRpZ2ljZXJ0LmNvbS9jYTMtZzE4LmNybDAqoCig\nJoYkaHR0cDovL2NybDQuZGlnaWNlcnQuY29tL2NhMy1nMTguY3JsMIIBxAYDVR0g\nBIIBuzCCAbcwggGzBglghkgBhv1sAQEwggGkMDoGCCsGAQUFBwIBFi5odHRwOi8v\nd3d3LmRpZ2ljZXJ0LmNvbS9zc2wtY3BzLXJlcG9zaXRvcnkuaHRtMIIBZAYIKwYB\nBQUHAgIwggFWHoIBUgBBAG4AeQAgAHUAcwBlACAAbwBmACAAdABoAGkAcwAgAEMA\nZQByAHQAaQBmAGkAYwBhAHQAZQAgAGMAbwBuAHMAdABpAHQAdQB0AGUAcwAgAGEA\nYwBjAGUAcAB0AGEAbgBjAGUAIABvAGYAIAB0AGgAZQAgAEQAaQBnAGkAQwBlAHIA\ndAAgAEMAUAAvAEMAUABTACAAYQBuAGQAIAB0AGgAZQAgAFIAZQBsAHkAaQBuAGcA\nIABQAGEAcgB0AHkAIABBAGcAcgBlAGUAbQBlAG4AdAAgAHcAaABpAGMAaAAgAGwA\naQBtAGkAdAAgAGwAaQBhAGIAaQBsAGkAdAB5ACAAYQBuAGQAIABhAHIAZQAgAGkA\nbgBjAG8AcgBwAG8AcgBhAHQAZQBkACAAaABlAHIAZQBpAG4AIABiAHkAIAByAGUA\nZgBlAHIAZQBuAGMAZQAuMHsGCCsGAQUFBwEBBG8wbTAkBggrBgEFBQcwAYYYaHR0\ncDovL29jc3AuZGlnaWNlcnQuY29tMEUGCCsGAQUFBzAChjlodHRwOi8vY2FjZXJ0\ncy5kaWdpY2VydC5jb20vRGlnaUNlcnRIaWdoQXNzdXJhbmNlQ0EtMy5jcnQwDAYD\nVR0TAQH/BAIwADANBgkqhkiG9w0BAQsFAAOCAQEAFfAsIxhBxzSVi5a9FpEp9JGc\n0wL5/4BVFv0lKYjHkRVoBdvN3gnAfGt2YXrAJZb7OCVwW3KFdSaTwm8T10eCVSXX\nASTrp6DWs6mHxw9HGIkVF9YESq6x5/ZGHDTovuRMCeHuIwn+nBL21z1WDqwozwcQ\nAxNXeRXJvXO4bOj301+26as9cOWjonGzkW9uc3WTWp89+YOpRo6RQ59Yc3UJlxjW\nHZR3Oqp/GM1jo2NPHnFeMpnFtVj+uuQBtNj7D7jiWhGtNxFIePizOBs8k+ao9lWO\nE2UHK5iM17YISRhBPNwi4YL+nf+jo5untE6WgvFYhEH2pwmCSKrIYBdGatbxfw==\n-----END CERTIFICATE-----', u'-----BEGIN CERTIFICATE-----\nMIIGWDCCBUCgAwIBAgIQCl8RTQNbF5EX0u/UA4w/OzANBgkqhkiG9w0BAQUFADBs\nMQswCQYDVQQGEwJVUzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMRkwFwYDVQQLExB3\nd3cuZGlnaWNlcnQuY29tMSswKQYDVQQDEyJEaWdpQ2VydCBIaWdoIEFzc3VyYW5j\nZSBFViBSb290IENBMB4XDTA4MDQwMjEyMDAwMFoXDTIyMDQwMzAwMDAwMFowZjEL\nMAkGA1UEBhMCVVMxFTATBgNVBAoTDERpZ2lDZXJ0IEluYzEZMBcGA1UECxMQd3d3\nLmRpZ2ljZXJ0LmNvbTElMCMGA1UEAxMcRGlnaUNlcnQgSGlnaCBBc3N1cmFuY2Ug\nQ0EtMzCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAL9hCikQH17+NDdR\nCPge+yLtYb4LDXBMUGMmdRW5QYiXtvCgFbsIYOBC6AUpEIc2iihlqO8xB3RtNpcv\nKEZmBMcqeSZ6mdWOw21PoF6tvD2Rwll7XjZswFPPAAgyPhBkWBATaccM7pxCUQD5\nBUTuJM56H+2MEb0SqPMV9Bx6MWkBG6fmXcCabH4JnudSREoQOiPkm7YDr6ictFuf\n1EutkozOtREqqjcYjbTCuNhcBoz4/yO9NV7UfD5+gw6RlgWYw7If48hl66l7XaAs\nzPw82W3tzPpLQ4zJ1LilYRyyQLYoEt+5+F/+07LJ7z20Hkt8HEyZNp496+ynaF4d\n32duXvsCAwEAAaOCAvowggL2MA4GA1UdDwEB/wQEAwIBhjCCAcYGA1UdIASCAb0w\nggG5MIIBtQYLYIZIAYb9bAEDAAIwggGkMDoGCCsGAQUFBwIBFi5odHRwOi8vd3d3\nLmRpZ2ljZXJ0LmNvbS9zc2wtY3BzLXJlcG9zaXRvcnkuaHRtMIIBZAYIKwYBBQUH\nAgIwggFWHoIBUgBBAG4AeQAgAHUAcwBlACAAbwBmACAAdABoAGkAcwAgAEMAZQBy\nAHQAaQBmAGkAYwBhAHQAZQAgAGMAbwBuAHMAdABpAHQAdQB0AGUAcwAgAGEAYwBj\nAGUAcAB0AGEAbgBjAGUAIABvAGYAIAB0AGgAZQAgAEQAaQBnAGkAQwBlAHIAdAAg\nAEMAUAAvAEMAUABTACAAYQBuAGQAIAB0AGgAZQAgAFIAZQBsAHkAaQBuAGcAIABQ\nAGEAcgB0AHkAIABBAGcAcgBlAGUAbQBlAG4AdAAgAHcAaABpAGMAaAAgAGwAaQBt\nAGkAdAAgAGwAaQBhAGIAaQBsAGkAdAB5ACAAYQBuAGQAIABhAHIAZQAgAGkAbgBj\nAG8AcgBwAG8AcgBhAHQAZQBkACAAaABlAHIAZQBpAG4AIABiAHkAIAByAGUAZgBl\nAHIAZQBuAGMAZQAuMBIGA1UdEwEB/wQIMAYBAf8CAQAwNAYIKwYBBQUHAQEEKDAm\nMCQGCCsGAQUFBzABhhhodHRwOi8vb2NzcC5kaWdpY2VydC5jb20wgY8GA1UdHwSB\nhzCBhDBAoD6gPIY6aHR0cDovL2NybDMuZGlnaWNlcnQuY29tL0RpZ2lDZXJ0SGln\naEFzc3VyYW5jZUVWUm9vdENBLmNybDBAoD6gPIY6aHR0cDovL2NybDQuZGlnaWNl\ncnQuY29tL0RpZ2lDZXJ0SGlnaEFzc3VyYW5jZUVWUm9vdENBLmNybDAfBgNVHSME\nGDAWgBSxPsNpA/i/RwHUmCYaCALvY2QrwzAdBgNVHQ4EFgQUUOpzidsp+xCPnuUB\nINTeeZlIg/cwDQYJKoZIhvcNAQEFBQADggEBAB7ipUiebNtTOA/vphoqrOIDQ+2a\nvD6OdRvw/S4iWawTwGHi5/rpmc2HCXVUKL9GYNy+USyS8xuRfDEIcOI3ucFbqL2j\nCwD7GhX9A61YasXHJJlIR0YxHpLvtF9ONMeQvzHB+LGEhtCcAarfilYGzjrpDq6X\ndF3XcZpCdF/ejUN83ulV7WkAywXgemFhM9EZTfkI7qA5xSU1tyvED7Ld8aW3DiTE\nJiiNeXf1L/BXunwH1OH8zVowV36GEEfdMR/X/KLCvzB8XSSq6PmuX2p0ws5rs0bY\nIb4p1I5eFdZCSucyb6Sxa1GDWL4/bcf72gMhy2oWGU4K8K2Eyl2Us1p292E=\n-----END CERTIFICATE-----'], u'method': u'SSL', u'prb_id': 1047, u'dst_port': u'443', u'dst_addr': u'93.95.227.222', u'ttc': 302.403, u'src_addr': u'192.168.6.137', u'group_id': 1443369, u'type': u'sslcert', u'msm_name': u'SSLCert', u'ver': u'3.0'}, {u'from': u'116.237.191.131', u'dst_name': u'torproject.org', u'err': u'connect: timeout', u'fw': 4600, u'timestamp': 1392727292, u'af': 4, u'dst_addr': u'209.145.54.50', u'prb_id': 11572, u'dst_port': u'443', u'group_id': 1443369, u'type': u'sslcert', u'msm_name': u'SSLCert', u'msm_id': 1443369}, {u'from': u'116.247.110.182', u'dst_name': u'torproject.org', u'err': u'connect: timeout', u'fw': 4580, u'timestamp': 1392727294, u'af': 4, u'dst_addr': u'202.181.7.85', u'prb_id': 11700, u'dst_port': u'443', u'group_id': 1443369, u'type': u'sslcert', u'msm_name': u'SSLCert', u'msm_id': 1443369}, {u'rt': 673.665, u'msm_id': 1443369, u'from': u'103.6.131.59', u'dst_name': u'torproject.org', u'af': 4, u'timestamp': 1392727294, u'fw': 4580, u'cert': [u'-----BEGIN CERTIFICATE-----\nMIIGujCCBaKgAwIBAgIQBt6X5R3DncJkjaxy3UEB/DANBgkqhkiG9w0BAQsFADBm\nMQswCQYDVQQGEwJVUzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMRkwFwYDVQQLExB3\nd3cuZGlnaWNlcnQuY29tMSUwIwYDVQQDExxEaWdpQ2VydCBIaWdoIEFzc3VyYW5j\nZSBDQS0zMB4XDTEzMDEyOTAwMDAwMFoXDTE2MDUwMzEyMDAwMFowcjELMAkGA1UE\nBhMCVVMxFjAUBgNVBAgTDU1hc3NhY2h1c2V0dHMxEDAOBgNVBAcTB1dhbHBvbGUx\nHjAcBgNVBAoTFVRoZSBUb3IgUHJvamVjdCwgSW5jLjEZMBcGA1UEAwwQKi50b3Jw\ncm9qZWN0Lm9yZzCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAN1oOe8B\n0kT0l6MXsIOWiBiXqLfGCk8nIeX+GXc0TNez14HBWPOzLMmA6Kfj3h9kJ0hLCzlS\nGui3xsT1ca5ZXONP/2beDkIoxwF+7/MCS8gOu4Cyua0CjR0ce6YWemKYVKxoqJvY\nH/S2UnzMHaBI/bhJ+QK5kMYg/JXoMx9IMIJnjl9clFt3TE34UR5/NZTsytXAtCjI\n5qMSpzKRE31RREGv1kxwTqJq/g5UFJWzZEwISDEhTeFTVOru0qjbEAqaip4hQH9D\nITjDOFw7Upgdab4TN4gLwDaZuo+Qcz+CQR6vCSlP2KziQAH9nlU+qT81eYVv+NOf\njogvdu/Atl/q+z0CAwEAAaOCA1YwggNSMB8GA1UdIwQYMBaAFFDqc4nbKfsQj57l\nASDU3nmZSIP3MB0GA1UdDgQWBBSx87Iq0fmAeNURYjYpnSG8riduZjArBgNVHREE\nJDAighAqLnRvcnByb2plY3Qub3Jngg50b3Jwcm9qZWN0Lm9yZzAOBgNVHQ8BAf8E\nBAMCBaAwHQYDVR0lBBYwFAYIKwYBBQUHAwEGCCsGAQUFBwMCMGEGA1UdHwRaMFgw\nKqAooCaGJGh0dHA6Ly9jcmwzLmRpZ2ljZXJ0LmNvbS9jYTMtZzE4LmNybDAqoCig\nJoYkaHR0cDovL2NybDQuZGlnaWNlcnQuY29tL2NhMy1nMTguY3JsMIIBxAYDVR0g\nBIIBuzCCAbcwggGzBglghkgBhv1sAQEwggGkMDoGCCsGAQUFBwIBFi5odHRwOi8v\nd3d3LmRpZ2ljZXJ0LmNvbS9zc2wtY3BzLXJlcG9zaXRvcnkuaHRtMIIBZAYIKwYB\nBQUHAgIwggFWHoIBUgBBAG4AeQAgAHUAcwBlACAAbwBmACAAdABoAGkAcwAgAEMA\nZQByAHQAaQBmAGkAYwBhAHQAZQAgAGMAbwBuAHMAdABpAHQAdQB0AGUAcwAgAGEA\nYwBjAGUAcAB0AGEAbgBjAGUAIABvAGYAIAB0AGgAZQAgAEQAaQBnAGkAQwBlAHIA\ndAAgAEMAUAAvAEMAUABTACAAYQBuAGQAIAB0AGgAZQAgAFIAZQBsAHkAaQBuAGcA\nIABQAGEAcgB0AHkAIABBAGcAcgBlAGUAbQBlAG4AdAAgAHcAaABpAGMAaAAgAGwA\naQBtAGkAdAAgAGwAaQBhAGIAaQBsAGkAdAB5ACAAYQBuAGQAIABhAHIAZQAgAGkA\nbgBjAG8AcgBwAG8AcgBhAHQAZQBkACAAaABlAHIAZQBpAG4AIABiAHkAIAByAGUA\nZgBlAHIAZQBuAGMAZQAuMHsGCCsGAQUFBwEBBG8wbTAkBggrBgEFBQcwAYYYaHR0\ncDovL29jc3AuZGlnaWNlcnQuY29tMEUGCCsGAQUFBzAChjlodHRwOi8vY2FjZXJ0\ncy5kaWdpY2VydC5jb20vRGlnaUNlcnRIaWdoQXNzdXJhbmNlQ0EtMy5jcnQwDAYD\nVR0TAQH/BAIwADANBgkqhkiG9w0BAQsFAAOCAQEAFfAsIxhBxzSVi5a9FpEp9JGc\n0wL5/4BVFv0lKYjHkRVoBdvN3gnAfGt2YXrAJZb7OCVwW3KFdSaTwm8T10eCVSXX\nASTrp6DWs6mHxw9HGIkVF9YESq6x5/ZGHDTovuRMCeHuIwn+nBL21z1WDqwozwcQ\nAxNXeRXJvXO4bOj301+26as9cOWjonGzkW9uc3WTWp89+YOpRo6RQ59Yc3UJlxjW\nHZR3Oqp/GM1jo2NPHnFeMpnFtVj+uuQBtNj7D7jiWhGtNxFIePizOBs8k+ao9lWO\nE2UHK5iM17YISRhBPNwi4YL+nf+jo5untE6WgvFYhEH2pwmCSKrIYBdGatbxfw==\n-----END CERTIFICATE-----', u'-----BEGIN CERTIFICATE-----\nMIIGWDCCBUCgAwIBAgIQCl8RTQNbF5EX0u/UA4w/OzANBgkqhkiG9w0BAQUFADBs\nMQswCQYDVQQGEwJVUzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMRkwFwYDVQQLExB3\nd3cuZGlnaWNlcnQuY29tMSswKQYDVQQDEyJEaWdpQ2VydCBIaWdoIEFzc3VyYW5j\nZSBFViBSb290IENBMB4XDTA4MDQwMjEyMDAwMFoXDTIyMDQwMzAwMDAwMFowZjEL\nMAkGA1UEBhMCVVMxFTATBgNVBAoTDERpZ2lDZXJ0IEluYzEZMBcGA1UECxMQd3d3\nLmRpZ2ljZXJ0LmNvbTElMCMGA1UEAxMcRGlnaUNlcnQgSGlnaCBBc3N1cmFuY2Ug\nQ0EtMzCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAL9hCikQH17+NDdR\nCPge+yLtYb4LDXBMUGMmdRW5QYiXtvCgFbsIYOBC6AUpEIc2iihlqO8xB3RtNpcv\nKEZmBMcqeSZ6mdWOw21PoF6tvD2Rwll7XjZswFPPAAgyPhBkWBATaccM7pxCUQD5\nBUTuJM56H+2MEb0SqPMV9Bx6MWkBG6fmXcCabH4JnudSREoQOiPkm7YDr6ictFuf\n1EutkozOtREqqjcYjbTCuNhcBoz4/yO9NV7UfD5+gw6RlgWYw7If48hl66l7XaAs\nzPw82W3tzPpLQ4zJ1LilYRyyQLYoEt+5+F/+07LJ7z20Hkt8HEyZNp496+ynaF4d\n32duXvsCAwEAAaOCAvowggL2MA4GA1UdDwEB/wQEAwIBhjCCAcYGA1UdIASCAb0w\nggG5MIIBtQYLYIZIAYb9bAEDAAIwggGkMDoGCCsGAQUFBwIBFi5odHRwOi8vd3d3\nLmRpZ2ljZXJ0LmNvbS9zc2wtY3BzLXJlcG9zaXRvcnkuaHRtMIIBZAYIKwYBBQUH\nAgIwggFWHoIBUgBBAG4AeQAgAHUAcwBlACAAbwBmACAAdABoAGkAcwAgAEMAZQBy\nAHQAaQBmAGkAYwBhAHQAZQAgAGMAbwBuAHMAdABpAHQAdQB0AGUAcwAgAGEAYwBj\nAGUAcAB0AGEAbgBjAGUAIABvAGYAIAB0AGgAZQAgAEQAaQBnAGkAQwBlAHIAdAAg\nAEMAUAAvAEMAUABTACAAYQBuAGQAIAB0AGgAZQAgAFIAZQBsAHkAaQBuAGcAIABQ\nAGEAcgB0AHkAIABBAGcAcgBlAGUAbQBlAG4AdAAgAHcAaABpAGMAaAAgAGwAaQBt\nAGkAdAAgAGwAaQBhAGIAaQBsAGkAdAB5ACAAYQBuAGQAIABhAHIAZQAgAGkAbgBj\nAG8AcgBwAG8AcgBhAHQAZQBkACAAaABlAHIAZQBpAG4AIABiAHkAIAByAGUAZgBl\nAHIAZQBuAGMAZQAuMBIGA1UdEwEB/wQIMAYBAf8CAQAwNAYIKwYBBQUHAQEEKDAm\nMCQGCCsGAQUFBzABhhhodHRwOi8vb2NzcC5kaWdpY2VydC5jb20wgY8GA1UdHwSB\nhzCBhDBAoD6gPIY6aHR0cDovL2NybDMuZGlnaWNlcnQuY29tL0RpZ2lDZXJ0SGln\naEFzc3VyYW5jZUVWUm9vdENBLmNybDBAoD6gPIY6aHR0cDovL2NybDQuZGlnaWNl\ncnQuY29tL0RpZ2lDZXJ0SGlnaEFzc3VyYW5jZUVWUm9vdENBLmNybDAfBgNVHSME\nGDAWgBSxPsNpA/i/RwHUmCYaCALvY2QrwzAdBgNVHQ4EFgQUUOpzidsp+xCPnuUB\nINTeeZlIg/cwDQYJKoZIhvcNAQEFBQADggEBAB7ipUiebNtTOA/vphoqrOIDQ+2a\nvD6OdRvw/S4iWawTwGHi5/rpmc2HCXVUKL9GYNy+USyS8xuRfDEIcOI3ucFbqL2j\nCwD7GhX9A61YasXHJJlIR0YxHpLvtF9ONMeQvzHB+LGEhtCcAarfilYGzjrpDq6X\ndF3XcZpCdF/ejUN83ulV7WkAywXgemFhM9EZTfkI7qA5xSU1tyvED7Ld8aW3DiTE\nJiiNeXf1L/BXunwH1OH8zVowV36GEEfdMR/X/KLCvzB8XSSq6PmuX2p0ws5rs0bY\nIb4p1I5eFdZCSucyb6Sxa1GDWL4/bcf72gMhy2oWGU4K8K2Eyl2Us1p292E=\n-----END CERTIFICATE-----'], u'method': u'SSL', u'prb_id': 12203, u'dst_port': u'443', u'dst_addr': u'86.59.30.40', u'ttc': 326.088, u'src_addr': u'103.6.131.59', u'group_id': 1443369, u'type': u'sslcert', u'msm_name': u'SSLCert', u'ver': u'3.0'}, {u'rt': 768.923, u'msm_id': 1443369, u'from': u'103.15.180.250', u'dst_name': u'torproject.org', u'af': 4, u'timestamp': 1392727296, u'fw': 4580, u'cert': [u'-----BEGIN CERTIFICATE-----\nMIIGujCCBaKgAwIBAgIQBt6X5R3DncJkjaxy3UEB/DANBgkqhkiG9w0BAQsFADBm\nMQswCQYDVQQGEwJVUzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMRkwFwYDVQQLExB3\nd3cuZGlnaWNlcnQuY29tMSUwIwYDVQQDExxEaWdpQ2VydCBIaWdoIEFzc3VyYW5j\nZSBDQS0zMB4XDTEzMDEyOTAwMDAwMFoXDTE2MDUwMzEyMDAwMFowcjELMAkGA1UE\nBhMCVVMxFjAUBgNVBAgTDU1hc3NhY2h1c2V0dHMxEDAOBgNVBAcTB1dhbHBvbGUx\nHjAcBgNVBAoTFVRoZSBUb3IgUHJvamVjdCwgSW5jLjEZMBcGA1UEAwwQKi50b3Jw\ncm9qZWN0Lm9yZzCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAN1oOe8B\n0kT0l6MXsIOWiBiXqLfGCk8nIeX+GXc0TNez14HBWPOzLMmA6Kfj3h9kJ0hLCzlS\nGui3xsT1ca5ZXONP/2beDkIoxwF+7/MCS8gOu4Cyua0CjR0ce6YWemKYVKxoqJvY\nH/S2UnzMHaBI/bhJ+QK5kMYg/JXoMx9IMIJnjl9clFt3TE34UR5/NZTsytXAtCjI\n5qMSpzKRE31RREGv1kxwTqJq/g5UFJWzZEwISDEhTeFTVOru0qjbEAqaip4hQH9D\nITjDOFw7Upgdab4TN4gLwDaZuo+Qcz+CQR6vCSlP2KziQAH9nlU+qT81eYVv+NOf\njogvdu/Atl/q+z0CAwEAAaOCA1YwggNSMB8GA1UdIwQYMBaAFFDqc4nbKfsQj57l\nASDU3nmZSIP3MB0GA1UdDgQWBBSx87Iq0fmAeNURYjYpnSG8riduZjArBgNVHREE\nJDAighAqLnRvcnByb2plY3Qub3Jngg50b3Jwcm9qZWN0Lm9yZzAOBgNVHQ8BAf8E\nBAMCBaAwHQYDVR0lBBYwFAYIKwYBBQUHAwEGCCsGAQUFBwMCMGEGA1UdHwRaMFgw\nKqAooCaGJGh0dHA6Ly9jcmwzLmRpZ2ljZXJ0LmNvbS9jYTMtZzE4LmNybDAqoCig\nJoYkaHR0cDovL2NybDQuZGlnaWNlcnQuY29tL2NhMy1nMTguY3JsMIIBxAYDVR0g\nBIIBuzCCAbcwggGzBglghkgBhv1sAQEwggGkMDoGCCsGAQUFBwIBFi5odHRwOi8v\nd3d3LmRpZ2ljZXJ0LmNvbS9zc2wtY3BzLXJlcG9zaXRvcnkuaHRtMIIBZAYIKwYB\nBQUHAgIwggFWHoIBUgBBAG4AeQAgAHUAcwBlACAAbwBmACAAdABoAGkAcwAgAEMA\nZQByAHQAaQBmAGkAYwBhAHQAZQAgAGMAbwBuAHMAdABpAHQAdQB0AGUAcwAgAGEA\nYwBjAGUAcAB0AGEAbgBjAGUAIABvAGYAIAB0AGgAZQAgAEQAaQBnAGkAQwBlAHIA\ndAAgAEMAUAAvAEMAUABTACAAYQBuAGQAIAB0AGgAZQAgAFIAZQBsAHkAaQBuAGcA\nIABQAGEAcgB0AHkAIABBAGcAcgBlAGUAbQBlAG4AdAAgAHcAaABpAGMAaAAgAGwA\naQBtAGkAdAAgAGwAaQBhAGIAaQBsAGkAdAB5ACAAYQBuAGQAIABhAHIAZQAgAGkA\nbgBjAG8AcgBwAG8AcgBhAHQAZQBkACAAaABlAHIAZQBpAG4AIABiAHkAIAByAGUA\nZgBlAHIAZQBuAGMAZQAuMHsGCCsGAQUFBwEBBG8wbTAkBggrBgEFBQcwAYYYaHR0\ncDovL29jc3AuZGlnaWNlcnQuY29tMEUGCCsGAQUFBzAChjlodHRwOi8vY2FjZXJ0\ncy5kaWdpY2VydC5jb20vRGlnaUNlcnRIaWdoQXNzdXJhbmNlQ0EtMy5jcnQwDAYD\nVR0TAQH/BAIwADANBgkqhkiG9w0BAQsFAAOCAQEAFfAsIxhBxzSVi5a9FpEp9JGc\n0wL5/4BVFv0lKYjHkRVoBdvN3gnAfGt2YXrAJZb7OCVwW3KFdSaTwm8T10eCVSXX\nASTrp6DWs6mHxw9HGIkVF9YESq6x5/ZGHDTovuRMCeHuIwn+nBL21z1WDqwozwcQ\nAxNXeRXJvXO4bOj301+26as9cOWjonGzkW9uc3WTWp89+YOpRo6RQ59Yc3UJlxjW\nHZR3Oqp/GM1jo2NPHnFeMpnFtVj+uuQBtNj7D7jiWhGtNxFIePizOBs8k+ao9lWO\nE2UHK5iM17YISRhBPNwi4YL+nf+jo5untE6WgvFYhEH2pwmCSKrIYBdGatbxfw==\n-----END CERTIFICATE-----', u'-----BEGIN CERTIFICATE-----\nMIIGWDCCBUCgAwIBAgIQCl8RTQNbF5EX0u/UA4w/OzANBgkqhkiG9w0BAQUFADBs\nMQswCQYDVQQGEwJVUzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMRkwFwYDVQQLExB3\nd3cuZGlnaWNlcnQuY29tMSswKQYDVQQDEyJEaWdpQ2VydCBIaWdoIEFzc3VyYW5j\nZSBFViBSb290IENBMB4XDTA4MDQwMjEyMDAwMFoXDTIyMDQwMzAwMDAwMFowZjEL\nMAkGA1UEBhMCVVMxFTATBgNVBAoTDERpZ2lDZXJ0IEluYzEZMBcGA1UECxMQd3d3\nLmRpZ2ljZXJ0LmNvbTElMCMGA1UEAxMcRGlnaUNlcnQgSGlnaCBBc3N1cmFuY2Ug\nQ0EtMzCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAL9hCikQH17+NDdR\nCPge+yLtYb4LDXBMUGMmdRW5QYiXtvCgFbsIYOBC6AUpEIc2iihlqO8xB3RtNpcv\nKEZmBMcqeSZ6mdWOw21PoF6tvD2Rwll7XjZswFPPAAgyPhBkWBATaccM7pxCUQD5\nBUTuJM56H+2MEb0SqPMV9Bx6MWkBG6fmXcCabH4JnudSREoQOiPkm7YDr6ictFuf\n1EutkozOtREqqjcYjbTCuNhcBoz4/yO9NV7UfD5+gw6RlgWYw7If48hl66l7XaAs\nzPw82W3tzPpLQ4zJ1LilYRyyQLYoEt+5+F/+07LJ7z20Hkt8HEyZNp496+ynaF4d\n32duXvsCAwEAAaOCAvowggL2MA4GA1UdDwEB/wQEAwIBhjCCAcYGA1UdIASCAb0w\nggG5MIIBtQYLYIZIAYb9bAEDAAIwggGkMDoGCCsGAQUFBwIBFi5odHRwOi8vd3d3\nLmRpZ2ljZXJ0LmNvbS9zc2wtY3BzLXJlcG9zaXRvcnkuaHRtMIIBZAYIKwYBBQUH\nAgIwggFWHoIBUgBBAG4AeQAgAHUAcwBlACAAbwBmACAAdABoAGkAcwAgAEMAZQBy\nAHQAaQBmAGkAYwBhAHQAZQAgAGMAbwBuAHMAdABpAHQAdQB0AGUAcwAgAGEAYwBj\nAGUAcAB0AGEAbgBjAGUAIABvAGYAIAB0AGgAZQAgAEQAaQBnAGkAQwBlAHIAdAAg\nAEMAUAAvAEMAUABTACAAYQBuAGQAIAB0AGgAZQAgAFIAZQBsAHkAaQBuAGcAIABQ\nAGEAcgB0AHkAIABBAGcAcgBlAGUAbQBlAG4AdAAgAHcAaABpAGMAaAAgAGwAaQBt\nAGkAdAAgAGwAaQBhAGIAaQBsAGkAdAB5ACAAYQBuAGQAIABhAHIAZQAgAGkAbgBj\nAG8AcgBwAG8AcgBhAHQAZQBkACAAaABlAHIAZQBpAG4AIABiAHkAIAByAGUAZgBl\nAHIAZQBuAGMAZQAuMBIGA1UdEwEB/wQIMAYBAf8CAQAwNAYIKwYBBQUHAQEEKDAm\nMCQGCCsGAQUFBzABhhhodHRwOi8vb2NzcC5kaWdpY2VydC5jb20wgY8GA1UdHwSB\nhzCBhDBAoD6gPIY6aHR0cDovL2NybDMuZGlnaWNlcnQuY29tL0RpZ2lDZXJ0SGln\naEFzc3VyYW5jZUVWUm9vdENBLmNybDBAoD6gPIY6aHR0cDovL2NybDQuZGlnaWNl\ncnQuY29tL0RpZ2lDZXJ0SGlnaEFzc3VyYW5jZUVWUm9vdENBLmNybDAfBgNVHSME\nGDAWgBSxPsNpA/i/RwHUmCYaCALvY2QrwzAdBgNVHQ4EFgQUUOpzidsp+xCPnuUB\nINTeeZlIg/cwDQYJKoZIhvcNAQEFBQADggEBAB7ipUiebNtTOA/vphoqrOIDQ+2a\nvD6OdRvw/S4iWawTwGHi5/rpmc2HCXVUKL9GYNy+USyS8xuRfDEIcOI3ucFbqL2j\nCwD7GhX9A61YasXHJJlIR0YxHpLvtF9ONMeQvzHB+LGEhtCcAarfilYGzjrpDq6X\ndF3XcZpCdF/ejUN83ulV7WkAywXgemFhM9EZTfkI7qA5xSU1tyvED7Ld8aW3DiTE\nJiiNeXf1L/BXunwH1OH8zVowV36GEEfdMR/X/KLCvzB8XSSq6PmuX2p0ws5rs0bY\nIb4p1I5eFdZCSucyb6Sxa1GDWL4/bcf72gMhy2oWGU4K8K2Eyl2Us1p292E=\n-----END CERTIFICATE-----'], u'method': u'SSL', u'prb_id': 12208, u'dst_port': u'443', u'dst_addr': u'93.95.227.222', u'ttc': 381.856, u'src_addr': u'103.15.180.250', u'group_id': 1443369, u'type': u'sslcert', u'msm_name': u'SSLCert', u'ver': u'3.0'}, {u'dnserr': u'non-recoverable failure in name resolution (1)', u'from': u'103.6.129.113', u'dst_name': u'torproject.org', u'fw': 4580, u'timestamp': 1392727293, u'prb_id': 12214, u'dst_port': u'443', u'group_id': 1443369, u'type': u'sslcert', u'msm_name': u'SSLCert', u'msm_id': 1443369}, {u'rt': 514.766, u'msm_id': 1443369, u'from': u'158.132.255.55', u'dst_name': u'torproject.org', u'af': 4, u'timestamp': 1392727293, u'fw': 4580, u'cert': [u'-----BEGIN CERTIFICATE-----\nMIIGujCCBaKgAwIBAgIQBt6X5R3DncJkjaxy3UEB/DANBgkqhkiG9w0BAQsFADBm\nMQswCQYDVQQGEwJVUzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMRkwFwYDVQQLExB3\nd3cuZGlnaWNlcnQuY29tMSUwIwYDVQQDExxEaWdpQ2VydCBIaWdoIEFzc3VyYW5j\nZSBDQS0zMB4XDTEzMDEyOTAwMDAwMFoXDTE2MDUwMzEyMDAwMFowcjELMAkGA1UE\nBhMCVVMxFjAUBgNVBAgTDU1hc3NhY2h1c2V0dHMxEDAOBgNVBAcTB1dhbHBvbGUx\nHjAcBgNVBAoTFVRoZSBUb3IgUHJvamVjdCwgSW5jLjEZMBcGA1UEAwwQKi50b3Jw\ncm9qZWN0Lm9yZzCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAN1oOe8B\n0kT0l6MXsIOWiBiXqLfGCk8nIeX+GXc0TNez14HBWPOzLMmA6Kfj3h9kJ0hLCzlS\nGui3xsT1ca5ZXONP/2beDkIoxwF+7/MCS8gOu4Cyua0CjR0ce6YWemKYVKxoqJvY\nH/S2UnzMHaBI/bhJ+QK5kMYg/JXoMx9IMIJnjl9clFt3TE34UR5/NZTsytXAtCjI\n5qMSpzKRE31RREGv1kxwTqJq/g5UFJWzZEwISDEhTeFTVOru0qjbEAqaip4hQH9D\nITjDOFw7Upgdab4TN4gLwDaZuo+Qcz+CQR6vCSlP2KziQAH9nlU+qT81eYVv+NOf\njogvdu/Atl/q+z0CAwEAAaOCA1YwggNSMB8GA1UdIwQYMBaAFFDqc4nbKfsQj57l\nASDU3nmZSIP3MB0GA1UdDgQWBBSx87Iq0fmAeNURYjYpnSG8riduZjArBgNVHREE\nJDAighAqLnRvcnByb2plY3Qub3Jngg50b3Jwcm9qZWN0Lm9yZzAOBgNVHQ8BAf8E\nBAMCBaAwHQYDVR0lBBYwFAYIKwYBBQUHAwEGCCsGAQUFBwMCMGEGA1UdHwRaMFgw\nKqAooCaGJGh0dHA6Ly9jcmwzLmRpZ2ljZXJ0LmNvbS9jYTMtZzE4LmNybDAqoCig\nJoYkaHR0cDovL2NybDQuZGlnaWNlcnQuY29tL2NhMy1nMTguY3JsMIIBxAYDVR0g\nBIIBuzCCAbcwggGzBglghkgBhv1sAQEwggGkMDoGCCsGAQUFBwIBFi5odHRwOi8v\nd3d3LmRpZ2ljZXJ0LmNvbS9zc2wtY3BzLXJlcG9zaXRvcnkuaHRtMIIBZAYIKwYB\nBQUHAgIwggFWHoIBUgBBAG4AeQAgAHUAcwBlACAAbwBmACAAdABoAGkAcwAgAEMA\nZQByAHQAaQBmAGkAYwBhAHQAZQAgAGMAbwBuAHMAdABpAHQAdQB0AGUAcwAgAGEA\nYwBjAGUAcAB0AGEAbgBjAGUAIABvAGYAIAB0AGgAZQAgAEQAaQBnAGkAQwBlAHIA\ndAAgAEMAUAAvAEMAUABTACAAYQBuAGQAIAB0AGgAZQAgAFIAZQBsAHkAaQBuAGcA\nIABQAGEAcgB0AHkAIABBAGcAcgBlAGUAbQBlAG4AdAAgAHcAaABpAGMAaAAgAGwA\naQBtAGkAdAAgAGwAaQBhAGIAaQBsAGkAdAB5ACAAYQBuAGQAIABhAHIAZQAgAGkA\nbgBjAG8AcgBwAG8AcgBhAHQAZQBkACAAaABlAHIAZQBpAG4AIABiAHkAIAByAGUA\nZgBlAHIAZQBuAGMAZQAuMHsGCCsGAQUFBwEBBG8wbTAkBggrBgEFBQcwAYYYaHR0\ncDovL29jc3AuZGlnaWNlcnQuY29tMEUGCCsGAQUFBzAChjlodHRwOi8vY2FjZXJ0\ncy5kaWdpY2VydC5jb20vRGlnaUNlcnRIaWdoQXNzdXJhbmNlQ0EtMy5jcnQwDAYD\nVR0TAQH/BAIwADANBgkqhkiG9w0BAQsFAAOCAQEAFfAsIxhBxzSVi5a9FpEp9JGc\n0wL5/4BVFv0lKYjHkRVoBdvN3gnAfGt2YXrAJZb7OCVwW3KFdSaTwm8T10eCVSXX\nASTrp6DWs6mHxw9HGIkVF9YESq6x5/ZGHDTovuRMCeHuIwn+nBL21z1WDqwozwcQ\nAxNXeRXJvXO4bOj301+26as9cOWjonGzkW9uc3WTWp89+YOpRo6RQ59Yc3UJlxjW\nHZR3Oqp/GM1jo2NPHnFeMpnFtVj+uuQBtNj7D7jiWhGtNxFIePizOBs8k+ao9lWO\nE2UHK5iM17YISRhBPNwi4YL+nf+jo5untE6WgvFYhEH2pwmCSKrIYBdGatbxfw==\n-----END CERTIFICATE-----', u'-----BEGIN CERTIFICATE-----\nMIIGWDCCBUCgAwIBAgIQCl8RTQNbF5EX0u/UA4w/OzANBgkqhkiG9w0BAQUFADBs\nMQswCQYDVQQGEwJVUzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMRkwFwYDVQQLExB3\nd3cuZGlnaWNlcnQuY29tMSswKQYDVQQDEyJEaWdpQ2VydCBIaWdoIEFzc3VyYW5j\nZSBFViBSb290IENBMB4XDTA4MDQwMjEyMDAwMFoXDTIyMDQwMzAwMDAwMFowZjEL\nMAkGA1UEBhMCVVMxFTATBgNVBAoTDERpZ2lDZXJ0IEluYzEZMBcGA1UECxMQd3d3\nLmRpZ2ljZXJ0LmNvbTElMCMGA1UEAxMcRGlnaUNlcnQgSGlnaCBBc3N1cmFuY2Ug\nQ0EtMzCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAL9hCikQH17+NDdR\nCPge+yLtYb4LDXBMUGMmdRW5QYiXtvCgFbsIYOBC6AUpEIc2iihlqO8xB3RtNpcv\nKEZmBMcqeSZ6mdWOw21PoF6tvD2Rwll7XjZswFPPAAgyPhBkWBATaccM7pxCUQD5\nBUTuJM56H+2MEb0SqPMV9Bx6MWkBG6fmXcCabH4JnudSREoQOiPkm7YDr6ictFuf\n1EutkozOtREqqjcYjbTCuNhcBoz4/yO9NV7UfD5+gw6RlgWYw7If48hl66l7XaAs\nzPw82W3tzPpLQ4zJ1LilYRyyQLYoEt+5+F/+07LJ7z20Hkt8HEyZNp496+ynaF4d\n32duXvsCAwEAAaOCAvowggL2MA4GA1UdDwEB/wQEAwIBhjCCAcYGA1UdIASCAb0w\nggG5MIIBtQYLYIZIAYb9bAEDAAIwggGkMDoGCCsGAQUFBwIBFi5odHRwOi8vd3d3\nLmRpZ2ljZXJ0LmNvbS9zc2wtY3BzLXJlcG9zaXRvcnkuaHRtMIIBZAYIKwYBBQUH\nAgIwggFWHoIBUgBBAG4AeQAgAHUAcwBlACAAbwBmACAAdABoAGkAcwAgAEMAZQBy\nAHQAaQBmAGkAYwBhAHQAZQAgAGMAbwBuAHMAdABpAHQAdQB0AGUAcwAgAGEAYwBj\nAGUAcAB0AGEAbgBjAGUAIABvAGYAIAB0AGgAZQAgAEQAaQBnAGkAQwBlAHIAdAAg\nAEMAUAAvAEMAUABTACAAYQBuAGQAIAB0AGgAZQAgAFIAZQBsAHkAaQBuAGcAIABQ\nAGEAcgB0AHkAIABBAGcAcgBlAGUAbQBlAG4AdAAgAHcAaABpAGMAaAAgAGwAaQBt\nAGkAdAAgAGwAaQBhAGIAaQBsAGkAdAB5ACAAYQBuAGQAIABhAHIAZQAgAGkAbgBj\nAG8AcgBwAG8AcgBhAHQAZQBkACAAaABlAHIAZQBpAG4AIABiAHkAIAByAGUAZgBl\nAHIAZQBuAGMAZQAuMBIGA1UdEwEB/wQIMAYBAf8CAQAwNAYIKwYBBQUHAQEEKDAm\nMCQGCCsGAQUFBzABhhhodHRwOi8vb2NzcC5kaWdpY2VydC5jb20wgY8GA1UdHwSB\nhzCBhDBAoD6gPIY6aHR0cDovL2NybDMuZGlnaWNlcnQuY29tL0RpZ2lDZXJ0SGln\naEFzc3VyYW5jZUVWUm9vdENBLmNybDBAoD6gPIY6aHR0cDovL2NybDQuZGlnaWNl\ncnQuY29tL0RpZ2lDZXJ0SGlnaEFzc3VyYW5jZUVWUm9vdENBLmNybDAfBgNVHSME\nGDAWgBSxPsNpA/i/RwHUmCYaCALvY2QrwzAdBgNVHQ4EFgQUUOpzidsp+xCPnuUB\nINTeeZlIg/cwDQYJKoZIhvcNAQEFBQADggEBAB7ipUiebNtTOA/vphoqrOIDQ+2a\nvD6OdRvw/S4iWawTwGHi5/rpmc2HCXVUKL9GYNy+USyS8xuRfDEIcOI3ucFbqL2j\nCwD7GhX9A61YasXHJJlIR0YxHpLvtF9ONMeQvzHB+LGEhtCcAarfilYGzjrpDq6X\ndF3XcZpCdF/ejUN83ulV7WkAywXgemFhM9EZTfkI7qA5xSU1tyvED7Ld8aW3DiTE\nJiiNeXf1L/BXunwH1OH8zVowV36GEEfdMR/X/KLCvzB8XSSq6PmuX2p0ws5rs0bY\nIb4p1I5eFdZCSucyb6Sxa1GDWL4/bcf72gMhy2oWGU4K8K2Eyl2Us1p292E=\n-----END CERTIFICATE-----'], u'method': u'SSL', u'prb_id': 13026, u'dst_port': u'443', u'dst_addr': u'93.95.227.222', u'ttc': 252.836, u'src_addr': u'192.168.1.185', u'group_id': 1443369, u'type': u'sslcert', u'msm_name': u'SSLCert', u'ver': u'3.0'}, {u'rt': 868.668, u'msm_id': 1443369, u'from': u'116.231.230.67', u'dst_name': u'torproject.org', u'af': 4, u'timestamp': 1392727294, u'fw': 4580, u'cert': [u'-----BEGIN CERTIFICATE-----\nMIIGujCCBaKgAwIBAgIQBt6X5R3DncJkjaxy3UEB/DANBgkqhkiG9w0BAQsFADBm\nMQswCQYDVQQGEwJVUzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMRkwFwYDVQQLExB3\nd3cuZGlnaWNlcnQuY29tMSUwIwYDVQQDExxEaWdpQ2VydCBIaWdoIEFzc3VyYW5j\nZSBDQS0zMB4XDTEzMDEyOTAwMDAwMFoXDTE2MDUwMzEyMDAwMFowcjELMAkGA1UE\nBhMCVVMxFjAUBgNVBAgTDU1hc3NhY2h1c2V0dHMxEDAOBgNVBAcTB1dhbHBvbGUx\nHjAcBgNVBAoTFVRoZSBUb3IgUHJvamVjdCwgSW5jLjEZMBcGA1UEAwwQKi50b3Jw\ncm9qZWN0Lm9yZzCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAN1oOe8B\n0kT0l6MXsIOWiBiXqLfGCk8nIeX+GXc0TNez14HBWPOzLMmA6Kfj3h9kJ0hLCzlS\nGui3xsT1ca5ZXONP/2beDkIoxwF+7/MCS8gOu4Cyua0CjR0ce6YWemKYVKxoqJvY\nH/S2UnzMHaBI/bhJ+QK5kMYg/JXoMx9IMIJnjl9clFt3TE34UR5/NZTsytXAtCjI\n5qMSpzKRE31RREGv1kxwTqJq/g5UFJWzZEwISDEhTeFTVOru0qjbEAqaip4hQH9D\nITjDOFw7Upgdab4TN4gLwDaZuo+Qcz+CQR6vCSlP2KziQAH9nlU+qT81eYVv+NOf\njogvdu/Atl/q+z0CAwEAAaOCA1YwggNSMB8GA1UdIwQYMBaAFFDqc4nbKfsQj57l\nASDU3nmZSIP3MB0GA1UdDgQWBBSx87Iq0fmAeNURYjYpnSG8riduZjArBgNVHREE\nJDAighAqLnRvcnByb2plY3Qub3Jngg50b3Jwcm9qZWN0Lm9yZzAOBgNVHQ8BAf8E\nBAMCBaAwHQYDVR0lBBYwFAYIKwYBBQUHAwEGCCsGAQUFBwMCMGEGA1UdHwRaMFgw\nKqAooCaGJGh0dHA6Ly9jcmwzLmRpZ2ljZXJ0LmNvbS9jYTMtZzE4LmNybDAqoCig\nJoYkaHR0cDovL2NybDQuZGlnaWNlcnQuY29tL2NhMy1nMTguY3JsMIIBxAYDVR0g\nBIIBuzCCAbcwggGzBglghkgBhv1sAQEwggGkMDoGCCsGAQUFBwIBFi5odHRwOi8v\nd3d3LmRpZ2ljZXJ0LmNvbS9zc2wtY3BzLXJlcG9zaXRvcnkuaHRtMIIBZAYIKwYB\nBQUHAgIwggFWHoIBUgBBAG4AeQAgAHUAcwBlACAAbwBmACAAdABoAGkAcwAgAEMA\nZQByAHQAaQBmAGkAYwBhAHQAZQAgAGMAbwBuAHMAdABpAHQAdQB0AGUAcwAgAGEA\nYwBjAGUAcAB0AGEAbgBjAGUAIABvAGYAIAB0AGgAZQAgAEQAaQBnAGkAQwBlAHIA\ndAAgAEMAUAAvAEMAUABTACAAYQBuAGQAIAB0AGgAZQAgAFIAZQBsAHkAaQBuAGcA\nIABQAGEAcgB0AHkAIABBAGcAcgBlAGUAbQBlAG4AdAAgAHcAaABpAGMAaAAgAGwA\naQBtAGkAdAAgAGwAaQBhAGIAaQBsAGkAdAB5ACAAYQBuAGQAIABhAHIAZQAgAGkA\nbgBjAG8AcgBwAG8AcgBhAHQAZQBkACAAaABlAHIAZQBpAG4AIABiAHkAIAByAGUA\nZgBlAHIAZQBuAGMAZQAuMHsGCCsGAQUFBwEBBG8wbTAkBggrBgEFBQcwAYYYaHR0\ncDovL29jc3AuZGlnaWNlcnQuY29tMEUGCCsGAQUFBzAChjlodHRwOi8vY2FjZXJ0\ncy5kaWdpY2VydC5jb20vRGlnaUNlcnRIaWdoQXNzdXJhbmNlQ0EtMy5jcnQwDAYD\nVR0TAQH/BAIwADANBgkqhkiG9w0BAQsFAAOCAQEAFfAsIxhBxzSVi5a9FpEp9JGc\n0wL5/4BVFv0lKYjHkRVoBdvN3gnAfGt2YXrAJZb7OCVwW3KFdSaTwm8T10eCVSXX\nASTrp6DWs6mHxw9HGIkVF9YESq6x5/ZGHDTovuRMCeHuIwn+nBL21z1WDqwozwcQ\nAxNXeRXJvXO4bOj301+26as9cOWjonGzkW9uc3WTWp89+YOpRo6RQ59Yc3UJlxjW\nHZR3Oqp/GM1jo2NPHnFeMpnFtVj+uuQBtNj7D7jiWhGtNxFIePizOBs8k+ao9lWO\nE2UHK5iM17YISRhBPNwi4YL+nf+jo5untE6WgvFYhEH2pwmCSKrIYBdGatbxfw==\n-----END CERTIFICATE-----', u'-----BEGIN CERTIFICATE-----\nMIIGWDCCBUCgAwIBAgIQCl8RTQNbF5EX0u/UA4w/OzANBgkqhkiG9w0BAQUFADBs\nMQswCQYDVQQGEwJVUzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMRkwFwYDVQQLExB3\nd3cuZGlnaWNlcnQuY29tMSswKQYDVQQDEyJEaWdpQ2VydCBIaWdoIEFzc3VyYW5j\nZSBFViBSb290IENBMB4XDTA4MDQwMjEyMDAwMFoXDTIyMDQwMzAwMDAwMFowZjEL\nMAkGA1UEBhMCVVMxFTATBgNVBAoTDERpZ2lDZXJ0IEluYzEZMBcGA1UECxMQd3d3\nLmRpZ2ljZXJ0LmNvbTElMCMGA1UEAxMcRGlnaUNlcnQgSGlnaCBBc3N1cmFuY2Ug\nQ0EtMzCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAL9hCikQH17+NDdR\nCPge+yLtYb4LDXBMUGMmdRW5QYiXtvCgFbsIYOBC6AUpEIc2iihlqO8xB3RtNpcv\nKEZmBMcqeSZ6mdWOw21PoF6tvD2Rwll7XjZswFPPAAgyPhBkWBATaccM7pxCUQD5\nBUTuJM56H+2MEb0SqPMV9Bx6MWkBG6fmXcCabH4JnudSREoQOiPkm7YDr6ictFuf\n1EutkozOtREqqjcYjbTCuNhcBoz4/yO9NV7UfD5+gw6RlgWYw7If48hl66l7XaAs\nzPw82W3tzPpLQ4zJ1LilYRyyQLYoEt+5+F/+07LJ7z20Hkt8HEyZNp496+ynaF4d\n32duXvsCAwEAAaOCAvowggL2MA4GA1UdDwEB/wQEAwIBhjCCAcYGA1UdIASCAb0w\nggG5MIIBtQYLYIZIAYb9bAEDAAIwggGkMDoGCCsGAQUFBwIBFi5odHRwOi8vd3d3\nLmRpZ2ljZXJ0LmNvbS9zc2wtY3BzLXJlcG9zaXRvcnkuaHRtMIIBZAYIKwYBBQUH\nAgIwggFWHoIBUgBBAG4AeQAgAHUAcwBlACAAbwBmACAAdABoAGkAcwAgAEMAZQBy\nAHQAaQBmAGkAYwBhAHQAZQAgAGMAbwBuAHMAdABpAHQAdQB0AGUAcwAgAGEAYwBj\nAGUAcAB0AGEAbgBjAGUAIABvAGYAIAB0AGgAZQAgAEQAaQBnAGkAQwBlAHIAdAAg\nAEMAUAAvAEMAUABTACAAYQBuAGQAIAB0AGgAZQAgAFIAZQBsAHkAaQBuAGcAIABQ\nAGEAcgB0AHkAIABBAGcAcgBlAGUAbQBlAG4AdAAgAHcAaABpAGMAaAAgAGwAaQBt\nAGkAdAAgAGwAaQBhAGIAaQBsAGkAdAB5ACAAYQBuAGQAIABhAHIAZQAgAGkAbgBj\nAG8AcgBwAG8AcgBhAHQAZQBkACAAaABlAHIAZQBpAG4AIABiAHkAIAByAGUAZgBl\nAHIAZQBuAGMAZQAuMBIGA1UdEwEB/wQIMAYBAf8CAQAwNAYIKwYBBQUHAQEEKDAm\nMCQGCCsGAQUFBzABhhhodHRwOi8vb2NzcC5kaWdpY2VydC5jb20wgY8GA1UdHwSB\nhzCBhDBAoD6gPIY6aHR0cDovL2NybDMuZGlnaWNlcnQuY29tL0RpZ2lDZXJ0SGln\naEFzc3VyYW5jZUVWUm9vdENBLmNybDBAoD6gPIY6aHR0cDovL2NybDQuZGlnaWNl\ncnQuY29tL0RpZ2lDZXJ0SGlnaEFzc3VyYW5jZUVWUm9vdENBLmNybDAfBgNVHSME\nGDAWgBSxPsNpA/i/RwHUmCYaCALvY2QrwzAdBgNVHQ4EFgQUUOpzidsp+xCPnuUB\nINTeeZlIg/cwDQYJKoZIhvcNAQEFBQADggEBAB7ipUiebNtTOA/vphoqrOIDQ+2a\nvD6OdRvw/S4iWawTwGHi5/rpmc2HCXVUKL9GYNy+USyS8xuRfDEIcOI3ucFbqL2j\nCwD7GhX9A61YasXHJJlIR0YxHpLvtF9ONMeQvzHB+LGEhtCcAarfilYGzjrpDq6X\ndF3XcZpCdF/ejUN83ulV7WkAywXgemFhM9EZTfkI7qA5xSU1tyvED7Ld8aW3DiTE\nJiiNeXf1L/BXunwH1OH8zVowV36GEEfdMR/X/KLCvzB8XSSq6PmuX2p0ws5rs0bY\nIb4p1I5eFdZCSucyb6Sxa1GDWL4/bcf72gMhy2oWGU4K8K2Eyl2Us1p292E=\n-----END CERTIFICATE-----'], u'method': u'SSL', u'prb_id': 13377, u'dst_port': u'443', u'dst_addr': u'82.195.75.101', u'ttc': 431.422, u'src_addr': u'10.0.10.39', u'group_id': 1443369, u'type': u'sslcert', u'msm_name': u'SSLCert', u'ver': u'3.0'}, {u'rt': 471.564, u'msm_id': 1443369, u'from': u'103.247.138.60', u'dst_name': u'torproject.org', u'af': 4, u'timestamp': 1392727304, u'fw': 4600, u'cert': [u'-----BEGIN CERTIFICATE-----\nMIIGujCCBaKgAwIBAgIQBt6X5R3DncJkjaxy3UEB/DANBgkqhkiG9w0BAQsFADBm\nMQswCQYDVQQGEwJVUzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMRkwFwYDVQQLExB3\nd3cuZGlnaWNlcnQuY29tMSUwIwYDVQQDExxEaWdpQ2VydCBIaWdoIEFzc3VyYW5j\nZSBDQS0zMB4XDTEzMDEyOTAwMDAwMFoXDTE2MDUwMzEyMDAwMFowcjELMAkGA1UE\nBhMCVVMxFjAUBgNVBAgTDU1hc3NhY2h1c2V0dHMxEDAOBgNVBAcTB1dhbHBvbGUx\nHjAcBgNVBAoTFVRoZSBUb3IgUHJvamVjdCwgSW5jLjEZMBcGA1UEAwwQKi50b3Jw\ncm9qZWN0Lm9yZzCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAN1oOe8B\n0kT0l6MXsIOWiBiXqLfGCk8nIeX+GXc0TNez14HBWPOzLMmA6Kfj3h9kJ0hLCzlS\nGui3xsT1ca5ZXONP/2beDkIoxwF+7/MCS8gOu4Cyua0CjR0ce6YWemKYVKxoqJvY\nH/S2UnzMHaBI/bhJ+QK5kMYg/JXoMx9IMIJnjl9clFt3TE34UR5/NZTsytXAtCjI\n5qMSpzKRE31RREGv1kxwTqJq/g5UFJWzZEwISDEhTeFTVOru0qjbEAqaip4hQH9D\nITjDOFw7Upgdab4TN4gLwDaZuo+Qcz+CQR6vCSlP2KziQAH9nlU+qT81eYVv+NOf\njogvdu/Atl/q+z0CAwEAAaOCA1YwggNSMB8GA1UdIwQYMBaAFFDqc4nbKfsQj57l\nASDU3nmZSIP3MB0GA1UdDgQWBBSx87Iq0fmAeNURYjYpnSG8riduZjArBgNVHREE\nJDAighAqLnRvcnByb2plY3Qub3Jngg50b3Jwcm9qZWN0Lm9yZzAOBgNVHQ8BAf8E\nBAMCBaAwHQYDVR0lBBYwFAYIKwYBBQUHAwEGCCsGAQUFBwMCMGEGA1UdHwRaMFgw\nKqAooCaGJGh0dHA6Ly9jcmwzLmRpZ2ljZXJ0LmNvbS9jYTMtZzE4LmNybDAqoCig\nJoYkaHR0cDovL2NybDQuZGlnaWNlcnQuY29tL2NhMy1nMTguY3JsMIIBxAYDVR0g\nBIIBuzCCAbcwggGzBglghkgBhv1sAQEwggGkMDoGCCsGAQUFBwIBFi5odHRwOi8v\nd3d3LmRpZ2ljZXJ0LmNvbS9zc2wtY3BzLXJlcG9zaXRvcnkuaHRtMIIBZAYIKwYB\nBQUHAgIwggFWHoIBUgBBAG4AeQAgAHUAcwBlACAAbwBmACAAdABoAGkAcwAgAEMA\nZQByAHQAaQBmAGkAYwBhAHQAZQAgAGMAbwBuAHMAdABpAHQAdQB0AGUAcwAgAGEA\nYwBjAGUAcAB0AGEAbgBjAGUAIABvAGYAIAB0AGgAZQAgAEQAaQBnAGkAQwBlAHIA\ndAAgAEMAUAAvAEMAUABTACAAYQBuAGQAIAB0AGgAZQAgAFIAZQBsAHkAaQBuAGcA\nIABQAGEAcgB0AHkAIABBAGcAcgBlAGUAbQBlAG4AdAAgAHcAaABpAGMAaAAgAGwA\naQBtAGkAdAAgAGwAaQBhAGIAaQBsAGkAdAB5ACAAYQBuAGQAIABhAHIAZQAgAGkA\nbgBjAG8AcgBwAG8AcgBhAHQAZQBkACAAaABlAHIAZQBpAG4AIABiAHkAIAByAGUA\nZgBlAHIAZQBuAGMAZQAuMHsGCCsGAQUFBwEBBG8wbTAkBggrBgEFBQcwAYYYaHR0\ncDovL29jc3AuZGlnaWNlcnQuY29tMEUGCCsGAQUFBzAChjlodHRwOi8vY2FjZXJ0\ncy5kaWdpY2VydC5jb20vRGlnaUNlcnRIaWdoQXNzdXJhbmNlQ0EtMy5jcnQwDAYD\nVR0TAQH/BAIwADANBgkqhkiG9w0BAQsFAAOCAQEAFfAsIxhBxzSVi5a9FpEp9JGc\n0wL5/4BVFv0lKYjHkRVoBdvN3gnAfGt2YXrAJZb7OCVwW3KFdSaTwm8T10eCVSXX\nASTrp6DWs6mHxw9HGIkVF9YESq6x5/ZGHDTovuRMCeHuIwn+nBL21z1WDqwozwcQ\nAxNXeRXJvXO4bOj301+26as9cOWjonGzkW9uc3WTWp89+YOpRo6RQ59Yc3UJlxjW\nHZR3Oqp/GM1jo2NPHnFeMpnFtVj+uuQBtNj7D7jiWhGtNxFIePizOBs8k+ao9lWO\nE2UHK5iM17YISRhBPNwi4YL+nf+jo5untE6WgvFYhEH2pwmCSKrIYBdGatbxfw==\n-----END CERTIFICATE-----', u'-----BEGIN CERTIFICATE-----\nMIIGWDCCBUCgAwIBAgIQCl8RTQNbF5EX0u/UA4w/OzANBgkqhkiG9w0BAQUFADBs\nMQswCQYDVQQGEwJVUzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMRkwFwYDVQQLExB3\nd3cuZGlnaWNlcnQuY29tMSswKQYDVQQDEyJEaWdpQ2VydCBIaWdoIEFzc3VyYW5j\nZSBFViBSb290IENBMB4XDTA4MDQwMjEyMDAwMFoXDTIyMDQwMzAwMDAwMFowZjEL\nMAkGA1UEBhMCVVMxFTATBgNVBAoTDERpZ2lDZXJ0IEluYzEZMBcGA1UECxMQd3d3\nLmRpZ2ljZXJ0LmNvbTElMCMGA1UEAxMcRGlnaUNlcnQgSGlnaCBBc3N1cmFuY2Ug\nQ0EtMzCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAL9hCikQH17+NDdR\nCPge+yLtYb4LDXBMUGMmdRW5QYiXtvCgFbsIYOBC6AUpEIc2iihlqO8xB3RtNpcv\nKEZmBMcqeSZ6mdWOw21PoF6tvD2Rwll7XjZswFPPAAgyPhBkWBATaccM7pxCUQD5\nBUTuJM56H+2MEb0SqPMV9Bx6MWkBG6fmXcCabH4JnudSREoQOiPkm7YDr6ictFuf\n1EutkozOtREqqjcYjbTCuNhcBoz4/yO9NV7UfD5+gw6RlgWYw7If48hl66l7XaAs\nzPw82W3tzPpLQ4zJ1LilYRyyQLYoEt+5+F/+07LJ7z20Hkt8HEyZNp496+ynaF4d\n32duXvsCAwEAAaOCAvowggL2MA4GA1UdDwEB/wQEAwIBhjCCAcYGA1UdIASCAb0w\nggG5MIIBtQYLYIZIAYb9bAEDAAIwggGkMDoGCCsGAQUFBwIBFi5odHRwOi8vd3d3\nLmRpZ2ljZXJ0LmNvbS9zc2wtY3BzLXJlcG9zaXRvcnkuaHRtMIIBZAYIKwYBBQUH\nAgIwggFWHoIBUgBBAG4AeQAgAHUAcwBlACAAbwBmACAAdABoAGkAcwAgAEMAZQBy\nAHQAaQBmAGkAYwBhAHQAZQAgAGMAbwBuAHMAdABpAHQAdQB0AGUAcwAgAGEAYwBj\nAGUAcAB0AGEAbgBjAGUAIABvAGYAIAB0AGgAZQAgAEQAaQBnAGkAQwBlAHIAdAAg\nAEMAUAAvAEMAUABTACAAYQBuAGQAIAB0AGgAZQAgAFIAZQBsAHkAaQBuAGcAIABQ\nAGEAcgB0AHkAIABBAGcAcgBlAGUAbQBlAG4AdAAgAHcAaABpAGMAaAAgAGwAaQBt\nAGkAdAAgAGwAaQBhAGIAaQBsAGkAdAB5ACAAYQBuAGQAIABhAHIAZQAgAGkAbgBj\nAG8AcgBwAG8AcgBhAHQAZQBkACAAaABlAHIAZQBpAG4AIABiAHkAIAByAGUAZgBl\nAHIAZQBuAGMAZQAuMBIGA1UdEwEB/wQIMAYBAf8CAQAwNAYIKwYBBQUHAQEEKDAm\nMCQGCCsGAQUFBzABhhhodHRwOi8vb2NzcC5kaWdpY2VydC5jb20wgY8GA1UdHwSB\nhzCBhDBAoD6gPIY6aHR0cDovL2NybDMuZGlnaWNlcnQuY29tL0RpZ2lDZXJ0SGln\naEFzc3VyYW5jZUVWUm9vdENBLmNybDBAoD6gPIY6aHR0cDovL2NybDQuZGlnaWNl\ncnQuY29tL0RpZ2lDZXJ0SGlnaEFzc3VyYW5jZUVWUm9vdENBLmNybDAfBgNVHSME\nGDAWgBSxPsNpA/i/RwHUmCYaCALvY2QrwzAdBgNVHQ4EFgQUUOpzidsp+xCPnuUB\nINTeeZlIg/cwDQYJKoZIhvcNAQEFBQADggEBAB7ipUiebNtTOA/vphoqrOIDQ+2a\nvD6OdRvw/S4iWawTwGHi5/rpmc2HCXVUKL9GYNy+USyS8xuRfDEIcOI3ucFbqL2j\nCwD7GhX9A61YasXHJJlIR0YxHpLvtF9ONMeQvzHB+LGEhtCcAarfilYGzjrpDq6X\ndF3XcZpCdF/ejUN83ulV7WkAywXgemFhM9EZTfkI7qA5xSU1tyvED7Ld8aW3DiTE\nJiiNeXf1L/BXunwH1OH8zVowV36GEEfdMR/X/KLCvzB8XSSq6PmuX2p0ws5rs0bY\nIb4p1I5eFdZCSucyb6Sxa1GDWL4/bcf72gMhy2oWGU4K8K2Eyl2Us1p292E=\n-----END CERTIFICATE-----'], u'method': u'SSL', u'prb_id': 1386, u'dst_port': u'443', u'dst_addr': u'38.229.72.16', u'ttc': 228.561, u'src_addr': u'103.247.138.60', u'group_id': 1443369, u'type': u'sslcert', u'msm_name': u'SSLCert', u'ver': u'3.0'}, {u'rt': 1161.992, u'msm_id': 1443369, u'from': u'193.162.21.10', u'dst_name': u'torproject.org', u'af': 4, u'timestamp': 1392727296, u'fw': 4600, u'cert': [u'-----BEGIN CERTIFICATE-----\nMIID/TCCAuWgAwIBAgIkNhPSsip1AJR2DEGtGdtSpPBb3qgBcuJXh2Gtln9+2aoo\nacTlMA0GCSqGSIb3DQEBBQUAMGAxCzAJBgNVBAYTAkVVMQswCQYDVQQIEwJFVTEL\nMAkGA1UEBxMCRVUxDDAKBgNVBAoTA0RTVjEMMAoGA1UECxMDRFNWMRswGQYDVQQD\nExJhcGFjLnByb3h5LmRzdi5jb20wHhcNMTMwMTI5MDAwMDAwWhcNMTYwNTAzMTIw\nMDAwWjByMQswCQYDVQQGEwJVUzEWMBQGA1UECBMNTWFzc2FjaHVzZXR0czEQMA4G\nA1UEBxMHV2FscG9sZTEeMBwGA1UEChMVVGhlIFRvciBQcm9qZWN0LCBJbmMuMRkw\nFwYDVQQDDBAqLnRvcnByb2plY3Qub3JnMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8A\nMIIBCgKCAQEAoGZlXATizI+cktME+tx2rjLndU2i0ATLGmzuhlxqwQ6RiFasuMkp\nfJcQp64tETk7mXxRWFGWVHiUFjCoeSQtO9Y0gxUk4mLTHPuh5nE8V3BzaDQ6sO9R\nPBXhVlTnbUatAitvgKM5PZnAgT+hi3ZcByGERBys/eITyCI/Uh9HZFW7Q2mUBcjr\n3/u36xeQdKMSH8hyq06I7HpRRwZpmv3T67jBQC6vo62rBnfGQ3vTcSVOmDoUOjD3\nGF1P/Ko7ch4Ao2P9/+T8/tNooJOPBK68VL168PTYRgG9Vyeh4hU2ipP/dd9DqoR+\nt3UwNO6o8jQmNSxdB/rjMcUtS7enSy2P7wIDAQABo4GMMIGJMB0GA1UdDgQWBBSx\n87Iq0fmAeNURYjYpnSG8riduZjArBgNVHREEJDAighAqLnRvcnByb2plY3Qub3Jn\ngg50b3Jwcm9qZWN0Lm9yZzAOBgNVHQ8BAf8EBAMCBaAwHQYDVR0lBBYwFAYIKwYB\nBQUHAwEGCCsGAQUFBwMCMAwGA1UdEwEB/wQCMAAwDQYJKoZIhvcNAQEFBQADggEB\nAHEvFwq+/VeBSa56orsVzq9Af34ZuTTST7zAUjfpDSYr/VjmHB4n0Uq80/cRWlm9\nDhRafM36IEk7tYpCN5tK4MYyMVg3FUESDsx37ZNDTpUiVSQPYjxpWQ7I/1kz7Sua\n6f/WQElnghZqa8Fj0s7zL/z6OxhDQMBjZvUzxtlsPhWjryo14COmQhsxW9b2uQA9\n14fPo4AcoFcmze/NAz4oMGyhsGKXwbkXFS5S2fW3ncSTxZBKJV+h8uslvB3owLvm\nhyqTwMk2iY8Z0jWDzR24jL76hPM3MWdFSSZzSkRLASEAcc/M/B50BcLfIFUSizNq\n4UoEZ4u6z2ZSChVJ3nHHaI8=\n-----END CERTIFICATE-----', u'-----BEGIN CERTIFICATE-----\nMIIF2jCCBMKgAwIBAgIKFA2gCwABAAZXojANBgkqhkiG9w0BAQUFADBIMRMwEQYK\nCZImiZPyLGQBGRYDQ09NMRMwEQYKCZImiZPyLGQBGRYDRFNWMRwwGgYDVQQDExNE\nU1YgRW50ZXJwcmlzZSBDQSAxMB4XDTEzMDMwNTA4MDIzMFoXDTE1MDMwNTA4MTIz\nMFowYDELMAkGA1UEBhMCRVUxCzAJBgNVBAgTAkVVMQswCQYDVQQHEwJFVTEMMAoG\nA1UEChMDRFNWMQwwCgYDVQQLEwNEU1YxGzAZBgNVBAMTEmFwYWMucHJveHkuZHN2\nLmNvbTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAM+mKjGl6lE+FEdk\nBOwxK4G0EdFOoFV1T43aCZHUgyVUJAz9Y3GiibGTcqrTqI6uMOnA6oxGGol28/JN\nr/ImPVThmOgcMuLJqvty9wbimHdpF4IYPZ2uHYApgJvhXoASQMfpsTxaCghHVreE\nvoa7p1G+crLp5PSZITyM4M2jcYucU7Q+V9ITBlEV1PaKB6jZIRDxuM+sBm9U6yVr\n/xVWBOLqTDw58ZmTtivqJwfn6G9OduKfjw1dK9xUCou8r9gBo+2iz9iQx+XuT27n\nYr2au4vHYJ0iwjX6d0QZVA4LS7fsyKlseY3mkPIA35GK0jaT35ym+KbQ2tjj2Uga\nYeB8mZ8CAwEAAaOCAqwwggKoMB0GA1UdDgQWBBQhlpFR3YPNERoEzLL7pAtqbCla\nnzAfBgNVHSMEGDAWgBTunBHm4QOkKHIfM0iu35w3p6lCcDCCAQsGA1UdHwSCAQIw\ngf8wgfyggfmggfaGgb5sZGFwOi8vL0NOPURTViUyMEVudGVycHJpc2UlMjBDQSUy\nMDEoMSksQ049STAwMTE1LENOPUNEUCxDTj1QdWJsaWMlMjBLZXklMjBTZXJ2aWNl\ncyxDTj1TZXJ2aWNlcyxDTj1Db25maWd1cmF0aW9uLERDPURTVixEQz1DT00/Y2Vy\ndGlmaWNhdGVSZXZvY2F0aW9uTGlzdD9iYXNlP29iamVjdENsYXNzPWNSTERpc3Ry\naWJ1dGlvblBvaW50hjNodHRwOi8vcGtpLmRzdi5jb20vRFNWJTIwRW50ZXJwcmlz\nZSUyMENBJTIwMSgxKS5jcmwwggEZBggrBgEFBQcBAQSCAQswggEHMIG0BggrBgEF\nBQcwAoaBp2xkYXA6Ly8vQ049RFNWJTIwRW50ZXJwcmlzZSUyMENBJTIwMSxDTj1B\nSUEsQ049UHVibGljJTIwS2V5JTIwU2VydmljZXMsQ049U2VydmljZXMsQ049Q29u\nZmlndXJhdGlvbixEQz1EU1YsREM9Q09NP2NBQ2VydGlmaWNhdGU/YmFzZT9vYmpl\nY3RDbGFzcz1jZXJ0aWZpY2F0aW9uQXV0aG9yaXR5ME4GCCsGAQUFBzAChkJodHRw\nOi8vcGtpLmRzdi5jb20vSTAwMTE1LmRzdi5jb21fRFNWJTIwRW50ZXJwcmlzZSUy\nMENBJTIwMSgxKS5jcnQwGQYJKwYBBAGCNxQCBAweCgBTAHUAYgBDAEEwDwYDVR0T\nAQH/BAUwAwEB/zAOBgNVHQ8BAf8EBAMCAYYwDQYJKoZIhvcNAQEFBQADggEBAJ+1\nbzf71F62JifL7uE08684cmT9sn5nmJLJjk5QEB6Fr+iCD9vafqG2XEjWbensyLhH\nPY/auJUSF7cdzu+l6jGFYvyjxM3AldQDlstzuhwcl6b0UgqOwfFTxkRzTCBobAUq\nkBgV60vXdJ2f29jPYk9qe0LUr8/01Yd8/f7JHkkjolmeBtGomXfNBhhxv7hW8/xN\neXtUTAE1yrbdiJhV5rGCTtWcAHDLzFo/sL4WbQPK5VqMzhUvJpf+zL2nyQ548kzk\nRLlSVBX+Jghb0Z9feRG+dCG5jr49farmtxI/J1BbCQHOinbtwufoW6skJUv9dzx4\nnzi9veZZknp8KIDEW6A=\n-----END CERTIFICATE-----'], u'method': u'SSL', u'prb_id': 2844, u'dst_port': u'443', u'dst_addr': u'38.229.72.16', u'ttc': 3.042, u'src_addr': u'145.218.88.254', u'group_id': 1443369, u'type': u'sslcert', u'msm_name': u'SSLCert', u'ver': u'3.0'}, {u'from': u'210.22.91.254', u'dst_name': u'torproject.org', u'err': u'connect: timeout', u'fw': 4570, u'timestamp': 1392727297, u'af': 4, u'dst_addr': u'4.36.66.178', u'prb_id': 3386, u'dst_port': u'443', u'group_id': 1443369, u'type': u'sslcert', u'msm_name': u'SSLCert', u'msm_id': 1443369}, {u'from': u'180.154.141.170', u'dst_name': u'torproject.org', u'err': u'connect: timeout', u'fw': 4600, u'timestamp': 1392727296, u'af': 4, u'dst_addr': u'209.145.54.50', u'prb_id': 3909, u'dst_port': u'443', u'group_id': 1443369, u'type': u'sslcert', u'msm_name': u'SSLCert', u'msm_id': 1443369}, {u'from': u'180.154.141.170', u'dst_name': u'torproject.org', u'err': u'connect: timeout', u'fw': 4600, u'timestamp': 1392727295, u'af': 4, u'dst_addr': u'216.234.179.13', u'prb_id': 3914, u'dst_port': u'443', u'group_id': 1443369, u'type': u'sslcert', u'msm_name': u'SSLCert', u'msm_id': 1443369}, {u'rt': 420.604, u'msm_id': 1443369, u'from': u'27.50.33.14', u'dst_name': u'torproject.org', u'af': 4, u'timestamp': 1392727295, u'fw': 4600, u'cert': [u'-----BEGIN CERTIFICATE-----\nMIIGujCCBaKgAwIBAgIQBt6X5R3DncJkjaxy3UEB/DANBgkqhkiG9w0BAQsFADBm\nMQswCQYDVQQGEwJVUzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMRkwFwYDVQQLExB3\nd3cuZGlnaWNlcnQuY29tMSUwIwYDVQQDExxEaWdpQ2VydCBIaWdoIEFzc3VyYW5j\nZSBDQS0zMB4XDTEzMDEyOTAwMDAwMFoXDTE2MDUwMzEyMDAwMFowcjELMAkGA1UE\nBhMCVVMxFjAUBgNVBAgTDU1hc3NhY2h1c2V0dHMxEDAOBgNVBAcTB1dhbHBvbGUx\nHjAcBgNVBAoTFVRoZSBUb3IgUHJvamVjdCwgSW5jLjEZMBcGA1UEAwwQKi50b3Jw\ncm9qZWN0Lm9yZzCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAN1oOe8B\n0kT0l6MXsIOWiBiXqLfGCk8nIeX+GXc0TNez14HBWPOzLMmA6Kfj3h9kJ0hLCzlS\nGui3xsT1ca5ZXONP/2beDkIoxwF+7/MCS8gOu4Cyua0CjR0ce6YWemKYVKxoqJvY\nH/S2UnzMHaBI/bhJ+QK5kMYg/JXoMx9IMIJnjl9clFt3TE34UR5/NZTsytXAtCjI\n5qMSpzKRE31RREGv1kxwTqJq/g5UFJWzZEwISDEhTeFTVOru0qjbEAqaip4hQH9D\nITjDOFw7Upgdab4TN4gLwDaZuo+Qcz+CQR6vCSlP2KziQAH9nlU+qT81eYVv+NOf\njogvdu/Atl/q+z0CAwEAAaOCA1YwggNSMB8GA1UdIwQYMBaAFFDqc4nbKfsQj57l\nASDU3nmZSIP3MB0GA1UdDgQWBBSx87Iq0fmAeNURYjYpnSG8riduZjArBgNVHREE\nJDAighAqLnRvcnByb2plY3Qub3Jngg50b3Jwcm9qZWN0Lm9yZzAOBgNVHQ8BAf8E\nBAMCBaAwHQYDVR0lBBYwFAYIKwYBBQUHAwEGCCsGAQUFBwMCMGEGA1UdHwRaMFgw\nKqAooCaGJGh0dHA6Ly9jcmwzLmRpZ2ljZXJ0LmNvbS9jYTMtZzE4LmNybDAqoCig\nJoYkaHR0cDovL2NybDQuZGlnaWNlcnQuY29tL2NhMy1nMTguY3JsMIIBxAYDVR0g\nBIIBuzCCAbcwggGzBglghkgBhv1sAQEwggGkMDoGCCsGAQUFBwIBFi5odHRwOi8v\nd3d3LmRpZ2ljZXJ0LmNvbS9zc2wtY3BzLXJlcG9zaXRvcnkuaHRtMIIBZAYIKwYB\nBQUHAgIwggFWHoIBUgBBAG4AeQAgAHUAcwBlACAAbwBmACAAdABoAGkAcwAgAEMA\nZQByAHQAaQBmAGkAYwBhAHQAZQAgAGMAbwBuAHMAdABpAHQAdQB0AGUAcwAgAGEA\nYwBjAGUAcAB0AGEAbgBjAGUAIABvAGYAIAB0AGgAZQAgAEQAaQBnAGkAQwBlAHIA\ndAAgAEMAUAAvAEMAUABTACAAYQBuAGQAIAB0AGgAZQAgAFIAZQBsAHkAaQBuAGcA\nIABQAGEAcgB0AHkAIABBAGcAcgBlAGUAbQBlAG4AdAAgAHcAaABpAGMAaAAgAGwA\naQBtAGkAdAAgAGwAaQBhAGIAaQBsAGkAdAB5ACAAYQBuAGQAIABhAHIAZQAgAGkA\nbgBjAG8AcgBwAG8AcgBhAHQAZQBkACAAaABlAHIAZQBpAG4AIABiAHkAIAByAGUA\nZgBlAHIAZQBuAGMAZQAuMHsGCCsGAQUFBwEBBG8wbTAkBggrBgEFBQcwAYYYaHR0\ncDovL29jc3AuZGlnaWNlcnQuY29tMEUGCCsGAQUFBzAChjlodHRwOi8vY2FjZXJ0\ncy5kaWdpY2VydC5jb20vRGlnaUNlcnRIaWdoQXNzdXJhbmNlQ0EtMy5jcnQwDAYD\nVR0TAQH/BAIwADANBgkqhkiG9w0BAQsFAAOCAQEAFfAsIxhBxzSVi5a9FpEp9JGc\n0wL5/4BVFv0lKYjHkRVoBdvN3gnAfGt2YXrAJZb7OCVwW3KFdSaTwm8T10eCVSXX\nASTrp6DWs6mHxw9HGIkVF9YESq6x5/ZGHDTovuRMCeHuIwn+nBL21z1WDqwozwcQ\nAxNXeRXJvXO4bOj301+26as9cOWjonGzkW9uc3WTWp89+YOpRo6RQ59Yc3UJlxjW\nHZR3Oqp/GM1jo2NPHnFeMpnFtVj+uuQBtNj7D7jiWhGtNxFIePizOBs8k+ao9lWO\nE2UHK5iM17YISRhBPNwi4YL+nf+jo5untE6WgvFYhEH2pwmCSKrIYBdGatbxfw==\n-----END CERTIFICATE-----', u'-----BEGIN CERTIFICATE-----\nMIIGWDCCBUCgAwIBAgIQCl8RTQNbF5EX0u/UA4w/OzANBgkqhkiG9w0BAQUFADBs\nMQswCQYDVQQGEwJVUzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMRkwFwYDVQQLExB3\nd3cuZGlnaWNlcnQuY29tMSswKQYDVQQDEyJEaWdpQ2VydCBIaWdoIEFzc3VyYW5j\nZSBFViBSb290IENBMB4XDTA4MDQwMjEyMDAwMFoXDTIyMDQwMzAwMDAwMFowZjEL\nMAkGA1UEBhMCVVMxFTATBgNVBAoTDERpZ2lDZXJ0IEluYzEZMBcGA1UECxMQd3d3\nLmRpZ2ljZXJ0LmNvbTElMCMGA1UEAxMcRGlnaUNlcnQgSGlnaCBBc3N1cmFuY2Ug\nQ0EtMzCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAL9hCikQH17+NDdR\nCPge+yLtYb4LDXBMUGMmdRW5QYiXtvCgFbsIYOBC6AUpEIc2iihlqO8xB3RtNpcv\nKEZmBMcqeSZ6mdWOw21PoF6tvD2Rwll7XjZswFPPAAgyPhBkWBATaccM7pxCUQD5\nBUTuJM56H+2MEb0SqPMV9Bx6MWkBG6fmXcCabH4JnudSREoQOiPkm7YDr6ictFuf\n1EutkozOtREqqjcYjbTCuNhcBoz4/yO9NV7UfD5+gw6RlgWYw7If48hl66l7XaAs\nzPw82W3tzPpLQ4zJ1LilYRyyQLYoEt+5+F/+07LJ7z20Hkt8HEyZNp496+ynaF4d\n32duXvsCAwEAAaOCAvowggL2MA4GA1UdDwEB/wQEAwIBhjCCAcYGA1UdIASCAb0w\nggG5MIIBtQYLYIZIAYb9bAEDAAIwggGkMDoGCCsGAQUFBwIBFi5odHRwOi8vd3d3\nLmRpZ2ljZXJ0LmNvbS9zc2wtY3BzLXJlcG9zaXRvcnkuaHRtMIIBZAYIKwYBBQUH\nAgIwggFWHoIBUgBBAG4AeQAgAHUAcwBlACAAbwBmACAAdABoAGkAcwAgAEMAZQBy\nAHQAaQBmAGkAYwBhAHQAZQAgAGMAbwBuAHMAdABpAHQAdQB0AGUAcwAgAGEAYwBj\nAGUAcAB0AGEAbgBjAGUAIABvAGYAIAB0AGgAZQAgAEQAaQBnAGkAQwBlAHIAdAAg\nAEMAUAAvAEMAUABTACAAYQBuAGQAIAB0AGgAZQAgAFIAZQBsAHkAaQBuAGcAIABQ\nAGEAcgB0AHkAIABBAGcAcgBlAGUAbQBlAG4AdAAgAHcAaABpAGMAaAAgAGwAaQBt\nAGkAdAAgAGwAaQBhAGIAaQBsAGkAdAB5ACAAYQBuAGQAIABhAHIAZQAgAGkAbgBj\nAG8AcgBwAG8AcgBhAHQAZQBkACAAaABlAHIAZQBpAG4AIABiAHkAIAByAGUAZgBl\nAHIAZQBuAGMAZQAuMBIGA1UdEwEB/wQIMAYBAf8CAQAwNAYIKwYBBQUHAQEEKDAm\nMCQGCCsGAQUFBzABhhhodHRwOi8vb2NzcC5kaWdpY2VydC5jb20wgY8GA1UdHwSB\nhzCBhDBAoD6gPIY6aHR0cDovL2NybDMuZGlnaWNlcnQuY29tL0RpZ2lDZXJ0SGln\naEFzc3VyYW5jZUVWUm9vdENBLmNybDBAoD6gPIY6aHR0cDovL2NybDQuZGlnaWNl\ncnQuY29tL0RpZ2lDZXJ0SGlnaEFzc3VyYW5jZUVWUm9vdENBLmNybDAfBgNVHSME\nGDAWgBSxPsNpA/i/RwHUmCYaCALvY2QrwzAdBgNVHQ4EFgQUUOpzidsp+xCPnuUB\nINTeeZlIg/cwDQYJKoZIhvcNAQEFBQADggEBAB7ipUiebNtTOA/vphoqrOIDQ+2a\nvD6OdRvw/S4iWawTwGHi5/rpmc2HCXVUKL9GYNy+USyS8xuRfDEIcOI3ucFbqL2j\nCwD7GhX9A61YasXHJJlIR0YxHpLvtF9ONMeQvzHB+LGEhtCcAarfilYGzjrpDq6X\ndF3XcZpCdF/ejUN83ulV7WkAywXgemFhM9EZTfkI7qA5xSU1tyvED7Ld8aW3DiTE\nJiiNeXf1L/BXunwH1OH8zVowV36GEEfdMR/X/KLCvzB8XSSq6PmuX2p0ws5rs0bY\nIb4p1I5eFdZCSucyb6Sxa1GDWL4/bcf72gMhy2oWGU4K8K2Eyl2Us1p292E=\n-----END CERTIFICATE-----'], u'method': u'SSL', u'prb_id': 400, u'dst_port': u'443', u'dst_addr': u'38.229.72.14', u'ttc': 203.67, u'src_addr': u'27.50.33.14', u'group_id': 1443369, u'type': u'sslcert', u'msm_name': u'SSLCert', u'ver': u'3.0'}, {u'dnserr': u'non-recoverable failure in name resolution (1)', u'from': u'211.68.70.39', u'dst_name': u'torproject.org', u'fw': 4600, u'timestamp': 1392727294, u'prb_id': 483, u'dst_port': u'443', u'group_id': 1443369, u'type': u'sslcert', u'msm_name': u'SSLCert', u'msm_id': 1443369} ] cls.probes = { 1003: CProbe(id=1003, meta_data={ "country_code": "GR", "asn_v4": 3333, "asn_v6": "4444"}), 1004: CProbe(id=1004, meta_data={ "country_code": "DE", "asn_v4": 3333, "asn_v6": "4444"}), 1008: CProbe(id=1008, meta_data={ "country_code": "DE", "asn_v4": 3332, "asn_v6": "4444"}), 1033: CProbe(id=1033, meta_data={ "country_code": "NL", "asn_v4": 3333, "asn_v6": "4444"}), 1038: CProbe(id=1038, meta_data={ "country_code": "GR", "asn_v4": 3333, "asn_v6": "4444"}), 1047: CProbe(id=1047, meta_data={ "country_code": "GR", "asn_v4": 3333, "asn_v6": "4444"}), 11672: CProbe(id=11572, meta_data={ "country_code": "GR", "asn_v4": 3333, "asn_v6": "4444"}), 11700: CProbe(id=11700, meta_data={ "country_code": "GR", "asn_v4": 3333, "asn_v6": "4444"}), 12203: CProbe(id=12203, meta_data={ "country_code": "GR", "asn_v4": 3333, "asn_v6": "4444"}), 12208: CProbe(id=12208, meta_data={ "country_code": "GR", "asn_v4": 3333, "asn_v6": "4444"}), 12214: CProbe(id=12214, meta_data={ "country_code": "GR", "asn_v4": 3333, "asn_v6": "4444"}), 13026: CProbe(id=13026, meta_data={ "country_code": "GR", "asn_v4": 3333, "asn_v6": "4444"}), 13377: CProbe(id=13377, meta_data={ "country_code": "GR", "asn_v4": 3333, "asn_v6": "4444"}), 1386: CProbe(id=1386, meta_data={ "country_code": "GR", "asn_v4": 3333, "asn_v6": "4444"}), 2844: CProbe(id=2844, meta_data={ "country_code": "GR", "asn_v4": 3333, "asn_v6": "4444"}), 3386: CProbe(id=3386, meta_data={ "country_code": "GR", "asn_v4": 3333, "asn_v6": "4444"}), 3909: CProbe(id=3909, meta_data={ "country_code": "GR", "asn_v4": 3333, "asn_v6": "4444"}), 3914: CProbe(id=3914, meta_data={ "country_code": "GR", "asn_v4": 3333, "asn_v6": "4444"}), 400: CProbe(id=400, meta_data={ "country_code": "GR", "asn_v4": 3333, "asn_v6": "4444"}), 483: CProbe(id=483, meta_data={ "country_code": "GR", "asn_v4": 3333, "asn_v6": "4444"}), } def test_additional(self): """Tests whole functionality of additional unit.""" expected_output = ( "Certificate:\n" " Issuer: C=US, O=DigiCert Inc, CN=DigiCert High Assurance CA-3\n" " Subject: C=US, O=The Tor Project, Inc., CN=*.torproject.org\n" " SHA256 Fingerprint=36:13:D2:B2:2A:75:00:94:76:0C:41:AD:19:DB:52:A4:F0:5B:DE:A8:01:72:E2:57:87:61:AD:96:7F:7E:D9:AA\n\n" " Seen by 11 probes\n\n" "Certificate:\n" " Issuer: C=US, O=DigiCert Inc, CN=DigiCert High Assurance EV Root CA\n" " Subject: C=US, O=DigiCert Inc, CN=DigiCert High Assurance CA-3\n" " SHA256 Fingerprint=21:EB:37:AB:4C:F6:EF:89:65:EC:17:66:40:9C:A7:6B:8B:2E:03:F2:D1:A3:88:DF:73:42:08:E8:6D:EE:E6:79\n\n" " Seen by 11 probes\n\n" "Certificate:\n" " Issuer: C=None, O=None, CN=DSV Enterprise CA 1\n" " Subject: C=EU, O=DSV, CN=apac.proxy.dsv.com\n" " SHA256 Fingerprint=1A:B8:9E:ED:1B:DD:A0:E2:EA:67:89:C1:C5:4B:20:1C:49:9D:74:27:B0:5D:11:F2:9A:5F:C1:0D:F9:18:48:DA\n\n" " Seen by 1 probe\n\n" " Below the threshold (80%)\n" " Probes that saw it: \n" " ID: 2844, country code: GR, ASN (v4/v6): 3333/4444\n" "Certificate:\n" " Issuer: C=EU, O=DSV, CN=apac.proxy.dsv.com\n" " Subject: C=US, O=The Tor Project, Inc., CN=*.torproject.org\n" " SHA256 Fingerprint=07:52:BE:65:72:BF:02:D4:C9:E2:93:09:A8:E0:BE:3A:EA:D4:30:41:B8:49:FA:C5:F2:12:33:07:37:57:EE:C7\n\n" " Seen by 1 probe\n\n" " Below the threshold (80%)\n" " Probes that saw it: \n" " ID: 2844, country code: GR, ASN (v4/v6): 3333/4444\n" ) path = 'ripe.atlas.tools.helpers.rendering.Probe.get_many' with mock.patch(path) as mock_get_many: mock_get_many.return_value = self.probes.values() self.assertEquals( set(Renderer().additional(SaganSet(self.results)).split("\n")), set(expected_output.split("\n")) ) def test_gather_unique_certs(self): """Test gathering of the unique certs in sagans set""" expected_certs = { '1A:B8:9E:ED:1B:DD:A0:E2:EA:67:89:C1:C5:4B:20:1C:49:9D:74:27:B0:5D:11:F2:9A:5F:C1:0D:F9:18:48:DA': {'cnt': 1, 'probes': [self.probes[2844]]}, '36:13:D2:B2:2A:75:00:94:76:0C:41:AD:19:DB:52:A4:F0:5B:DE:A8:01:72:E2:57:87:61:AD:96:7F:7E:D9:AA': { 'cnt': 11, 'probes': [self.probes[1003], self.probes[1004], self.probes[1033], self.probes[1038], self.probes[1047], self.probes[12203], self.probes[12208], self.probes[13026], self.probes[13377], self.probes[1386], self.probes[400]] }, '21:EB:37:AB:4C:F6:EF:89:65:EC:17:66:40:9C:A7:6B:8B:2E:03:F2:D1:A3:88:DF:73:42:08:E8:6D:EE:E6:79': { 'cnt': 11, 'probes': [self.probes[1003], self.probes[1004], self.probes[1033], self.probes[1038], self.probes[1047], self.probes[12203], self.probes[12208], self.probes[13026], self.probes[13377], self.probes[1386], self.probes[400]] }, '07:52:BE:65:72:BF:02:D4:C9:E2:93:09:A8:E0:BE:3A:EA:D4:30:41:B8:49:FA:C5:F2:12:33:07:37:57:EE:C7': {'cnt': 1, 'probes': [self.probes[2844]]} } path = 'ripe.atlas.tools.helpers.rendering.Probe.get_many' with mock.patch(path) as mock_get_many: mock_get_many.return_value = self.probes.values() sagans = SaganSet(self.results) renderer = Renderer() renderer.gather_unique_certs(sagans) # remove cert key to make easier to test for k in renderer.uniqcerts: renderer.uniqcerts[k].pop("cert") self.assertEquals(renderer.uniqcerts, expected_certs) def test_bucketize_result_cert(self): """Tests bucketize of a single sagan result""" expected_certs = { '36:13:D2:B2:2A:75:00:94:76:0C:41:AD:19:DB:52:A4:F0:5B:DE:A8:01:72:E2:57:87:61:AD:96:7F:7E:D9:AA': {'cnt': 1, 'probes': [self.probes[1003]]}, '21:EB:37:AB:4C:F6:EF:89:65:EC:17:66:40:9C:A7:6B:8B:2E:03:F2:D1:A3:88:DF:73:42:08:E8:6D:EE:E6:79': {'cnt': 1, 'probes': [self.probes[1003]]} } path = 'ripe.atlas.tools.helpers.rendering.Probe.get_many' with mock.patch(path) as mock_get_many: mock_get_many.return_value = self.probes.values() sagans = SaganSet(self.results) renderer = Renderer() renderer.bucketize_result_cert(list(sagans)[0]) # remove cert key to make easier to test for k in renderer.uniqcerts: renderer.uniqcerts[k].pop("cert") self.assertEquals(renderer.uniqcerts, expected_certs) def test_get_nprobes_ofpopular_cert(self): """Tests fetching the number of probes for the most popular cert""" path = 'ripe.atlas.tools.helpers.rendering.Probe.get_many' with mock.patch(path) as mock_get_many: mock_get_many.return_value = self.probes.values() sagans = SaganSet(self.results) renderer = Renderer() renderer.gather_unique_certs(sagans) self.assertEquals(renderer.get_nprobes_ofpopular_cert(), 11) def test_render_certificate(self): """Tests rendering of single certificate.""" expected_output = ( "Certificate:\n" " Issuer: C=US, O=DigiCert Inc, CN=DigiCert High Assurance CA-3\n" " Subject: C=US, O=The Tor Project, Inc., CN=*.torproject.org\n" " SHA256 Fingerprint=36:13:D2:B2:2A:75:00:94:76:0C:41:AD:19:DB:52:A4:F0:5B:DE:A8:01:72:E2:57:87:61:AD:96:7F:7E:D9:AA\n\n" " Seen by 11 probes\n" ) path = 'ripe.atlas.tools.helpers.rendering.Probe.get_many' with mock.patch(path) as mock_get_many: mock_get_many.return_value = self.probes.values() sagans = SaganSet(self.results) renderer = Renderer() renderer.gather_unique_certs(sagans) self.assertEquals( renderer.render_certificate("36:13:D2:B2:2A:75:00:94:76:0C:41:AD:19:DB:52:A4:F0:5B:DE:A8:01:72:E2:57:87:61:AD:96:7F:7E:D9:AA"), expected_output ) def test_render_below_thershold(self): """Tests rendering string for certs that are seen for below threshold probes.""" expected_output = [ " Below the threshold (80%)", " Probes that saw it: ", " ID: 2844, country code: GR, ASN (v4/v6): 3333/4444" ] path = 'ripe.atlas.tools.helpers.rendering.Probe.get_many' with mock.patch(path) as mock_get_many: mock_get_many.return_value = self.probes.values() sagans = SaganSet(self.results) renderer = Renderer() renderer.gather_unique_certs(sagans) self.assertEquals( renderer.render_below_threshold("07:52:BE:65:72:BF:02:D4:C9:E2:93:09:A8:E0:BE:3A:EA:D4:30:41:B8:49:FA:C5:F2:12:33:07:37:57:EE:C7"), expected_output ) def test_report_with_ssl_consistency_renderer(self): """Tests the report with the ssl renderer.""" results = self.results[:2] expected_output = ( "Certificate:\n" " Issuer: C=US, O=DigiCert Inc, CN=DigiCert High Assurance CA-3\n" " Subject: C=US, O=The Tor Project, Inc., CN=*.torproject.org\n" " SHA256 Fingerprint=36:13:D2:B2:2A:75:00:94:76:0C:41:AD:19:DB:52:A4:F0:5B:DE:A8:01:72:E2:57:87:61:AD:96:7F:7E:D9:AA\n\n" " Seen by 2 probes\n\n" "Certificate:\n" " Issuer: C=US, O=DigiCert Inc, CN=DigiCert High Assurance EV Root CA\n" " Subject: C=US, O=DigiCert Inc, CN=DigiCert High Assurance CA-3\n" " SHA256 Fingerprint=21:EB:37:AB:4C:F6:EF:89:65:EC:17:66:40:9C:A7:6B:8B:2E:03:F2:D1:A3:88:DF:73:42:08:E8:6D:EE:E6:79\n\n" " Seen by 2 probes\n\n" ) with capture_sys_output() as (stdout, stderr): path = 'ripe.atlas.cousteau.AtlasRequest.get' with mock.patch(path) as mock_get: mock_get.side_effect = [ (True, {"creation_time": 1, "start_time": 1, "type": {"name": "sslcert"}, "description": ""}), (True, results) ] mpath = 'ripe.atlas.tools.helpers.rendering.Probe.get_many' with mock.patch(mpath) as mock_get_many: mock_get_many.return_value = [self.probes[1003], self.probes[1004]] cmd = Command() cmd.init_args(["1", "--renderer", "ssl_consistency"]) cmd.run() expected_set = set(expected_output.split("\n")) returned_set = set(stdout.getvalue().split("\n")) self.assertEquals(returned_set, expected_set) ripe-atlas-tools-1.2.2/tests/test_filters.py000066400000000000000000000151451264541652500211560ustar00rootroot00000000000000# Copyright (c) 2015 RIPE NCC # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . import unittest from ripe.atlas.sagan import Result from ripe.atlas.cousteau import Probe from ripe.atlas.tools.exceptions import RipeAtlasToolsException from ripe.atlas.tools.filters import ( FilterFactory, Filter, ASNFilter, filter_results ) class TestFilterFactory(unittest.TestCase): def test_factory_create(self): """Test factory create.""" self.assertIsInstance(FilterFactory.create("country", "GR"), Filter) self.assertIsInstance(FilterFactory.create("asn", "3333"), ASNFilter) class TestFilter(unittest.TestCase): def test_filter(self): """Tests filter method of general Filter class.""" result = {'af': 4, 'prb_id': 1216, 'result': [{'rtt': 27.429}, {'rtt': 25.672}, {'rtt': 25.681}], 'ttl': 54, 'avg': 26.2606666667, 'size': 20, 'from': '109.190.83.40', ' proto': 'ICMP', 'timestamp': 1445025400, 'dup': 0, 'type': 'ping', 'sent': 3, 'msm_id': 1000192, 'fw': 4700, 'max': 27.429, 'step': 360, 'src_addr': '192.168.103.132', 'rcvd': 3, 'msm_name': 'Ping', 'lts': 377, 'dst_name': 'hsi.cablecom.ch', 'min': 25.672, 'dst_addr': '62.2.16.24'} sagan_result = Result.get( result, on_error=Result.ACTION_IGNORE, on_warning=Result.ACTION_IGNORE ) sagan_result.probe = Probe( id=1216, meta_data={"country_code": "GR", "asn_v4": 3337, "asn_v6": "4445"} ) self.assertTrue(Filter("country_code", "GR").filter(sagan_result)) self.assertFalse(Filter("country_code", "NL").filter(sagan_result)) self.assertFalse(Filter("asn_v4", 3336).filter(sagan_result)) self.assertTrue(Filter("asn_v6", "4445").filter(sagan_result)) with self.assertRaises(RipeAtlasToolsException): self.assertTrue(Filter("country", "GR").filter(sagan_result)) class TestASNFilter(unittest.TestCase): def test_filter(self): """Tests filter method of probe's asn filter class.""" result = {'af': 4, 'prb_id': 1216, 'result': [{'rtt': 27.429}, {'rtt': 25.672}, {'rtt': 25.681}], 'ttl': 54, 'avg': 26.2606666667, 'size': 20, 'from': '109.190.83.40', ' proto': 'ICMP', 'timestamp': 1445025400, 'dup': 0, 'type': 'ping', 'sent': 3, 'msm_id': 1000192, 'fw': 4700, 'max': 27.429, 'step': 360, 'src_addr': '192.168.103.132', 'rcvd': 3, 'msm_name': 'Ping', 'lts': 377, 'dst_name': 'hsi.cablecom.ch', 'min': 25.672, 'dst_addr': '62.2.16.24'} sagan_result = Result.get( result, on_error=Result.ACTION_IGNORE, on_warning=Result.ACTION_IGNORE ) sagan_result.probe = Probe( id=1216, meta_data={"country_code": "GR", "asn_v4": 3337, "asn_v6": "4445"} ) self.assertTrue(ASNFilter(3337).filter(sagan_result)) self.assertFalse(ASNFilter(3336).filter(sagan_result)) self.assertTrue(ASNFilter("4445").filter(sagan_result)) self.assertFalse(ASNFilter(4445).filter(sagan_result)) class TestFilterResults(unittest.TestCase): def setUp(self): result = {'af': 4, 'prb_id': 1216, 'result': [{'rtt': 27.429}, {'rtt': 25.672}, {'rtt': 25.681}], 'ttl': 54, 'avg': 26.2606666667, 'size': 20, 'from': '109.190.83.40', ' proto': 'ICMP', 'timestamp': 1445025400, 'dup': 0, 'type': 'ping', 'sent': 3, 'msm_id': 1000192, 'fw': 4700, 'max': 27.429, 'step': 360, 'src_addr': '192.168.103.132', 'rcvd': 3, 'msm_name': 'Ping', 'lts': 377, 'dst_name': 'hsi.cablecom.ch', 'min': 25.672, 'dst_addr': '62.2.16.24'} probes = [ Probe( id=1216, meta_data={"country_code": "GR", "asn_v4": 3337, "asn_v6": "4445"} ), Probe( id=121, meta_data={"country_code": "GR", "asn_v4": 3338, "asn_v6": "4445"} ), Probe( id=12, meta_data={"country_code": "DE", "asn_v4": 3339, "asn_v6": 3337} ), Probe( id=1, meta_data={"country_code": "NL", "asn_v4": 3337, "asn_v6": "4446"} ) ] self.sagan_results = [] for probe in probes: sagan_result = Result.get( result, on_error=Result.ACTION_IGNORE, on_warning=Result.ACTION_IGNORE ) sagan_result.probe = probe self.sagan_results.append(sagan_result) def test_filter_results1(self): """Tests filter results where we have 3 results match with 2 filters.""" expected_results = [ self.sagan_results[0], self.sagan_results[2], self.sagan_results[3] ] filters = [ FilterFactory.create("asn", 3337), FilterFactory.create("country_code", "NL") ] self.assertEqual( filter_results(filters, self.sagan_results), expected_results ) def test_filter_results2(self): """Tests filter results where we have no results match with multiple filters.""" expected_results = [] filters = [ FilterFactory.create("asn", 3336), FilterFactory.create("country_code", "DK"), FilterFactory.create("asn_v6", "3") ] self.assertEqual( filter_results(filters, self.sagan_results), expected_results ) def test_filter_results3(self): """Tests filter results where we have success from one of the filters.""" expected_results = [self.sagan_results[3]] filters = [ FilterFactory.create("asn", 3336), FilterFactory.create("country_code", "NL") ] self.assertEqual( filter_results(filters, self.sagan_results), expected_results ) def test_filter_results4(self): """Tests filter results where we have success from both of the filters.""" expected_results = [self.sagan_results[3]] filters = [ FilterFactory.create("asn_v6", "4446"), FilterFactory.create("country_code", "NL") ] self.assertEqual( filter_results(filters, self.sagan_results), expected_results ) ripe-atlas-tools-1.2.2/tests/test_ipdetails.py000066400000000000000000000311131264541652500214550ustar00rootroot00000000000000# Copyright (c) 2015 RIPE NCC # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see . import mock import unittest import requests from ripe.atlas.tools.ipdetails import IP class FakeResponse(object): def __init__(self, json_return={}, ok=True): self.json_return = json_return self.ok = ok self.text = "testing" def json(self): return self.json_return class FakeErrorResponse(FakeResponse): def json(self): raise ValueError("json breaks") class TestIPDetails(unittest.TestCase): IP = '193.0.6.1' ASN = '3333' HOLDER = 'RIPE-NCC-AS Reseaux IP Europeens Network Coordination Centre (RIPE NCC),NL' SAME_PREFIX_IP = '193.0.6.2' PREFIX = '193.0.0.0/21' SAME_AS_DIFFERENT_PREFIX_IP = '193.0.22.1' NOT_ANNOUNCED_IP = '80.81.192.1' MOCK_RESULTS = { IP: {"status": "ok", "server_id": "stat-app2", "cached": False, "status_code": 200, "time": "2015-10-12T15:30:00.113317", "messages": [["warning", "Given resource is not announced but result has been aligned to first-level less-specific (193.0.0.0/21)."]], "version": "1.3", "data_call_status": "supported - connecting to ursa", "see_also": [], "process_time": 561, "query_id": "196d2754-70f6-11e5-b8ba-782bcb346712", "data": {"query_time": "2015-10-12T08:00:00", "is_less_specific": True, "resource": "193.0.0.0/21", "actual_num_related": 0, "num_filtered_out": 0, "asns": [{"holder": "RIPE-NCC-AS Reseaux IP Europeens Network Coordination Centre (RIPE NCC),NL", "asn": 3333}], "announced": True, "related_prefixes": [], "type": "prefix", "block": {"resource": "193.0.0.0/8", "name": "IANA IPv4 Address Space Registry", "desc": "RIPE NCC (Status: ALLOCATED)"}}}, SAME_AS_DIFFERENT_PREFIX_IP: {"status": "ok", "server_id": "stat-app2", "cached": False, "status_code": 200, "time": "2015-10-12T15:32:25.778643", "messages": [["warning", "Given resource is not announced but result has been aligned to first-level less-specific (193.0.22.0/23)."]], "version": "1.3", "data_call_status": "supported - connecting to ursa", "see_also": [], "process_time": 818, "query_id": "7018b6cc-70f6-11e5-8bf8-782bcb346712", "data": {"query_time": "2015-10-12T08:00:00", "is_less_specific": True, "resource": "193.0.22.0/23", "actual_num_related": 0, "num_filtered_out": 0, "asns": [{"holder": "RIPE-NCC-AS Reseaux IP Europeens Network Coordination Centre (RIPE NCC),NL", "asn": 3333}], "announced": True, "related_prefixes": [], "type": "prefix", "block": {"resource": "193.0.0.0/8", "name": "IANA IPv4 Address Space Registry", "desc": "RIPE NCC (Status: ALLOCATED)"}}}, NOT_ANNOUNCED_IP: {"status": "ok", "server_id": "stat-app2", "cached": False, "status_code": 200, "time": "2015-10-12T15:33:58.911309", "messages": [["info", "2 routes were filtered due to low visibility (min peers:3)."]], "version": "1.3", "data_call_status": "supported - connecting to ursa", "see_also": [], "process_time": 462, "query_id": "a7d1daee-70f6-11e5-aaec-782bcb346712", "data": {"query_time": "2015-10-12T08:00:00", "is_less_specific": False, "resource": "80.81.192.1", "actual_num_related": 0, "num_filtered_out": 2, "asns": [], "announced": False, "related_prefixes": [], "type": "prefix", "block": {"resource": "80.0.0.0/8", "name": "IANA IPv4 Address Space Registry", "desc": "RIPE NCC (Status: ALLOCATED)"}}} } def setUp(self): # the poor man's fake cache self.db = {} def db_get(k): return self.db.get(k) def db_set(k, v, e): self.db[k] = v def db_keys(): return self.db.keys() self.mock_cache = mock.patch( "ripe.atlas.tools.ipdetails.cache" ).start() self.mock_cache.get.side_effect = db_get self.mock_cache.set.side_effect = db_set self.mock_cache.keys.side_effect = db_keys self.mock_get = mock.patch( 'ripe.atlas.tools.ipdetails.requests.get' ).start() self.mock_get.return_value = FakeResponse( json_return=self.MOCK_RESULTS[self.IP] ) def tearDown(self): mock.patch.stopall() def test_loopback4(self): """IPv4 loopback address""" det = IP("127.0.0.1") self.assertEquals(det.asn, None) self.assertEquals(det.is_querable(), False) # no query to stat self.assertEquals(self.mock_get.call_count, 0) # no access to cache get self.assertEquals(self.mock_cache.get.call_count, 0) def test_loopback6(self): """IPv6 loopback address""" det = IP("::1") self.assertEquals(det.asn, None) self.assertEquals(det.is_querable(), False) # no query to stat self.assertEquals(self.mock_get.call_count, 0) # no access to cache get self.assertEquals(self.mock_cache.get.call_count, 0) def test_nocache(self): """No cache""" det = IP(self.IP) self.assertEquals(det.asn, self.ASN) self.assertEquals(det.prefix, self.PREFIX) self.assertEquals(det.holder, self.HOLDER) # query to stat self.assertEquals(self.mock_get.call_count, 1) # access to cache get self.assertEquals(self.mock_cache.get.call_count, 1) # access to cache set self.assertEquals(self.mock_cache.set.call_count, 2) def test_fakecache_sameip(self): """Fake cache, same IP""" det = IP(self.IP) det = IP(self.IP) self.assertEquals(det.asn, self.ASN) # query to stat self.assertEquals(self.mock_get.call_count, 1) # access to cache get self.assertEquals(self.mock_cache.get.call_count, 2) # access to cache set self.assertEquals(self.mock_cache.set.call_count, 2) def test_fakecache_sameprefix(self): """Fake cache, same prefix""" det1 = IP(self.IP) det2 = IP(self.SAME_PREFIX_IP) self.assertEquals(det1.asn, self.ASN) self.assertEquals(det2.asn, det1.asn) # query to stat self.assertEquals(self.mock_get.call_count, 1) # access to cache get self.assertEquals(self.mock_cache.get.call_count, 3) # access to cache set self.assertEquals(self.mock_cache.set.call_count, 3) def test_fakecache_diffprefix(self): """Fake cache, same AS, different prefix""" self.mock_get.return_value = FakeResponse( json_return=self.MOCK_RESULTS[self.IP] ) det1 = IP(self.IP) self.mock_get.return_value = FakeResponse( json_return=self.MOCK_RESULTS[self.SAME_AS_DIFFERENT_PREFIX_IP] ) det2 = IP(self.SAME_AS_DIFFERENT_PREFIX_IP) self.assertEquals(det1.asn, self.ASN) self.assertEquals(det2.asn, det1.asn) self.assertEquals(self.mock_get.call_count, 2) # access to cache get self.assertEquals(self.mock_cache.get.call_count, 3) # access to cache set self.assertEquals(self.mock_cache.set.call_count, 4) def test_fakecache_notannounced(self): """Fake cache, IP not announced""" self.mock_get.return_value = FakeResponse( json_return=self.MOCK_RESULTS[self.NOT_ANNOUNCED_IP] ) det = IP(self.NOT_ANNOUNCED_IP) self.assertEquals(det.asn, None) self.assertEquals(self.mock_get.call_count, 1) IP(self.NOT_ANNOUNCED_IP) # now it should be cached self.assertEquals(det.asn, None) self.assertEquals(self.mock_get.call_count, 2) def test_valid_query_stat(self): """Test case for valid stat response""" ip = IP(self.IP) self.assertEqual( ip.query_stat(), { 'Prefix': '193.0.0.0/21', 'Holder': 'RIPE-NCC-AS Reseaux IP Europeens Network Coordination Centre (RIPE NCC),NL', 'ASN': '3333' } ) def test_invalid_query_stat(self): """Test case where stat returns not ok status""" self.mock_get.return_value = FakeResponse( json_return={"status": "notok"} ) ip = IP(self.IP) self.assertEqual(ip.query_stat(), {}) self.mock_get.return_value = FakeResponse( json_return={} ) ip = IP(self.IP) self.assertEqual(ip.query_stat(), {}) def test_invalid_query_stat1(self): """Test case where stat returns not valid data structure""" # no data at all self.mock_get.return_value = FakeResponse( json_return={"status": "ok"} ) ip = IP(self.IP) self.assertEqual(ip.query_stat(), {}) # data is dict but no asns key self.mock_get.return_value = FakeResponse( json_return={"status": "ok", "data": {"bla": "bla"}} ) ip = IP(self.IP) self.assertEqual(ip.query_stat(), {}) # data is not dict self.mock_get.return_value = FakeResponse( json_return={"status": "ok", "data": [1, 2]} ) ip = IP(self.IP) self.assertEqual(ip.query_stat(), {}) # asns is not list self.mock_get.return_value = FakeResponse( json_return={"status": "ok", "data": {"asns": {}}} ) ip = IP(self.IP) self.assertEqual(ip.query_stat(), {}) # asns is empty self.mock_get.return_value = FakeResponse( json_return={"status": "ok", "data": {"asns": [{}]}} ) ip = IP(self.IP) self.assertEqual(ip.query_stat(), {}) def test_invalid_query_stat2(self): """Test case where stat returns exception""" self.mock_get.side_effect = requests.exceptions.RequestException ip = IP(self.IP) self.assertEqual(ip.query_stat(), {}) self.mock_get.return_value = FakeErrorResponse() ip = IP(self.IP) self.assertEqual(ip.query_stat(), {}) def test_update_cache(self): """Test case where we store both prefix/address""" details = { 'Prefix': '193.0.0.0/21', 'Holder': 'RIPE-NCC-AS Reseaux IP Europeens Network Coordination Centre (RIPE NCC),NL', 'ASN': '3333' } IP(self.IP) self.assertEquals(self.mock_cache.set.call_count, 2) self.assertEquals(self.mock_cache.get("IPDetails:193.0.6.1"), details) self.assertEquals(self.mock_cache.get( "IPDetailsPrefix:193.0.0.0/21"), details ) def test_update_cache1(self): """Test case where we store only address""" details = { 'Prefix': '193.0.0.0/21', 'Holder': 'RIPE-NCC-AS Reseaux IP Europeens Network Coordination Centre (RIPE NCC),NL', 'ASN': '3333' } self.mock_cache.set("IPDetailsPrefix:193.0.0.0/21", details, 1) IP(self.IP) # we already called it above, so it should be 2 by bow self.assertEquals(self.mock_cache.set.call_count, 2) self.assertEquals(self.mock_cache.get("IPDetails:193.0.6.1"), details) self.assertEquals(self.mock_cache.get("IPDetailsPrefix:193.0.0.0/21"), details) def test_get_from_cache_prefix(self): """Test case where we have a matching prefix in cache""" details = { 'Prefix': '193.0.0.0/21', 'Holder': 'test', 'ASN': 'test' } self.mock_cache.set("IPDetailsPrefix:193.0.0.0/20", details, 1) ip = IP(self.IP) self.assertTrue(ip.cached_prefix_found) self.assertEquals(ip.asn, "test") self.assertEquals(ip.holder, "test") self.assertEquals(ip.get_from_cached_prefix(), details) def test_get_from_cache_prefix1(self): """Test case where we dont' have a matching prefix in cache""" ip = IP(self.IP) # clear out db to test specific function self.db = {} self.assertFalse(ip.cached_prefix_found) self.assertEquals(ip.get_from_cached_prefix(), None) def test_is_querable(self): """Test case where IP is quearable""" ip = IP(self.IP) self.assertTrue(ip.is_querable()) def test_is_querable1(self): """Test case where IP is not quearable""" ip = IP("127.0.0.1") self.assertFalse(ip.is_querable()) ripe-atlas-tools-1.2.2/tox.ini000066400000000000000000000011331264541652500162360ustar00rootroot00000000000000[tox] envlist=py27-pyflakes, py27-pep8, py27-coverage, py27, py34, py34-syntax [testenv] commands=python setup.py test [] deps=nose [testenv:py27-pep8] commands=pep8 ripe --max-line-length=100 basepython=python2.7 deps=pep8 [testenv:py27-pyflakes] commands=pyflakes ripe basepython=python2.7 deps=pyflakes [testenv:py27-coverage] basepython=python2.7 commands= coverage run --source=ripe setup.py test coverage report -m deps=coverage [testenv:py34-syntax] whitelist_externals=bash commands=bash -c "find ripe/ tests/ -name "*.py" | xargs python -m py_compile" basepython=python3.4 deps=